User Guide for Sesame 2.0-rc2


Table of Contents

1. Sesame 2: what's new?
2. Source code and binaries
3. Short introduction to Sesame's components
4. Setting up to use the Sesame libraries
4.1. Downloading the libraries
4.2. Logging: SLF4J initialization
5. Application directory configuration
6. Server software installation
6.1. Required software
6.2. Sesame server installation
6.3. Logging Configuration
6.4. Repository Configuration
7. Sesame Console
7.1. Getting started
7.2. Connecting to a set of repositories
7.3. Repository list
7.4. Creating a repository
7.5. Other commands
7.6. Repository configuration
7.6.1. Memory store configuration
7.6.2. Native store configuration
7.6.3. HTTP repository configuration
8. The Repository API
8.1. Creating a Repository object
8.1.1. Creating a main memory RDF Repository
8.1.2. Creating a Native RDF Repository
8.1.3. Creating a repository with RDF Schema inferencing
8.1.4. Accessing a remote repository
8.2. Using a repository: RepositoryConnections
8.2.1. Adding RDF to a repository
8.2.2. Querying a repository
8.2.3. Creating, retrieving, removing individual statements
8.2.4. Working with Collections and Iterations
8.2.5. Using context
8.2.6. Transactions
9. The SeRQL query language (revision 3.0)
9.1. Revisions
9.1.1. revision 1.1
9.1.2. revision 1.2
9.1.3. revision 2.0
9.1.4. revision 3.0
9.2. Introduction
9.3. URIs, literals and variables
9.3.1. Variables
9.3.2. URIs
9.3.3. Literals
9.3.4. Blank Nodes (R1.2)
9.4. Path expressions
9.4.1. Basic path expressions
9.4.2. Path expression short cuts
9.4.3. Optional path expressions
9.5. Select- and construct queries
9.6. Select queries
9.7. Construct queries
9.8. The WHERE clause
9.8.1. Boolean constants
9.8.2. Value (in)equality
9.8.3. Numerical comparisons
9.8.4. bound() (R3.0)
9.8.5. isURI() and isBNode() (R1.2)
9.8.6. The LIKE operator (R1.2)
9.8.7. AND, OR, NOT
9.8.8. Nested WHERE clauses (R1.2)
9.9. Other functions
9.9.1. label(), lang() and datatype()
9.9.2. namespace() and localName() (R1.2)
9.10. The LIMIT and OFFSET clauses
9.11. The USING NAMESPACE clause
9.12. Built-in predicates (REVISED in R2.0)
9.13. Set combinatory operations
9.13.1. UNION (REVISED in R3.0)
9.13.2. INTERSECT (R1.2)
9.13.3. MINUS (R1.2)
9.14. Query Nesting
9.14.1. The IN operator (R1.2)
9.14.2. ANY and ALL (R1.2)
9.14.3. EXISTS (R1.2)
9.15. Querying context (R2.0)
9.16. Example SeRQL queries
9.16.1. Query 1
9.16.2. Query 2
9.16.3. Query 3
9.17. References
9.18. SeRQL grammar

List of Figures

3.1. A high-level overview of Sesame's most prominent components and their dependencies
9.1. A basic path expression
9.2. Multi-value nodes
9.3. Multi-value nodes in a longer path expression
9.4. Branches in a path expression
9.5. Branches in a longer path expression
9.6. A reification path expression
9.7. Path expression for query 1
9.8. Path expression for query 2
9.9. Path expression for query 3

List of Tables

9.1. Default namespaces

Chapter 1. Sesame 2: what's new?

For those who have previously worked with Sesame 1.x: Sesame 2 is an important step forward from the Sesame 1.x series, building on the feedback that has been gathered during the past years. A lot of improvements have been made to various APIs as well as the framework as a whole.

Unfortunately, because of all the changes to APIs and the like, Sesame 2 is not backwards compatible with earlier releases. Therefore, we suggest you read through this user guide before starting to port any applications to Sesame 2. Please also note that Sesame 2 is currently in beta stage, meaning that although the core APIs are now stable, many features are not yet implemented (for example, the custom inferencer and the RDBMS backend are not yet available).

So what's new in Sesame 2? Well, in short:

  • Sesame 2 is completely targeted at Java 5. All APIs use Java 5 features such as typed collections and iterators.
  • Revised Repository API that is much more targeted at the use of Sesame as a library.
  • Support for context/provenance, allowing you to keep track of individual RDF data units (like files, for instance).
  • Proper transaction/rollback support.
  • A REST-ful HTTP protocol that includes support for the SPARQL protocol and SPARQL Query Results XML Format, which are currently both Last Call Working Drafts from the W3C.
  • Support for the SPARQL Query Language.

A lot of these new features will be covered in the next chapters.

Chapter 2. Source code and binaries

If you follow the download links from the download section at openRDF.org you'll find that there are a number of different files that you can download. Which one you need depends on what you want to do with it:

  • openrdf-sesame-(version)-sdk.tar.gz. This is a GNU-zipped tar archive of the complete binary version of the Sesame SDK. It includes all the Sesame libraries packaged as jar files and a set of Web Archive (.war) files for easy deployment of Sesame's Web applications (see Chapter 6, Server software installation). It also includes documentation (such as this user manual, and the API JavaDoc).
  • openrdf-sesame-(version)-sdk.zip. This is a zip archive that has the same contents as the tar.gz file mentioned before.
  • openrdf-sesame-(version)-onejar.jar. This is a Java Archive (.jar) file containing all the relevant Sesame libraries. The main purpose of this jar file is easy inclusion of all Sesame components when using it as an embedded component/library in your own application: you only need to add this one jar file to your classpath and you can start programming against it.

Apart from these binary downloads, Aduna also offers the Sesame libraries as maven artifacts in a maven repository. This maven repository is located at http://repository.aduna-software.org/maven2/. Developers using maven can use this repository to automatically resolve library dependencies and to get the latest released versions of each library. For instructions how to use the maven build system, see http://maven.apache.org/.

Sesame's source code is available from Aduna's Subversion (SVN) repository: https://src.aduna-software.org/svn/org.openrdf/. There is an SVN Web Viewer available as well at http://src.aduna-software.org/viewvc/org.openrdf/. Source code for each release is located in the releases directory, the development version of the code is in the projects directory.

Chapter 3. Short introduction to Sesame's components

Before diving into the internals of Sesame, we will start with a short introduction to Sesame by giving a high-level overview of its components. It's important to have some basic knowledge about this as the rest of this document will often refer to various components that are touched upon here. It is assumed that the reader has at least some basic knowledge about RDF, RDF Schema, OWL, etc. If this is not the case, some introductory articles can be found at the following locations:

We will try to explain the Sesame framework using the following figure, which shows the most prominent components and APIs in Sesame and how they are built on top of each other. Each component/API depends on the components/APIs that are beneath them.

Figure 3.1. A high-level overview of Sesame's most prominent components and their dependencies

A high-level overview of Sesame's most prominent components and their dependencies

All the way at the bottom of the diagram is the RDF Model, the foundation of the Sesame framework. Being an RDF-oriented framework, all parts of Sesame are to some extent dependent on this RDF model, which defines interfaces and implementation for all basic RDF entities: URI, blank node, literal and statement.

Rio, which stands for "RDF I/O", consists of a set of parsers and writers for various RDF file formats. The parsers can be used to translate RDF files to lists of statements, and the writers for the reverse operation. Rio can also be used independent of the rest of Sesame.

The Storage And Inference Layer (Sail) API is a low level System API (SPI) for RDF stores and inferencers. Its purpose is to abstract from the storage and inference details, allowing various types of storage and inference to be used. The Sail API is mainly of interest for those who are developing Sail implementations, for all others it suffices to know how to create and configure one. There are several implementations of the Sail API, for example the MemoryStore which stores RDF data in main memory, and the NativeStore which uses dedicated on-disk data structures for storage.

The Repository API is a higer level API that offers a large number of developer-oriented methods for handling RDF data. The main goal of this API is to make the life of application developers as easy as possible. It offers various methods for uploading data files, querying, and extracting and manipulating data. There are several implementations of this API, the ones shown in this figure are the SailRepository and the HTTPRepository. The former translates calls to a Sail implementation of choice, the latter offers transparent client-server communication with a Sesame server over HTTP.

The top-most component in the diagram is the HTTP Server. The HTTP Server consists of a number of Java Servlets that implement a protocol for accessing Sesame repositories over HTTP. The details of this protocol can be found in Sesame's system documentation, but most people can simply use a client library to handle the communication. The HTTPClient that is used by the HTTPRepository is one such library.

While each part of the Sesame code is publicly available and extensible, most application developers will be primarily interested in the Repository API. This API is described in more detail in one of the following chapters.

Chapter 4. Setting up to use the Sesame libraries

In this chapter, we explain some basics about setting up your application development environment to work with Sesame. In Chapter 8, The Repository API we go into details of the use of the APIs. If you do not want to program against the Sesame libraries but just want to install and run the Sesame HTTP server, please skip ahead to Chapter 6, Server software installation.

4.1. Downloading the libraries

As was explained in Chapter 2, Source code and binaries, various download options are available to you. The quickest way to get started with using the Sesame libraries is to download the Sesame onejar library and include it in your classpath.

However, it is important to note that the Sesame Framework consists of a set of libraries: Sesame is not a monolithic piece of software, you can pick and choose which parts you want and which ones you don't. In those cases where you don't care about picking and choosing and just want to get on with it, the onejar is a good choice. If, however, you want a little more control over what is included, you can download the complete SDK and select (from the lib directory) those sesame libraries that you require.

An alternative to picking libraries by hand is to use maven. Apache Maven is a software management tool that helps you by offering things like library version management and dependency management (which is very useful because it means that once you decide you need a particular Sesame library, maven automatically downloads all the libraries that your library of choice requires in turn), and giving you a handy-dandy build environment. For details on how to start using maven, we advise you to take a look at the Apache Maven website at http://maven.apache.org/.

4.2. Logging: SLF4J initialization

Before you begin using the Sesame libraries, one important configuration step needs to be taken: the initialization and configuration of a logging framework.

Sesame uses the Simple Logging Facade for Java (SLF4J), which is a framework for abstracting from the actual logging implementation. SLF4J allows you, as a user of the Sesame framework, to plug in your own favorite logging implementation at deployment time. SLF4J supports the most popular logging implementations such as Java Logging, Apache Commons Logging, log4j, etc. See the SLF4J website for more info.

What you need to do is to determine/decide which logging implementation you (are going to) use and include the appropriate SLF4J-bridge jar-file in your classpath. For example, if you decide to use Apache log4j 1.2, you need to include the SLF4J-to-log4j bridge in your classpath. The SLF4J release packages include bridges to various logging implementations; just download the SLF4J release package (Sesame currently uses 1.3.0) and include the appropriate bridge in your classpath; slf4j-log4j12-1.3.0.jar, for example. That's all!

Chapter 5. Application directory configuration

In this chapter, we explain how to change the default data directory for OpenRDF applications. You can skip this chapter if you only use Sesame as a library, or if you consider the defaults to be fine.

All OpenRDF applications (Sesame Server, Workbench, and Console) store configuration files and repository data in a single directory (with subdirectories). On Windows machines, this directory is %APPDATA%\Aduna\ by default, where %APPDATA% is the application data directory of the user that runs the application. For example, in case the application runs under the 'LocalService' user account on Windows XP, the directory is C:\Documents and Settings\LocalService\Application Data\Aduna\. On Linux/UNIX, the default location is $HOME/.aduna/, for example /home/tomcat/.aduna/. We will refer to this data directory as [ADUNA_DATA] in the rest of this manual.

The location of this data directory can be reconfigured using the Java system property info.aduna.platform.appdata.basedir. When you are using Tomcat as the servlet container then you can set this property using the JAVA_OPTS parameter, for example:

  • set JAVA_OPTS='-Dinfo.aduna.platform.appdata.basedir=\path\to\other\dir\' (on Windows)
  • export JAVA_OPTS='-Dinfo.aduna.platform.appdata.basedir=/path/to/other/dir/' (on Linux/UNIX)

If you are using Apache Tomcat as a Windows Service you should use the Windows Services configuration tool to set this property. Other users can either edit the Tomcat startup script or set the property some other way.

Chapter 6. Server software installation

In this section, we explain how you can install a Sesame HTTP Server. You can skip this if you are not planning to run a Sesame server but intend to use Sesame as a library to program against.

6.1. Required software

The Sesame server software requires the following software:

  • Java 5 or newer
  • A Java Servlet Container that supports Java Servlet API 2.4 and Java Server Pages (JSP) 2.0, or newer. We recommend using a recent, stable version of Apache Tomcat. At the time of writing, this is either version 5.5.x or 6.x.

6.2. Sesame server installation

The Sesame 2.0 server software comes in the form of two Java Web Applications: Sesame (HTTP) server and OpenRDF Workbench.

Sesame Server provides HTTP access to Sesame repositories and is meant to be accessed by other applications. Apart from some functionality to view the server's log messages, it doesn't provide any user oriented functionality. Instead, the user oriented functionality is part of OpenRDF Workbench. OpenRDF Workbench provides a web interface for querying, updating and exploring the repositories of a Sesame Server.

If you have not done so already, you will first need to download the Sesame 2.0 SDK. Both Sesame Server and OpenRDF Workbench can be found in the war directory of the SDK. The war-files in this directory need to be deployed in a Java Servlet Container (see Section 6.1, “Required software”). The deployment process is container-specific, please consult the documentation for your container on how to deploy a web application.

After you have deployed the Sesame Server webapp, you should be able to access it, by default, at path /openrdf-sesame. You can point your browser at this location to verify that the deployment succeeded. Your browser should show the Sesame welcome screen as well as some options to view the server logs, among other things. Similarly, after deployment, the OpenRDF Workbench should be available at path /openrdf-workbench.

6.3. Logging Configuration

Both Sesame Server and OpenRDF Workbench use the Apache log4j logging framework. In its default configuration, all Sesame Server log messages are sent to the log file [ADUNA_DATA]/OpenRDF Sesame/logs/main.log (and log messages for the Workbench to the same file in [ADUNA_DATA]/OpenRDF workbench). See Chapter 5, Application directory configuration for more info about data directories.

The default log level is INFO, indicating that only important status messages, warnings and errors are logged. The log level can be adjusted by modifying the .../logs/log4j.properties .../logs/log4j.properties file. To set the log level to, for example, DEBUG level, modify the line setting the log level for the org.openrdf classes:

log4j.logger.org.openrdf=DEBUG

For other logging configuration options, please consult the log4j documentation.

6.4. Repository Configuration

A clean installation of a Sesame Server has a single repository by default: the SYSTEM repository. This SYSTEM repository contains all configuration data for the server, including data on which other repositories exists and (in future releases) the access rights on these repositories. This SYSTEM repository should not be used to store data that is not related to the server configuration.

For now, the easiest way to create and manage repositories in a SYSTEM repository is to use the Sesame Console. The Sesame Console is a command-line application for interacting with Sesame, a UI-version of this application is currently being developed. See Chapter 7, Sesame Console.

Chapter 7. Sesame Console

This chapter describes Sesame Console, a command-line application for interacting with Sesame, a UI-version of this application is currently being developed. For now, the easiest way to create and manage repositories in a SYSTEM repository is to use the Sesame Console.

7.1. Getting started

Sesame Console can be started using the start-console.bat/.sh scripts that can be found in the bin directory of the Sesame SDK. By default, the console will connect to the "default data directory", which contains the console's own set of repositories. See Chapter 5, Application directory configuration for more info on data directories.

The console can be operated by typing commands. Commands can span multiple lines and end with a '.' at the end of a line. For example, to get an overview of the available commands, type:

help.

To get help for a specific command, type 'help' followed by the command name, e.g.:

help connect.

7.2. Connecting to a set of repositories

As indicating in the previous section, the console connects to its own set of repositories by default. Using the connect command you can make the console connect to a Sesame Server or to a set of repositories of your file system. For example, to connect to a Sesame Server running on localhost, enter the following command:

connect http://localhost:8080/openrdf-sesame.

7.3. Repository list

To get an overview of the repositories that are available in the set that your console is connected to, use the 'show' command:

show repositories.

7.4. Creating a repository

The 'create' command can be used to add new repositories to the set that the console is connected to. This command expects the name of a template that describes the repository's configuration. Currently, there are seven templates that are included with the console by default:

  • "memory", a memory based RDF repository
  • "memory-rdfs", a main-memory repository with RDF Schema inferencing
  • "memory-rdfs-dt", a main-memory repository with RDF Schema and direct type hierarchy inferencing
  • "native", a repository that uses on-disk data structure
  • "native-rdfs", a native repository with RDF Schema inferencing
  • "native-rdfs-dt", a native repository with RDF Schema and direct type hierarchy inferencing
  • "remote", a repository that serves as a proxy for a repository on a Sesame Server

When the 'create' command is executed, the console will then ask you to fill in a number of parameters for the type of repository that you chose. For example, to create a native repository, you execute the following command:

create native.

The console will then ask you to provide an ID and title for the repository, as well as the triple indexes that need to be created for this kind of store. The values between square brackets indicate default values which you can select by simply hitting enter. The output of this dialogue looks something like this:

Please specify values for the following variables:
Repository ID [native]: myRepo
Repository title [Native store]: My repository
Triple indexes [spoc,posc]: 
Repository created

Please see Section 7.6, “Repository configuration” for more info on the repository configuration options.

7.5. Other commands

Please check the documentation that is provided by the console itself for help on how to use the other commands. Most commands should be self explanatory.

7.6. Repository configuration

7.6.1. Memory store configuration

A memory store is an RDF repository that stores its data in main memory. Apart from the standard ID and title parameters, this type of repository has a Persist and Sync delay parameter.

7.6.1.1. Memory Store persistence

The Persist parameter controls whether the memory store will use a data file for persistence over sessions. Persistent memory stores write their data to disk before being shut down and read this data back in the next time they are initialized. Non-persistent memory stores are always empty upon initialization.

7.6.1.2. Synchronization delay

By default, the memory store persistence mechanism synchronizes the disk backup directly upon any change to the contents of the store. That means that directly after an update operation (upload, removal) completes, the disk backup is updated. It is possible to configure a synchronization delay however. This can be useful if your application performs several transactions in sequence and you want to prevent disk synchronization in the middle of this sequence to improve update performance.

The synchronization delay is specified by a number, indicating the time in milliseconds that the store will wait before it synchronizes changes to disk. The value 0 indicates that there should be no delay. Negative values can be used to postpone the synchronization indefinitely, i.e. until the store is shut down.

7.6.2. Native store configuration

A native store stores and retrieves its data directly to/from disk. The advantage of this over the memory store is that it scales much better as it isn't limited to the size of available memory. Of course, since it has to access the disk, it is also slower than the in-memory store, but it is a good solution for larger data sets.

7.6.2.1. Native store indexes

The native store uses on-disk indexes to speed up querying. It uses B-Trees for indexing statements, where the index key consists of four fields: subject (s), predicate (p), object (o) and context (c). The order in which each of these fields is used in the key determines the usability of an index on a specify statement query pattern: searching statements with a specific subject in an index that has the subject as the first field is signifantly faster than searching these same statements in an index where the subject field is second or third. In the worst case, the 'wrong' statement pattern will result in a sequential scan over the entire set of statements.

By default, the native repository only uses two indexes, one with a subject-predicate-object-context (spoc) key pattern and one with a predicate-object-subject-context (posc) key pattern. However, it is possible to define more or other indexes for the native repository, using the Triple indexes parameter. This can be used to optimize performance for query patterns that occur frequently.

The subject, predicate, object and context fields are represented by the characters 's', 'p', 'o' and 'c' respectively. Indexes can be specified by creating 4-letter words from these four characters. Multiple indexes can be specified by separating these words with commas, spaces and/or tabs. For example, the string "spoc, posc" specifies two indexes; a subject-predicate-object-context index and a predicate-object-subject-context index.

Creating more indexes potentially speeds up querying (a lot), but also adds overhead for maintaining the indexes. Also, every added index takes up additional disk space.

The native store automatically creates/drops indexes upon (re)initialization, so the parameter can be adjusted and upon the first refresh of the configuration the native store will change its indexing strategy, without loss of data.

7.6.3. HTTP repository configuration

An HTTP repository isn't an actual store, but serves as a proxy for a store on a (remote) Sesame server. Apart from the standard ID and title parameters, this type of repository has a Sesame server location and a Remote repository ID parameter.

7.6.3.1. Sesame server location

This parameter specifies the URL of the Sesame Server that the repository should communicate with. Default value is http://localhost:8080/openrdf-sesame, which corresponds to a Sesame Server that is running on your own machine.

7.6.3.2. Remote repository ID

This is the ID of the remote repository that the HTTP repository should communicate with. Please note an HTTP repository two repository ID parameters: one identifying the remote repository and one that specifies the HTTP repository's own ID.

Chapter 8. The Repository API

The Repository API is the central access point for Sesame repositories. Its purpose is to give a developer-friendly access point to RDF repositories, offering various methods for querying and updating the data, while hiding a lot of the nitty gritty details of the underlying machinery.

In this chapter, we will try to explain the basics of how to program against the Repository API. The interfaces for the Repository API can be found in package org.openrdf.repository. Several implementations for these interface exist in various sub-packages. The Javadoc reference for the API is available online and can also be found in the doc directory of the download.

If you need more information about how to set up your environment for working with the Sesame APIs, take a look at Chapter 4, Setting up to use the Sesame libraries.

8.1. Creating a Repository object

The first step in any action that involves Sesame repositories is to create a Repository for it. Repository objects operate on (stacks of) Sail object(s) for storage and retrieval of RDF data. An important thing to remember is that the behaviour of a repository is determined by the Sail(s) that it operates on; for example, the repository will only support RDF Schema or OWL semantics if the Sail stack includes an inferencer for this.

The central interface of the repository API is the Repository interface. There are several implementations available of this interface:

  • org.openrdf.repository.sail.SailRepository is Repository that operates directly on top of a Sail. This is the class most commonly used when accessing a local Sesame repository.
  • org.openrdf.repository.http.HTTPRepository is, as the name implies, a Repository implementation that acts as a proxy to a Sesame repository available on a remote Sesame server, accessible through HTTP.

In the following section, we will first take a look at the use of the SailRepository class in order to create and use a local Sesame repository.

8.1.1. Creating a main memory RDF Repository

One of the simplest configurations is a repository that just stores RDF data in main memory without applying any inferencing or whatsoever. This is also by far the fastest type of repository that can be used. The following code creates and initialize a non-inferencing main-memory repository:

import org.openrdf.repository.Repository;
import org.openrdf.repository.sail.SailRepository;
import org.openrdf.sail.memory.MemoryStore;


...

Repository myRepository = new SailRepository(new MemoryStore());
myRepository.initialize();

The constructor of the SailRepository class accepts any object of type Sail, so we simply pass it a new main-memory store object (which is, of course, a Sail implementation). Following this, the repository needs to be initialized to prepare the Sail(s) that it operates on, which includes operations such as restoring previously stored data, setting up connections to a relational database, etc.

The repository that is created by the above code is volatile: its contents are lost when the object is garbage collected or when the program is shut down. This is fine for cases where, for example, the repository is used as a means for manipulating an RDF model in memory.

Different types of Sail objects take parameters in their constructor that change their behaviour. The MemoryStore for example takes a data directory parameter that specifies a data directory for persisent storage. If specified, the MemoryStore will write its contents to this directory so that it can restore it when it is initialized in a future session:

File dataDir = new File("c:\\temp\\myRepository\\");
Repository myRepository = new SailRepository( new MemoryStore(dataDir) );
myRepository.initialize();

As you can see, we can fine-tune the configuration of our repository by passing parameters to the constructor of the Sail object. Some Sail types may offer additional configuration methods, all of which need to be called before the repository is initialized. The MemoryStore currently has one such method: setSyncDelay(long), which can be used to control the strategy that is used for writing to the data file, e.g.:

File dataDir = new File("c:\\temp\\myRepository\\");
MemoryStore memStore = new MemoryStore(dataDir);
memStore.setSyncDelay(1000L);

Repository myRepository = new SailRepository(memStore);
myRepository.initialize();

8.1.2. Creating a Native RDF Repository

A Native RDF Repository does not keep its data in main memory, but instead stores it directly to disk (in a binary format optimized for compact storage and fast retrieval). It is an efficient, scalable and fast solution for RDF storage of datasets that are too large to keep entirely in memory.

The code for creation of a Native RDF repository is almost identical to that of a main memory repository:

import org.openrdf.repository.Repository;
import org.openrdf.repository.sail.SailRepository;
import org.openrdf.sail.nativerdf.NativeStore;


...
File dataDir = new File("/path/to/datadir/");
Repository myRepository = new SailRepository(new NativeStore(dataDir));
myRepository.initialize();

By default, the Native store creates a set of two indexes (see Section 7.6.2, “Native store configuration”). To configure which indexes it should create, we can either use the NativeStore.setTripleIndexes(String) method, or we can directly supply a index configuration string to the constructor:

import org.openrdf.repository.Repository;
import org.openrdf.repository.sail.SailRepository;
import org.openrdf.sail.nativerdf.NativeStore;


...
File dataDir = new File("/path/to/datadir/");
String indexes = "spoc,posc,cosp";
Repository myRepository = new SailRepository(new NativeStore(dataDir, indexes));
myRepository.initialize();

8.1.3. Creating a repository with RDF Schema inferencing

As we have seen, we can create Repository objects for any kind of back-end store by passing them a reference to the appropriate Sail object. We can pass any stack of Sails this way, allowing all kinds of repository configurations to be created quite easily. For example, to stack an RDF Schema inferencer on top of a memory store, we simply create a repository like so:

import org.openrdf.repository.Repository;
import org.openrdf.repository.sail.SailRepository;
import org.openrdf.sail.memory.MemoryStore;
import org.openrdf.sail.inferencer.fc.ForwardChainingRDFSInferencer;

...

Repository myRepository = new SailRepository(
                          new ForwardChainingRDFSInferencer(
                          new MemoryStore()));
myRepository.initialize();

Each layer in the Sail stack is created by a constructor that takes the underlying Sail as a parameter. Finally, we create the SailRepository object as a functional wrapper around the Sail stack.

The ForwardChainingRDFSInferencer that is used in this example is a generic RDF Schema inferencer; it can be used on top of any Sail that supports the methods it requires. Both MemoryStore and NativeStore support these methods.

8.1.4. Accessing a remote repository

Working with remote repositories is just as easy as working with local ones. We can simply use a different Repository object, the HTTPRepository, instead of the SailRepository class.

A requirement is of course that there is a Sesame 2 server running on some remote system, which is accessible over HTTP. For example, suppose that at http://example.org/sesame2/ a Sesame server is running, which has a repository with the identification 'example-db'. We can access this repository in our code as follows:

import org.openrdf.repository.Repository;
import org.openrdf.repository.http.HTTPRepository;

...

String sesameServer = "http://example.org/sesame2";
String repositoryID = "example-db";

Repository myRepository = new HTTPRepository(sesameServer, repositoryID);
myRepository.initialize();

8.2. Using a repository: RepositoryConnections

Now that we have created a Repository, we want to do something with it. In Sesame 2, this is achieved through the use of RepositoryConnection objects, which can be created by the Repository.

A RepositoryConnection represents - as the name suggests - a connection to the actual store. We can issue operations over this connection, and close it when we are done to make sure we are not keeping resources unnnecessarily occupied.

In the following sections, we will show some examples of basic operations.

8.2.1. Adding RDF to a repository

The Repository API offers various methods for adding data to a repository. Data can be added by specifying the location of a file that contains RDF data, and statements can be added individually or in collections.

We perform operations on a repository by requesting a RepositoryConnection from the repository. On this RepositoryConnection object we can the various operations, such as query evaluation, getting, adding, or removing statements, etc.

The following example code adds two files, one local and one available through HTTP, to a repository:

import org.openrdf.OpenRDFException;
import org.openrdf.repository.Repository;
import org.openrdf.repository.RepositoryConnection;
import org.openrdf.rio.RDFFormat;
import java.io.File;
import java.net.URL;

...

File file = new File("/path/to/example.rdf");
String baseURI = "http://example.org/example/local";

try {
   RepositoryConnection con = myRepository.getConnection();
   try {
      con.add(file, baseURI, RDFFormat.RDFXML);

      URL url = new URL("http://example.org/example/remote");
      con.add(url, url.toString(), RDFFormat.RDFXML);
   }
   finally {
      con.close();
   }
}
catch (OpenRDFException e) {
   // handle exception
}
catch (java.io.IOEXception e) {
   // handle io exception
}

More information on other available methods can be found in the javadoc reference of the RepositoryConnection interface.

8.2.2. Querying a repository

The Repository API has a number of methods for creating and evaluating queries. Three types of queries are distinguished: tuple queries, graph queries and boolean queries. The query types differ in the type of results that they produce.

The result of a tuple query is a set of tuples (or variable bindings), where each tuple represents a solution of a query. This type of query is commonly used to get specific values (URIs, blank nodes, literals) from the stored RDF data.

The result of graph queries is an RDF graph (or set of statements). This type of query is very useful for extracting sub-graphs from the stored RDF data, which can then be queried further, serialized to an RDF document, etc.

The result of boolean queries is a simple boolean value, i.e. true of false. This type of query can be used to check if a repository contains specific information.

Note: Sesame 2 currently supports two query languages: SeRQL and SPARQL. The former is explained in Chapter 9, The SeRQL query language (revision 3.0), the specification for the latter is available online.

8.2.2.1. Evaluating a tuple query

To evaluate a tuple query we simply do the following:

import java.util.List;
import org.openrdf.OpenRDFException;
import org.openrdf.repository.RepositoryConnection;
import org.openrdf.query.TupleQuery;
import org.openrdf.query.TupleQueryResult;
import org.openrdf.query.BindingSet;
import org.openrdf.query.QueryLanguage;

...

try {
   RepositoryConnection con = myRepository.getConnection();
   try {
      String queryString = "SELECT x, y FROM {x} p {y}";
      TupleQuery tupleQuery = con.prepareTupleQuery(QueryLanguage.SERQL, queryString);
      TupleQueryResult result = tupleQuery.evaluate();
      try {
         .... // do something with the result
      }
      finally {
         result.close();
      }
   }
   finally {
      con.close();
   }
}
catch (OpenRDFException e) {
   // handle exception
}

This evaluates a SeRQL query and returns a TupleQueryResult, which consists of a sequence of BindingSet objects. Each BindingSet contains a set of Binding objects. A binding is pair relating a name (as used in the query's SELECT clause) with a value.

We can use the TupleQueryResult to iterate over all results and get each individual result for x and y:

while (result.hasNext()) {
   BindingSet bindingSet = result.next();
   Value valueOfX = bindingSet.getValue("x");
   Value valueOfY = bindingSet.getValue("y");

   // do something interesting with the values here...
}

As you can see, we retrieve values by name rather than by an index. The names used should be the names of variables as specified in your query. The TupleQueryResult.getBindingNames() method returns a list of binding names, in the order in which they were specified in the query. To process the bindings in each binding set in the order specified by the projection, you can do the following:

List<String> bindingNames = result.getBindingNames();
while (result.hasNext()) {
   BindingSet bindingSet = result.next();
   Value firstValue = bindingSet.getValue(bindingNames.get(0));
   Value secondValue = bindingSet.getValue(bindingNames.get(1));

   // do something interesting with the values here...
}

It is important to invoke the close() operation on the TupleQueryResult, after we are done with it. A TupleQueryResult evaluates lazily and keeps resources (such as connections to the underlying database) open. Closing the TupleQueryResult frees up these resources. Do not forget that iterating over a result may cause exceptions! The best way to make sure no connections are kept open unnecessarily is to invoke close() in the finally clause.

An alternative to producing a TupleQueryResult is to supply an object that implements the TupleQueryResultHandler interface to the query's evaluate() method. The main difference is that when using a return object, the caller has control over when the next answer is retrieved, whereas with the use of a handler, the connection simply pushes answers to the handler object as soon as it has them available.

As an example we will use SPARQLResultsXMLWriter, which is a TupleQueryResultHandler implementation that writes SPARQL Results XML documents to an output stream or to a writer:

import org.openrdf.query.resultio.sparqlxml.SPARQLResultsXMLWriter;

...

FileOutputStream out = new FileOutputStream("/path/to/result.srx");
try {
   SPARQLResultsXMLWriter sparqlWriter = new SPARQLResultsXMLWriter(out);

   RepositoryConnection con = myRepository.getConnection();
   try {
      String queryString = "SELECT * FROM {x} p {y}";
      TupleQuery tupleQuery = con.prepareTupleQuery(QueryLanguage.SERQL, queryString);
      tupleQuery.evaluate(sparqlWriter);
   }
   finally {
      con.close();
   }
}
finally {
   out.close();
}

You can just as easily supply your own application-specific implementation of TupleQueryResultHandler though.

Lastly, an important warning: as soon as you are done with the RepositoryConnection object, you should close it. Notice that during processing of the TupleQueryResult object (for example, when iterating over its contents), the RepositoryConnection should still be open. We can invoke con.close() after we have finished with the result.

8.2.2.2. Evaluating a graph query

The following code evaluates a graph query on a repository:

import org.openrdf.query.GraphQueryResult;

GraphQueryResult graphResult = con.prepareGraphQuery(
      QueryLanguage.SERQL, "CONSTRUCT * FROM {x} p {y}").evaluate();

A GraphQueryResult is similar to TupleQueryResult in that is an object that iterates over the query results. However, for graph queries the query results are RDF statements, so a GraphQueryResult iterates over Statement objects:

while (graphResult.hasNext()) {
   Statement st = graphResult.next();
   // ... do something with the resulting statement here.
}

The TupleQueryResultHandler equivalent for graph queries is org.openrdf.rio.RDFHandler. Again, this is a generic interface, each object implementing it can process the reported RDF statements in any way it wants.

All writers from Rio (such as the RDFXMLWriter, TurtleWriter, TriXWriter, etc.) implement the RDFHandler interface. This allows them to be used in combination with querying quite easily. In the following example, we use a TurtleWriter to write the result of a SeRQL graph query to standard output in Turtle format:

import org.openrdf.rio.turtle.TurtleWriter;

...

RepositoryConnection con = myRepository.getConnection();
try {
   TurtleWriter turtleWriter = new TurtleWriter(System.out);

   con.prepareGraphQuery(QueryLanguage.SERQL,
         "CONSTRUCT * FROM {x} p {y}").evaluate(turtleWriter);
}
finally {
   con.close();
}

Again, note that as soon as we are done with the result of the query (either after iterating over the contents of the GraphQueryResult or after invoking the RDFHandler), we invoke con.close() to close the connection and free resources.

8.2.2.3. Preparing and Reusing Queries

In the previous sections we have simply created a query from a string and immediately evaluated it. However, the prepareTupleQuery and prepareGraphQuery methods return objects of type Query, specifically TupleQuery and GraphQuery.

A Query object, once created, can be (re)used. For example, we can evaluate a Query object, then add some data to our repository, and evaluate the same query again.

The Query object also has a setBinding method, which can be used to specify specific values for query variables. As a simple example, suppose we have a repository containing names and e-mail addresses of people, and we want to do a query for each person, retrieve his/her e-mail address, for example, but we want to do a separate query for each person. This can be achieved using the setBinding functionality, as follows:

RepositoryConnection con = myRepository.getConnection();

try {
   // First, prepare a query that retrieves all names of persons
   TupleQuery nameQuery = con.prepareTupleQuery(QueryLanguage.SERQL,
         "SELECT name FROM {person} ex:name {name}");

   // Then, prepare another query that retrieves all e-mail addresses of persons:
   TupleQuery mailQuery = con.prepareTupleQuery(QueryLanguage.SERQL,
         "SELECT mail FROM {person} ex:mail {mail}; ex:name {name}");

   // Evaluate the first query to get all names
   TupleQueryResult nameResult = nameQuery.evaluate();
   try {
      // Loop over all names, and retrieve the corresponding e-mail address.
      while (nameResult.hasNext()) {
         BindingSet bindingSet = nameResult.next();
         Value name = bindingSet.get("name");

         // Retrieve the matching mailbox, by setting the binding for
         // the variable 'name' to the retrieved value:
         mailQuery.setBinding("name", name);

         TupleQueryResult mailResult = mailQuery.evaluate();

         // mailResult now contains the e-mail addresses for one particular person
         try {
            ....
         }
         finally {
            // after we are done, close the result
            mailResult.close();
         }
      }
   }
   finally {
      nameResult.close();
   }
}
finally {
   con.close();
}

The values with which you perform the setBinding operation of course do not necessarily have to come from a previous query result (as they do in the above example). Using a ValueFactory you can create your own value objects. You can use this functionality to, for example, query for a particular keyword that is given by user input:

ValueFactory factory = myRepository.getValueFactory();

// In this example, we specify the keyword string. Of course, this
// could just as easily be obtained by user input, or by reading from
// a file, or...
String keyword = "foobar";

// We prepare a query that retrieves all documents for a keyword.
// Notice that in this query the 'keyword' variable is not bound to
// any specific value yet.
TupleQuery keywordQuery = con.prepareTupleQuery(QueryLanguage.SERQL,
      "SELECT document FROM {document} ex:keyword {keyword}");

// Then we set the binding to a literal representation of our keyword.
// Evaluation of the query object will now effectively be the same as
// if we had specified the query as follows:
//   SELECT document FROM {document} ex:keyword {"foobar"}
keywordQuery.setBinding("keyword", factory.createLiteral(keyword));

// We then evaluate the prepared query and can process the result:
TupleQueryResult keywordQueryResult = keywordQuery.evaluate();

8.2.3. Creating, retrieving, removing individual statements

The RepositoryConnection can also be used for adding, retrieving, removing or otherwise manipulating individual statements, or sets of statements.

To be able to add new statements, we can use a ValueFactory to create the Values out of which the statements consist. For example, we want to add a few statements about two resources, Alice and Bob:

import org.openrdf.model.vocabulary.RDF;
import org.openrdf.model.vocabulary.RDFS;
...

ValueFactory f = myRepository.getValueFactory();

// create some resources and literals to make statements out of
URI alice = f.createURI("http://example.org/people/alice");
URI bob = f.createURI("http://example.org/people/bob");
URI name = f.createURI("http://example.org/ontology/name");
URI person = f.createURI("http://example.org/ontology/Person");
Literal bobsName = f.createLiteral("Bob");
Literal alicesName = f.createLiteral("Alice");

try {
   RepositoryConnection con = myRepository.getConnection();

   try {
      // alice is a person
      con.add(alice, RDF.TYPE, person);
      // alice's name is "Alice"
      con.add(alice, name, alicesName);

      // bob is a person
      con.add(bob, RDF.TYPE, person);
      // bob's name is "Bob"
      con.add(bob, name, bobsName);
   }
   finally {
      con.close();
   }
}
catch (OpenRDFException e) {
   // handle exception
}

Of course, it will not always be necessary to use a ValueFactory to create URIs. In practice, you will find that you quite often retrieve existing URIs from the repository (for example, by evaluating a query) and then use those values to add new statements.

As you can see in the above code, for the default RDF and RDF Schema properties (such as 'rdf:type' and 'rdfs:subClassOf') it is not necessary to create new URI objects. Instead, you can import the vocabulary classes org.openrdf.model.vocabulary.RDF and RDFS which provide you static references to the vocabulary primitives.

Retrieving statements works in a very similar way. One way of retrieving statements we have already seen actually: we can get a GraphQueryResult containing statements by evaluating a graph query. However, we can also use direct method calls to retrieve (sets of) statements. For example, to retrieve all statements about Alice, we could do:

RepositoryResult<Statement> statements = con.getStatements(alice, null, null, true);

The additional boolean parameter at the end (set to 'true' in this example) indicates whether inferred triples should be included in the result. Of course, this parameter only makes a difference if your repository uses an inferencer.

The RepositoryResult is an iterator-like object that lazily retrieves each matching statement from the repository when its next() method is called. Note that, like is the case with QueryResult objects, iterating over a RepositoryResult may result in exceptions which you should catch to make sure that the RepositoryResult is always properly closed after use:

RepositoryResult<Statement> statements = con.getStatements(alice, null, null, true);

try {
   while (statements.hasNext()) {
      Statement st = statements.next();

      ... // do something with the statement
   }
}
finally {
   statements.close(); // make sure the result object is closed properly
}

In the above method invocation, we see four parameters being passed. The first three represent the subject, predicate and object of the RDF statements which should be retrieved. A null value indicates a wildcard, so the above method call retrieves all statements which have as their subject Alice, and have any kind of predicate and object. The fourth parameter indicates whether or not inferred statements should be included or not.

Removing statements again works in a very similar fashion. Suppose we want to retract the statement that the name of Alice is "Alice"):

con.remove(alice, name, alicesName);

Or, if we want to erase all statements about Alice completely, we can do:

con.remove(alice, null, null);

8.2.4. Working with Collections and Iterations

Most of these examples have been on the level of individual statements. However, the Repository API offers several methods that work with Collections of statements, allowing more batch-like update operations.

For example, in the following bit of code, we first retrieve all statements about Alice, put them in a Collection and then remove them:

import info.aduna.iteration.Iterations;

// Retrieve all statements about Alice and put them in a list
RepositoryResult<Statement> statements = con.getStatements(alice, null, null, true));
List<Statement> aboutAlice = Iterations.addAll(statements, new ArrayList<Statement>());

// Then, remove them from the repository
con.remove(aboutAlice);

As you can see, the info.aduna.iteration.Iterations class provides a convenient method that takes an Iteration (of which RepositoryResult is a subclass) and a Collection as input, and returns the Collection with the contents of the iterator added to it. It also automatically closes the Iteration for you.

In the above code, you first retrieve all statements, put them in a list, and then remove them. Although this works fine, it can be done in an easier fashion, by simply supplying the resulting object directly:

con.remove(con.getStatements(alice, null, null, true));

The RepositoryConnection interface has several variations of add, retrieve and remove operations. See the Javadoc API documentation for a full overview of the options.

8.2.5. Using context

Sesame 2 supports the notion of context, which you can think of as a way to group sets of statements together through a single group identifier (this identifier can be a blank node or a URI).

A very typical way to use context is tracking provenance of the statements in a repository, that is, which file these statements originate from. For example, consider an application where you add RDF data from different files to a repository, and then one of those files is updated. You would then like to replace the data from that single file in the repository, and to be able to do this you need a way to figure out which statements need to be removed. The context mechanism gives you a way to do that.

In the following example, we add an RDF document from the Web to our repository, in a context. In the example, we make the context identifier equal to the Web location of the file being uploaded.

String location = "http://example.org/example/example.rdf";
String baseURI = location;
URL url = new URL(location);
URI context = f.createURI(location);

con.add(url, baseURI, RDFFormat.RDFXML, context);

We can now use the context mechanism to specifically address these statements in the repository for retrieve and remove operations:

// Get all statements in the context
RepositoryResult<Statement> result = con.getStatements(null, null, null, true, context);

try {
   while (result.hasNext()) {
      Statement st = result.next();
      ... // do something interesting with the result
   }
}
finally {
   result.close();
}

// Export all statements in the context to System.out, in RDF/XML format
RDFHandler rdfxmlWriter = new RDFXMLWriter(System.out);
con.export(context, rdfxmlWriter);

// Remove all statements in the context from the repository
con.clear(context);

In most methods in the Repository API, the context parameter is a vararg, meaning that you can specify an arbitrary number (zero, one, or more) of context identifiers. This way, you can combine different contexts together. For example, we can very easily retrieve statements that appear in either 'context1' or 'context2'.

In the following example we add information about Bob and Alice again, but this time each has their own context. We also create a new property called 'creator' that has as its value the name of the person who is the creator a particular context. The knowledge about creators of contexts we do not add to any particular context, however:

URI context1 = f.createURI("http://example.org/context1");
URI context2 = f.createURI("http://example.org/context2");
URI creator = f.createURI("http://example.org/ontology/creator");

// Add stuff about Alice to context1
con.add(alice, RDF.TYPE, person, context1);
con.add(alice, name, alicesName, context1);

// Alice is the creator of context1
con.add(context1, creator, alicesName);

// Add stuff about Bob to context2
con.add(bob, RDF.TYPE, person, context2);
con.add(bob, name, bobsName, context2);

// Bob is the creator of context2
con.add(context2, creator, bobsName);

Once we have this information in our repository, we can retrieve all statements about either Alice or Bob by using the context vararg:

// Get all statements in either context1 or context2
RepositoryResult<Statement> result =
      con.getStatements(null, null, null, true, context1, context2);

You should observe that the above RepositoryResult will not contain the information that context1 was created by Alice and context2 by Bob. This is because those statements were added without any context, thus they do not appear in context1 or context2, themselves.

To explicitly retrieve statements that do not have an associated context, we do the following:

// Get all statements that do not have an associated context
RepositoryResult<Statement> result =
      con.getStatements(null, null, null, true, (Resource)null);

This will give us only the statements about the creators of the contexts, because those are the only statements that do not have an associated context. Note that we have to explicitly cast the null argument to Resource, because otherwise it is ambiguous whether we are specifying a single value or an entire array that is null (a vararg is internally treated as an array). Simply invoking getStatements(s, p, o, true, null) without an explicit cast will result in an IllegalArgumentException.

We can also get everything that either has no context or is in context1:

// Get all statements that do not have an associated context, or that are in context1
RepositoryResult<Statement> result =
      con.getStatements(null, null, null, true, (Resource)null, context1);

So as you can see, you can freely combine contexts in this fashion.

Important:

getStatements(null, null, null, true);

is not the same as:

getStatements(null, null, null, true, (Resource)null);

The former (without any context id parameter) retrieves all statements in the repository, ignoring any context information. The latter, however, only retrieves statements that explicitly do not have any associated context.

8.2.6. Transactions

So far, we have shown individual operations on repositories: adding statements, removing them, etc. By default, a RepositoryConnection runs in autoCommit mode, meaning that each operation on a RepositoryConnection is immediately sent to the store and committed.

The RepositoryConnection interface supports a full transactional mechanism that allows one to group modification operations together and treat them as a single update: before the transaction is committed, none of the operations in the transaction has taken effect, and after, they all take effect. If something goes wrong at any point during a transaction, it can be rolled back so that the state of the repository is the same as before the transaction started. Bundling update operations in a single transaction often also improves update performance compared to multiple smaller transactions.

We can achieve this behaviour by switching off the RepositoryConnection's autoCommit mode. In the following example, we use a non-autocommit connection to bundle two file addition operations in a single transaction:

File inputFile1 = new File("/path/to/example1.rdf");
String baseURI1 = "http://example.org/example1/";

File inputFile2 = new File("/path/to/example2.rdf");
String baseURI2 = "http://example.org/example2/";

RepositoryConnection con = myRepository.getConnection();
try {
   con.setAutoCommit(false);

   // Add the first file
   con.add(inputFile1, baseURI1, RDFFormat.RDFXML);

   // Add the second file
   con.add(inputFile2, baseURI2, RDFFormat.RDFXML);

   // If everything went as planned, we can commit the result
   con.commit();
}
catch (RepositoryException e) {
   // Something went wrong during the transaction, so we roll it back
   con.rollback();
}
finally {
   // Whatever happens, we want to close the connection when we are done.
   con.close();
}

In the above example, we use a transaction to add two files to the repository. Only if both files can be successfully added will the repository change. If one of the files can not be added (for example because it can not be read), then the entire transaction is cancelled and none of the files is added to the repository.

Chapter 9. The SeRQL query language (revision 3.0)

9.1. Revisions

9.1.1. revision 1.1

SeRQL revision 1.1 is a syntax revision (see issue tracker item SES-75). This document describes the revised syntax. From Sesame release 1.2-RC1 onwards, the old syntax is no longer supported.

9.1.2. revision 1.2

SeRQL revision 1.2 covers a set of new functions and operators:

New operations have been marked with (R1.2) where appropriate in this document.

9.1.3. revision 2.0

SeRQL revision 2.0 is an extension of SeRQL that offers functionality for querying contexts. See Section 9.15, “Querying context (R2.0)” for details.

9.1.4. revision 3.0

SeRQL revision 3.0 modifies SeRQL to be more like SPARQL, adopting its semantics and operators. Main backwards compatiblity issues with revision 2.0 are:

  • The NULL value has been deprecated; the BOUND-operator should now be used instead. For now, the SeRQL parser will automatically translate NULL values to BOUND-operators as much as possible.
  • The semantics of optional joins have been changed from the existing iterative semantics to the better defined compositional semantics that is used in SPARQL. This change will only affect some corner cases that are unlikely to appear in actual queries.

9.2. Introduction

SeRQL ("Sesame RDF Query Language", pronounced "circle") is an RDF query language that is very similar to SPARQL, but with other syntax. SeRQL was originally developed as a better alternative for the query languages RQL and RDQL. A lot of SeRQL's features can now be found in SPARQL and SeRQL has adopted some of SPARQL's features in return.

This document briefly shows all of these features. After reading through this document one should be able to write SeRQL queries.

9.3. URIs, literals and variables

URIs and literals are the basic building blocks of RDF. For a query language like SeRQL, variables are added to this list. The following sections will show how to write these down in SeRQL.

9.3.1. Variables

Variables are identified by names. These names must start with a letter or an underscore ('_') and can be followed by zero or more letters, numbers, underscores, dashes ('-') or dots ('.'). Examples variable names are:

  • Var1
  • _var2
  • unwise.var-name_isnt-it

SeRQL keywords are not allowed to be used as variable names. Currently, the following keywords are used or reserved for future use in SeRQL: select, construct, from, where, using, namespace, true, false, not, and, or, like, label, lang, datatype, null, bound, isresource, isliteral, sort, in, union, intersect, minus, exists, forall, distinct, limit, offset.

Keywords in SeRQL are all case-insensitive, this in contrast to variable names; these are case-sensitive.

9.3.2. URIs

There are two ways to write down URIs in SeRQL: either as full URIs or as abbreviated URIs. Full URIs must be surrounded with "<" and ">". Examples of this are:

  • <http://www.openrdf.org/index.html>
  • <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
  • <mailto:sesame@openrdf.org>
  • <file:///C:\rdffiles\test.rdf>

As URIs tend to be long strings with the first part being shared by several of them (i.e. the namespace), SeRQL allows one to use abbreviated URIs (or QNames) by defining (short) names for these namespaces which are called "prefixes". A QName always starts with one of the defined prefixes and a colon (":"). After this colon, the part of the URI that is not part of the namespace follows. The first part, consisting of the prefix and the colon, is replaced by the full namespace by the query engine. Some example QNames are:

  • sesame:index.html
  • rdf:type
  • foaf:Person

9.3.3. Literals

RDF literals consist of three parts: a label, a language tag, and a datatype. The language tag and the datatype are optional and at most one of these two can accompany a label (a literal can not have both a language tag and a datatype). The notation of literals in SeRQL has been modelled after their notation in N-Triples; literals start with the label, which is surrounded by double quotes, optionally followed by a language tag with a "@" prefix or by a datatype URI with a "^^" prefix. Example literals are:

  • "foo"
  • "foo"@en
  • "<foo/>"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>

The SeRQL notation for abbreviated URIs can also be used. When the prefix rdf is mapped to the namespace http://www.w3.org/1999/02/22-rdf-syntax-ns#, the last example literal could also have been written down like:

  • "<foo/>"^^rdf:XMLLiteral

SeRQL has also adopted the character escapes from N-Triples; special characters can be escaped by prefixing them with a backslash. One of the special characters is the double quote. Normally, a double quote would signal the end of a literal's label. If the double quote is part of the label, it needs to be escaped. For example, the sentence John said: "Hi!" can be encoded in a SeRQL literals as: "John said: \"Hi!\"".

As the backslash is a special character itself, it also needs to be escaped. To encode a single backslash in a literal's label, two backslashes need to be written in the label. For example, a Windows directory would be encoded as: "C:\\Program Files\\Apache Tomcat\\".

SeRQL has functions for extracting each of the three parts of a literal. These functions are label, lang, and datatype. label("foo"@en) extracts the label "foo", lang("foo"@en) extracts the language tag "en", and datatype("foo"^^rdf:XMLLiteral) extracts the datatype rdf:XMLLiteral. The use of these functions is explained later.

9.3.4. Blank Nodes (R1.2)

RDF has a notion of blank nodes. These are nodes in the RDF graph that are not labeled with a URI or a literal. The interpretation of such blank nodes is as a form of existential quantification: it allows one to assert that "there exists a node such that..." without specifying what that particular node is. Blank nodes do in fact often have identifiers, but these identifiers are assigned internally by whatever processor is processing the graph and they are only valid in the local context, not as global identifiers (unlike URIs).

Strictly speaking blank nodes are only addressable indirectly, by querying for one or more properties of the node. However, SeRQL, as a practical shortcut, allows blank node identifiers to be used in queries. The syntax for blank nodes is adopted from N-Triples, using a QName-like syntax with "_" as the namespace prefix, and the internal blank node identifier as the local name. For example:

  • _:bnode1

This identifies the blank node with internal identifier "bnode1". These blank node identifiers can be used in the same way that normal URIs or QNames can be used.

Caution: It is important to realize that addressing blank nodes in this way makes SeRQL queries non-portable across repositories. There is no guarantee that in two repositories, even if they contain identical datasets, the blank node identifiers will be identical. It may well be that "bnode1" in repository A is a completely different blank node than "bnode1" in repository B. Even in the same repository, it is not guaranteed that blank node identifiers are stable over updates: if certain statements are added to or removed from a repository, it is not guaranteed "bnode1" still identifies the same blank node that it did before the update operation.

9.4. Path expressions

One of the most prominent parts of SeRQL are path expressions. Path expressions are expressions that match specific paths through an RDF graph.

9.4.1. Basic path expressions

Imagine that we want to query an RDF graph for persons who work for companies that are IT companies. Querying for this information comes down to finding the following pattern in the RDF graph (gray nodes denote variables):

Figure 9.1. A basic path expression

A basic path expression

The SeRQL notation for path expressions resembles the picture above; it is written down as:

{Person} foo:worksFor {Company} rdf:type {foo:ITCompany}

The parts surrounded by curly brackets represent the nodes in the RDF graph, the parts between these nodes represent the edges in the graph. The direction of the arcs (properties) in SeRQL path expressions is always from left to right.

In SeRQL queries, multiple path expressions can be specified by seperating them with commas. For example, the path expression show before can also be written down as two smaller path expressions:

{Person} foo:worksFor {Company},
{Company} rdf:type {foo:ITCompany}

The nodes and edges in the path expressions can be variables, URIs and literals. Also, a node can be left empty in case one is not interested in the value of that node. Here are some more example path expressions to illustrate this:

  • {Person} foo:worksFor {} rdf:type {foo:ITCompany}
  • {Painting} ex:painted_by {} ex:name {"Picasso"}
  • {comic:RoadRunner} SomeRelation {foo:WillyECoyote}

9.4.2. Path expression short cuts

Each and every path can be constructed using a set of basic path expressions. Sometimes, however, it is nicer to use one of the available short cuts. There are three types of short cuts:

9.4.2.1. Multi-value nodes

In situations where one wants to query for two or more statements with identical subject and predicate, the subject and predicate do not have to be repeated over and over again. Instead, a multi-value node can be used:

{subj1} pred1 {obj1, obj2, obj3}

A built-in constraint on this construction is that each value for the variables in the multi-value node is unique (i.e. they are pairwise disjoint). Therefore, this path expression is equivalent to the following combination of path expressions and boolean constraints:

FROM
  {subj1} pred1 {obj1},
  {subj1} pred1 {obj2},
  {subj1} pred1 {obj3}
WHERE obj1 != obj2 AND obj1 != obj3 AND obj2 != obj3

Or graphically:

Figure 9.2. Multi-value nodes

Multi-value nodes

Multi-value nodes can also be used when statements share the predicate and object, e.g.:

{subj1, subj2, subj3} pred1 {obj1}

When used in a longer path expression, multi-value nodes apply to both the part left of the node and the part right of the node. The following path expression:

{first} pred1 {middle1, middle2} pred2 {last}

matches the following graph:

Figure 9.3. Multi-value nodes in a longer path expression

Multi-value nodes in a longer path expression

9.4.2.2. Branches

One of the shorts cuts that is probably used most, is the notation for branches in path expressions. There are lots of situations where one wants to query multiple properties of a single subject. Instead of repeating the subject over and over again, one can use a semi-colon to attach a predicate-object combination to the subject of the last part of a path expression, e.g.:

{subj1} pred1 {obj1};
        pred2 {obj2}

Which is equivalent to:

{subj1} pred1 {obj1},
{subj1} pred2 {obj2}

Or graphically:

Figure 9.4. Branches in a path expression

Branches in a path expression

A more advanced example is:

{first} pred {} pred1 {obj1};
                pred2 {obj2} pred3 {obj3}

Which matches the following graph:

Figure 9.5. Branches in a longer path expression

Branches in a longer path expression

Note that an anonymous variable is used in the middle of the path expressions.

9.4.2.3. Reified statements

The last short cut is a short cut for reified statements. A path expression representing a single statement (i.e. {node} edge {node}) can be written between the curly brackets of a node, e.g.:

{ {reifSubj} reifPred {reifObj} } pred {obj}

This would be equivalent to querying (using "rdf:" as a prefix for the RDF namespace, and "Statement" as a variable for storing the statement's URI):

{Statement} rdf:type {rdf:Statement},
{Statement} rdf:subject {reifSubj},
{Statement} rdf:predicate {reifPred},
{Statement} rdf:object {reifObj},
{Statement} pred {obj}

Again, graphically:

Figure 9.6. A reification path expression

A reification path expression

9.4.3. Optional path expressions

Optional path expressions differ from 'normal' path expressions in that they do not have to be matched to find query results. The SeRQL query engine will try to find paths in the RDF graph matching the path expression, but when it cannot find any paths it will skip the expression and leave any variables in it unbound.

Consider an RDF graph that contains information about people that have names, ages, and optionally e-mail addresses. This is a situation that is very common in RDF data. A logical query on this data is a query that yields all names, ages and, when available, e-mail addresses of people, e.g.:

{Person} ex:name {Name};
         ex:age  {Age};
         ex:email {EmailAddress}

However, using normal path expressions like in the query above, people without e-mail address will not be included in the query result. With optional path expressions, one can indicate that a specific (part of a) path expression is optional. This is done using square brackets, i.e.:

{Person} ex:name {Name};
         ex:age  {Age};
        [ex:email {EmailAddress}]

Or alternatively:

 {Person} ex:name {Name};
          ex:age  {Age},
[{Person} ex:email {EmailAddress}]

In contrast to the first path expressions, this expression will also match people without an e-mail address. For these people, the variable EmailAddress will be unbound.

Optional path expressions can also be nested. This is useful in situations where the existence of a specific path is dependent on the existence of another path. For example, the following path expression queries for the titles of all known documents and, if the author of the document is known, the name of the author (if it is known) and his e-mail address (if it is known):

{Document} ex:title {Title};
          [ex:author {Author} [ex:name {Name}];
                              [ex:email {Email}]]

With this path expression, the SeRQL query engine will not try to find the name and e-mail address of an author when it cannot even find the resource representing the author.

9.5. Select- and construct queries

The SeRQL query language supports two querying concepts. The first one can be characterized as returning a table of values, or a set of variable-value bindings. The second one returns an RDF graph, which can be a subgraph of the graph being queried, or a graph containing information that is derived from it. The first type of queries are called "select queries", the second type of queries are called "construct queries".

A SeRQL query is typically built up from one to seven clauses. For select queries these clauses are: SELECT, FROM, FROM CONTEXT, WHERE, LIMIT, OFFSET and USING NAMESPACE. One might recognize some of these clauses from SQL, but their usage is slightly different. For construct queries the clauses are the same with the exception of the first; construct queries start with a CONSTRUCT clause instead of a SELECT clause. Except for the first clause (SELECT or CONSTRUCT), all clauses are optional.

The first clause (i.e. SELECT or CONSTRUCT) determines what is done with the results that are found. In a SELECT clause, one can specify which variable values should be returned. In a CONSTRUCT clause, one can specify which statements should be returned.

The FROM clause specifies path expressions, which were explained in the previous section. It defines the paths in an RDF graph that are relevant to the query. Note that, when the FROM clause is not specified, the query will simply return the constants specified in the SELECT or CONSTRUCT clause.

The FROM CONTEXT clause is new in SeRQL revision 2.0. It is a variant of the FROM clause that allows one to constrain the path expressions in the clause to one or more contexts. Using context in querying will be explained in more detail in Section 9.15, “Querying context (R2.0)”.

The WHERE clause specifies additional (Boolean) constraints on the values in the path expressions. These are constraints on the nodes and edges of the paths that cannot be expressed in the path expressions themselves.

The LIMIT and OFFSET clauses can be used separately or combined in order to get a subset of all query answers. Their usage is very similar to the LIMIT and OFFSET clauses in SQL queries. The LIMIT clause determines the (maximum) number of query answers that will be returned. The OFFSET clause determines which query answer will be returned as the first result, skipping as many query results as specified in this clause.

Finally, the USING NAMESPACE clause can be used to declare namespace prefixes. These are the mappings from prefixes to namespaces that were referred to in one of previous sections about (abbreviated) URIs.

The WHERE, LIMIT, OFFSET and USING NAMESPACE clauses will be explained in one of the next sections. The following section will explain the SELECT and FROM clause.

9.6. Select queries

As said before, select queries return tables of values, or sets of variable-value bindings. Which values are returned can be specified in the select clause. One can specify variables and/or values in the select clause, seperated by commas. The following example query returns all URIs of classes:

SELECT C
FROM {C} rdf:type {rdfs:Class}

It is also possible to use a '*' in the SELECT clause. In that case, all variable values will be returned, e.g.:

SELECT *
FROM {S} rdfs:label {O}

This query will return the values of the variables S and O.

SELECT O, S
FROM {S} rdfs:label {O}

By default, the results of a select query are not filtered for duplicate rows. Because of the nature of the above queries, these queries will never return duplicates. However, more complex queries might result in duplicate result rows. These duplicates can be filtered out by the SeRQL query engine. To enable this functionality, one needs to specify the DISTINCT keyword after the select keyword. For example:

SELECT DISTINCT *
FROM {Country1} ex:borders {} ex:borders {Country2}
USING NAMESPACE
    ex = <http://example.org/things#>

9.7. Construct queries

Construct queries return RDF graphs as set of statements. The statements that a query should return can be specified in the construct clause using the previously explained path expressions. The following is an example construct query:

CONSTRUCT {Parent} ex:hasChild {Child}
FROM {Child} ex:hasParent {Parent}
USING NAMESPACE
    ex = <http://example.org/things#>

This query defines the inverse of the property ex:hasParent to be ex:hasChild. This is just one example of a query that produces information that is derived from the original information. Here is one more example:

CONSTRUCT
    {Artist} rdf:type {ex:Painter};
             ex:hasPainted {Painting}
FROM
    {Artist} rdf:type {ex:Artist};
             ex:hasCreated {Painting} rdf:type {ex:Painting}
USING NAMESPACE
    ex = <http://example.org/things#>

This query derives that an artist who has created a painting, is a painter. The relation between the painter and the painting is modelled to be art:hasPainted.

Instead of specifying a path expression in the CONSTRUCT clause, one can also use a '*'. In that case, the CONSTRUCT clause is identical to the FROM clause. This allows one to extract a subgraph from a larger graph, e.g.:

CONSTRUCT *
FROM {SUB} rdfs:subClassOf {SUPER}

This query extracts all rdfs:subClassOf relations from an RDF graph.

Just like with select queries, the results of a construct query are not filtered for duplicate statements by default. Again, these duplicates are filtered out by the SeRQL query engine if the DISTINCT keyword is specified after the construct keyword, for example:

CONSTRUCT DISTINCT
    {Artist} rdf:type {ex:Painter}
FROM
    {Artist} rdf:type {ex:Artist};
             ex:hasCreated {} rdf:type {ex:Painting}
USING NAMESPACE
    ex = <http://example.org/things#>

9.8. The WHERE clause

The third clause in a query is the WHERE clause. This is an optional clause in which one can specify Boolean constraints on variables.

The following sections will explain the available Boolean expressions for use in the WHERE clause. Section 9.8.8, “Nested WHERE clauses (R1.2)” will explain how WHERE clauses can be nested inside optional path expressions.

9.8.1. Boolean constants

There are two Boolean constants, TRUE and FALSE. The first one is simply always true, the last one is always false. The following query will never produce any results because the constraint in the where clause will never evaluate to true:

SELECT *
FROM {X} Y {Z}
WHERE FALSE

9.8.2. Value (in)equality

The most common boolean constraint is equality or inequality of values. Values can be compared using the operators "=" (equality) and "!=" (inequality). The expression

Var = <foo:bar>

is true if the variable Var has been bound to the URI <foo:bar>, and the expression

Var1 != Var2

checks whether two variables are bound to unequal values.

9.8.3. Numerical comparisons

Numbers can be compared to each other using the operators "<" (lower than), "<=" (lower than or equal to), ">" (greater than) and ">=" (greater than or equal to). SeRQL uses a literal's datatype to determine whether its value is numerical. All XML Schema built-in numerical datatypes are supported, i.e.: xsd:float, xsd:double, xsd:decimal and all subtypes of xsd:decimal (xsd:long, xsd:nonPositiveInteger, xsd:byte, etc.), where the prefix xsd is used to reference the XML Schema namespace.

In the following query, a comparison between values of type xsd:positiveInteger is used to retrieve all countries that have a population of less than 1 million:

SELECT Country
FROM {Country} ex:population {Population}
WHERE Population < "1000000"^^xsd:positiveInteger
USING NAMESPACE
    ex = <http://example.org/things#>

If one want to compare values of incompatible types, one can try to cast one or both of the values to another type. For example in the above query, if the values that Population is bound to generally do not have a datatype, one can cast these values to xsd:integer to make the comparison work, e.g.:

SELECT Country
FROM {Country} ex:population {Population}
WHERE xsd:integer(Population) < "1000000"^^xsd:positiveInteger
USING NAMESPACE
    ex = <http://example.org/things#>

SeRQL supports all value casting methods from SPARQL, see SPARQL's Constructor Functions for more details.

9.8.4. bound() (R3.0)

The bound() boolean function checks whether a specific variable has been bound to a value. For example, the following query returns the names of all people without a (known) e-mail address.

SELECT Name
FROM {Person} foaf:name {Name};
             [ex:email {EmailAddress}]
WHERE NOT BOUND(EmailAddress)

9.8.5. isURI() and isBNode() (R1.2)

The isURI() and isBNode() boolean functions are more specific versions of isResource(). They check whether a variable is bound to a URI value or a BNode value, respectively. For example, the following query returns only URIs (and filters out all bNodes and literals):

SELECT V
FROM {R} prop {V}
WHERE isURI(V)

9.8.6. The LIKE operator (R1.2)

The LIKE operator can check whether a value matches a specified pattern of characters. '*' characters can be used as wildcards, matching with zero or more characters. The rest of the characters are compared lexically. The pattern is surrounded with double quotes, just like a literal's label.

SELECT Country
FROM {Country} ex:name {Name}
WHERE Name LIKE "Belgium"
USING NAMESPACE
    ex = <http://example.org/things#>

By default, the LIKE operator does a case-sensitive comparison: in the above query, the operator fails is the variable Name is bound to the value "belgium" instead of "Belgium". Optionally, one can specify that the operator should perform a case-insensitive comparison:

SELECT Country
FROM {Country} ex:name {Name}
WHERE Name LIKE "belgium" IGNORE CASE
USING NAMESPACE
    ex = <http://example.org/things#>

In this query, the operator will succeed for "Belgium", "belgium", "BELGIUM", etc.

The '*' character can be used as a wildcard to indicate substring matches, for example:

SELECT Country
FROM {Country} ex:name {Name}
WHERE Name LIKE "*Netherlands"
USING NAMESPACE
    ex = <http://example.org/things#>

This query will match any country names that end with the string "Netherlands", for example "The Netherlands".

9.8.7. AND, OR, NOT

Boolean constraints and functions can be combined using the AND and OR operators, and negated using the NOT operator. The NOT operator has the highest presedence, then the AND operator, and finally the OR operator. Parentheses can be used to override the default presedence of these operators. The following query is a (kind of artifical) example of this:

SELECT *
FROM {X} Prop {Y} rdfs:label {L}
WHERE NOT L LIKE "*FooBar*" AND
      (Y = <foo:bar> OR Y = <bar:foo>) AND
      isLiteral(L)

9.8.8. Nested WHERE clauses (R1.2)

In order to be able to express boolean constraints on variables in optional path expressions, it is possibl