Virtuoso Redland RDF Storage Provider

What is Redland
What is the Virtuoso Redland Provider

Fig 1: Redland Component Stack

Setup

Required files
Compling Redland with Virtuoso storage support
Connection Attributes

Related

What is Redland

Redland is a set of free software 'C' libraries that provide support for the Resource Description Framework (RDF), providing modular, object-based libraries, and APIs for manipulating the RDF graph, triples, URIs, and Literals. Redland includes several high-level language APIs providing RDF manipulation and storage, and requires the Raptor RDF parser and Rasqal RDF syntax and query library for its use.

What is the Virtuoso Redland Provider

The Virtuoso Redland RDF Provider is an implementation of the Storage API, Model, and Query interfaces of the Redland framework for RDF. This provider enables the execution of queries via the Redland Rasqal query engine or via Virtuoso query engine directly against the Virtuoso Quad store. The Virtuoso Redland Provider uses ODBC as the data access mechanism for communicating with the Virtuoso Quad Store and therefore requires the Virtuoso ODBC Driver be installed on the Redland client and a suitable ODBC DSN be configured for connecting to the target Virtuoso Quad Store instance. The provider has been tested against the Redland 1.0.8 version available for download at the time of writing.

Fig 1: Redland Component Stack

As indicated in the diagram above, the Virtuoso Provider can be used to execute RDF queries either directly against the Virtuoso graph storage module (which supports the SPARQL, SPARUL, and SPARQL-BI query languages) or via the Rasqal query engine built into Redland (which supports only the SPARQL query language). The query handler is specified by the query; use "vsparql" for Virtuoso, or the default "sparql" for Redland, as indicated in the sample queries below:

rdfproc -r xml -t "user='dba',password='dba',dsn='Demo'" gr query sparql - "SELECT * WHERE { ?s ?p ?o }"           ;; via Redland Rasqal engine

rdfproc -r xml -t "user='dba',password='dba',dsn='Demo'" gr query vsparql - "SELECT * WHERE { ?s ?p ?o }"          ;; direct to Virtuoso storage module

The Virtuoso Provider uses the SPASQL query language for querying the remote Virtuoso QUAD store.

Setup

Required files

The Virtuoso Redland Provider has been integrated into the Redland RDF Framework and submitted to the open source project to become part of the standard distribution available for download.

Compling Redland with Virtuoso storage support

Download Redland from the svn repository.

Use the following additional configure options to enable support for Virtuoso storage:


--with-virtuoso(=yes|no)  Enable Virtuoso RDF store 
                            (default=auto)
--with-iodbc(=DIR)        Select iODBC support
                            DIR is the iODBC base install directory
                            (default=/usr/local)
--with-unixodbc(=DIR)     Select UnixODBC support
                            DIR is the UnixODBC base install directory
                            (default=/usr/local)
--with-datadirect(=DIR)   Select DataDirect support
                            DIR is the DataDirect base install directory
                            (default=/usr/local)
--with-odbc-inc=DIR       Specify custom ODBC include directory
                            (default=/usr/local/include)
--with-odbc-lib=DIR       Specify custom ODBC lib directory
                            (default=/usr/local/lib)

The --with-virtuoso option is auto-enabled if a valid ODBC Driver Manager (iODBC, UnixODBC?, or DataDirect?) or include and lib directories for required ODBC header files and libraries are discovered via the suitable setting for one or more of the other ODBC related options above.
Assuming iODBC is installed, the following command can be used to enable Virtuoso storage support to be configured for compilation into your Redland build:
```
./configure --with-iodbc=/usr/local/iODBC
```

Run make to compile the Redland libraries and sudo make install to install in the default /usr/local location

Test compilation with the utils/rdfproc test utility. This test will use the default hashes storage.


rdfproc test parse http://planetrdf.com/guide/rss.rdf
rdfproc test print
rdfproc test serialize ntriples

Ensure you have the Virtuoso ODBC Driver installed, and a valid ODBC DSN called "Local Virtuoso" configured for your target Virtuoso Server.

Set the following environment variable:


export RDFPROC_STORAGE_TYPE=virtuoso                                   ;; Enable Virtuoso Storage 
export ODBCINI=<path_to_odbcini_directory>/odbc.ini                    ;; Enable ODBC DSN to be located
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH                 ;; May be required to enable Redland libraries to be located

Test Virtuoso storage with the provided utils/vtest test program:


$ utils/vtest 
  1: Remove all triples in <http://red> context
**PASSED**: removed context triples from the graph
  2: Add triples to <http://red> context
**PASSED**: add triple to context
  3: Print all triples in <http://red> context
[[
  {[aa], [bb], [cc]} with context [http://red]
  {[aa], [bb1], [cc]} with context [http://red]
  {[aa], [a2], "cc"} with context [http://red]
  {[aa], [a2], (cc)} with context [http://red]
  {[mm], [nn], "Some long literal with language@en"} with context [http://red]
  {[oo], [pp], "12345^^<http://www.w3.org/2001/XMLSchema#int>"} with context [http://red]
]]
**PASSED**:
  4: Count of triples in <http://red> context
**PASSED**: graph has 6 triples
  5: Exec:  ARC  aa bb 
Matched node: [cc]
**PASSED**:
  6: Exec:  ARCS  aa cc 
Matched node: [bb] with context [http://red]
Matched node: [bb1] with context [http://red]
: matching nodes: 2
**PASSED**:
  7: Exec:  ARCS-IN  cc 
Matched arc: [bb] with context [http://red]
Matched arc: [bb1] with context [http://red]
**PASSED**: matching arcs: 2
  8: Exec:  ARCS-OUT  aa 
Matched arc: [bb] with context [http://red]
Matched arc: [bb1] with context [http://red]
Matched arc: [a2] with context [http://red]
Matched arc: [a2] with context [http://red]
**PASSED**: matching arcs: 4
  9: Exec:  CONTAINS aa bb1 cc 
**PASSED**: the graph contains the triple
 10: Exec:  FIND aa - - 
Matched triple: {[aa], [bb], [cc]} with context [http://red]
Matched triple: {[aa], [bb1], [cc]} with context [http://red]
Matched triple: {[aa], [a2], "cc"} with context [http://red]
Matched triple: {[aa], [a2], (cc)} with context [http://red]
**PASSED**: matching triples: 4
 11: Exec:  HAS-ARC-IN cc bb 
**PASSED**: the graph contains the arc
 12: Exec:  HAS-ARC-OUT aa bb 
**PASSED**: the graph contains the arc
 13: Exec:  SOURCE  aa cc 
Matched node: [aa]
**PASSED**:
 14: Exec:  SOURCES  bb cc 
Matched node: [aa] with context [http://red]
: matching nodes: 1
**PASSED**:
 15: Exec:  TARGET  aa bb 
Matched node: [cc]
**PASSED**:
 16: Exec:  TARGETS  aa bb 
Matched node: [cc] with context [http://red]
: matching nodes: 1
**PASSED**:
 17: Exec:  REMOVE aa bb1 cc 
**PASSED**: removed triple from the graph
 18: Exec:  QUERY "CONSTRUCT {?s ?p ?o} FROM <http://red> WHERE {?s ?p ?o}" 
Matched triple: {[aa], [a2], "cc"}
Matched triple: {[oo], [pp], "12345^^<http://www.w3.org/2001/XMLSchema#int>"}
Matched triple: {[aa], [a2], (cc)}
Matched triple: {[aa], [bb], [cc]}
Matched triple: {[mm], [nn], "Some long literal with language@en"}
**PASSED**: matching triples: 5
 19: Exec1:  QUERY_AS_BINDINGS "SELECT * WHERE {graph <http://red> { ?s ?p ?o }}" 
**: Formatting query result as 'xml':
<?xml version="1.0" encoding="utf-8"?>
<sparql xmlns="http://www.w3.org/2005/sparql-results#">
  <head>
    <variable name="s"/>
    <variable name="p"/>
    <variable name="o"/>
  </head>
  <results>
    <result>
      <binding name="s"><uri>aa</uri></binding>
      <binding name="p"><uri>bb</uri></binding>
      <binding name="o"><uri>cc</uri></binding>
    </result>
    <result>
      <binding name="s"><uri>aa</uri></binding>
      <binding name="p"><uri>a2</uri></binding>
      <binding name="o"><literal>cc</literal></binding>
    </result>
    <result>
      <binding name="s"><uri>aa</uri></binding>
      <binding name="p"><uri>a2</uri></binding>
      <binding name="o"><bnode>cc</bnode></binding>
    </result>
    <result>
      <binding name="s"><uri>mm</uri></binding>
      <binding name="p"><uri>nn</uri></binding>
      <binding name="o"><literal>Some long literal with language@en</literal></binding>
    </result>
    <result>
      <binding name="s"><uri>oo</uri></binding>
      <binding name="p"><uri>pp</uri></binding>
      <binding name="o"><literal>12345^^<http://www.w3.org/2001/XMLSchema#int></literal></binding>
    </result>
  </results>
</sparql>
**PASSED**:
 20: Exec2:  QUERY_AS_BINDINGS "SELECT * WHERE {graph <http://red> { ?s ?p ?o }}" 
: Query returned bindings results:
result: [s=[aa], p=[bb], o=[cc]]
result: [s=[aa], p=[a2], o=cc]
result: [s=[aa], p=[a2], o=(cc)]
result: [s=[mm], p=[nn], o=Some long literal with language@en]
result: [s=[oo], p=[pp], o=12345^^<http://www.w3.org/2001/XMLSchema#int>]
: Query returned 5 results
**PASSED**:
=============================================
PASSED: 20  FAILED:  0

Connection Attributes

The Virtuoso Redland Provider has the following connection attributes available for use:

dsn - ODBC datasource name
user - user name of database server
password - password of database server
host - hostname:portno of the database server
charset - database charset to use

NOTE: Take care exposing the password as, for example, program arguments or environment variables. The rdfproc utility can help with this by reading the password from standard input. Inside programs, one way to prevent storing the password in a string is to construct a Redland hash of the storage options such as via librdf_hash_from_string and use librdf_new_storage_with_options to create a storage. The rdfproc utility source code demonstrates this.

The storage name parameter given to the storage constructor librdf_new_storage is used inside the Virtuoso store to allow multiple stores inside one Virtuoso database instance, as parametrized with the options above.

This store always provides contexts; the Boolean storage option contexts is not checked.

Examples:

  /* A new Virtuoso store */
  storage=librdf_new_storage(world, "virtuoso", "db1",
      "dsn='Local Virtuoso',user='demo',password='demo'");

  /* A different, existing Virtuoso store in the same database as above */
  storage=librdf_new_storage(world, "virtuoso", "db2",
      "dsn='Local Virtuoso',user='demo',password='demo'");

  /* An existing Virtuoso store on a different database server */
  storage=librdf_new_storage(world, "virtuoso", "http://red3",
      "dsn='Remote Virtuoso',user='demo',password='demo'");

  /* Opening with an options hash */
  options=librdf_new_hash(world, NULL);
  librdf_hash_from_string(options, 
      "dsn='Local Virtuoso',user='demo'");
  librdf_hash_put_strings(options, "password", user_password);
  storage=librdf_new_storage_with_options(world, "virtuoso", "http://red3", options);