[RIO-29] blank node handling in parsers doesn't scale

Wed, 21 Sep 2005 12:54:57 +0200 (CEST)

The processing of blank nodes in Rio's parsers doesn't scale for documents with huge amount of named blank nodes in it. The reason for this is that the parsers keep track of all encountered named blank nodes as they may be referenced again later in the document. As blank node names scope to the entire document, the same blank node name must map to the same blank node objects. Possibly the best way to solve this is to do the mapping in an external handler. One such handler might use a database or the file system to store mapping information instead of keeping this information in main memory.

[RIO-47] Reduce memory requirements for verifying uniqueness of rdf:ID attributes

Wed, 14 Jun 2006 12:24:04 +0200 (CEST)

Source: http://www.openrdf.org/forum/mvnforum/viewthread?thread=967

The RDF/XML specification requires the values of rdf:ID attributes combined with the applicable base URI to be unique within a single RDF/XML document[1]. The RDF/XML parser currently stores these ID-base URI-combinations as URIs in a HashMap. As a result, a considerable amount of memory is needed to be able to parse documents that contain large amounts of rdf:ID attributes, such as the Uniprot benchmark files[2]. As the number of base URIs normally is much smaller than the number of IDs it is possible to reduce the memory requirements by storing the IDs per base URI so that the base URI only needs to be stored once.

[1] http://www.w3.org/TR/2004/REC-rdf-syntax-grammar-20040210/#section-constraints-id
[2] http://expasy3.isb-sib.ch/~ejain//rdf/data-for-benchmarks/

openRDF.org Issue Tracker

[RIO-29] blank node handling in parsers doesn't scale

[RIO-47] Reduce memory requirements for verifying uniqueness of rdf:ID attributes