History | Log In     View a printable version of the current page. Get help!  
Issue Details [XML]

Key: SES-977
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Jeen Broekstra
Reporter: Paul Hermans
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Sesame

query working in 2.6.0 leads to 500 internal server error in 2.6.5

Created: 11/Apr/12 07:59 PM   Updated: 09/May/12 11:12 PM
Component/s: Query Engine
Affects Version/s: 2.6.5
Fix Version/s: 2.6.6

File Attachments: 1. Zip Archive export_20120411.n3.zip (126 kb)

Environment: Mac OS X 10.7.3
Issue Links:
Cause
 
This issue is caused by:
SES-993 ArbitraryLengthPath when both start a... Major Resolved


 Description   
This query leads to 500 internal server error.
Wasn't the case in version 2.0 and in other stores.

PREFIX overheidbuza:<http://standaarden.overheid.nl/buza/terms/>
PREFIX skosxl:<http://www.w3.org/2008/05/skos-xl#>
PREFIX foaf:<http://xmlns.com/foaf/0.1/>
PREFIX rijksoverheid:<http://standaarden.overheid.nl/rijksoverheid/terms/>
PREFIX inspectieloket:<http://standaarden.overheid.nl/inspectieloket/terms/>
PREFIX void:<http://rdfs.org/ns/void#>
PREFIX overheidbm:<http://standaarden.overheid.nl/bm/terms/>
PREFIX ow:<http://standaarden.overheid.nl/owms/terms>
PREFIX dcterms:<http://purl.org/dc/terms/>
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>
PREFIX overheidvg:<http://standaarden.overheid.nl/vergunningen/terms/>
PREFIX overheidproduct:<http://standaarden.overheid.nl/product/terms/>
PREFIX overheidrg:<http://standaarden.overheid.nl/cvdr/terms/>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
PREFIX owl:<http://www.w3.org/2002/07/owl#>
PREFIX overheid:<http://standaarden.overheid.nl/owms/terms/>
PREFIX rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX scovo:<http://purl.org/NET/scovo#>
PREFIX overheidvac:<http://standaarden.overheid.nl/vac/terms/>
PREFIX skos:<http://www.w3.org/2004/02/skos/core#>
PREFIX overheidop:<http://standaarden.overheid.nl/op/terms/>
CONSTRUCT {
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> a <http://rdfs.org/ns/void#Dataset> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> dcterms:creator <http://standaarden.overheid.nl/owms/terms/ICTU> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> dcterms:description "De dataset van de Nederlandse overheid"@nl .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> dcterms:issued ?date .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> dcterms:license <http://www.gnu.org/copuleft/fdl.html> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> dcterms:title "OWMS dataset" .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#Feature> _:b0 .
    _:b0 dcterms:format "text/rdf+n3" .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#Feature> _:b1 .
    _:b1 dcterms:format "application/rdf+xml" .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#exampleResource> <http://standaarden.overheid.nl/owms/terms/Schiermonnikoog_(gemeente)> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#exampleResource> <http://standaarden.overheid.nl/owms/terms/omgevingsvergunning> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#exampleResource> <http://standaarden.overheid.nl/owms/terms/ROM_Rijnmond> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#statItem> _:b2 .
    _:b2 rdf:value ?agents .
    _:b2 <http://purl.org/NET/scovo#dimension> <http://rdfs.org/ns/void#numberOfResources> .
    _:b2 <http://purl.org/NET/scovo#dimension> dcterms:Agent .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#statItem> _:b3 .
    _:b3 rdf:value ?concepts .
    _:b3 <http://purl.org/NET/scovo#dimension> <http://rdfs.org/ns/void#numberOfResources> .
    _:b3 <http://purl.org/NET/scovo#dimension> <http://www.w3.org/2004/02/skos/core#Concept> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#statItem> _:b4 .
    _:b4 rdf:value ?geos .
    _:b4 <http://purl.org/NET/scovo#dimension> <http://rdfs.org/ns/void#numberOfResources> .
    _:b4 <http://purl.org/NET/scovo#dimension> dcterms:Location .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#statItem> _:b5 .
    _:b5 rdf:value ?triples .
    _:b5 <http://purl.org/NET/scovo#dimension> <http://rdfs.org/ns/void#numberOfTriples> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#statItem> _:b6 .
    _:b6 rdf:value ?subjects .
    _:b6 <http://purl.org/NET/scovo#dimension> <http://rdfs.org/ns/void#numberOfDistinctSubjects> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#statItem> _:b7 .
    _:b7 rdf:value ?objects .
    _:b7 <http://purl.org/NET/scovo#dimension> <http://rdfs.org/ns/void#numberOfDistinctObjects> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#uriRegexPattern> "http://standaarden.overheid.nl/owms/terms/.+" .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#vocabulary> <http://www.w3.org/2008/05/skos-xl#> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#vocabulary> <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#vocabulary> <http://www.w3.org/2000/01/rdf-schema#> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#vocabulary> <http://xmlns.com/foaf/0.1/> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#vocabulary> <http://www.w3.org/2004/02/skos/core#> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#vocabulary> <http://purl.org/dc/terms/> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#vocabulary> xsd: .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#vocabulary> <http://standaarden.overheid.nl/owms/terms/> .
    <http://standaarden.overheid.nl/owms/terms/OWMSdataset> <http://rdfs.org/ns/void#vocabulary> <http://www.w3.org/2002/07/owl#> .
}

WHERE {
{SELECT ( count(?a) as ?geos )
WHERE {
?a a ?type.
?type rdfs:subClassOf+ dcterms:Location.}}
{SELECT ( count(?b) as ?agents )
WHERE {
?b a ?type.
?type rdfs:subClassOf+ dcterms:Agent.}}
{SELECT ( count(?c) as ?concepts )
WHERE {
?c a ?type.
?type rdfs:subClassOf+ skos:Concept.}}
{SELECT (count (*) as ?triples) (count(distinct ?a) as ?subjects) (count (distinct ?c) as ?objects)
WHERE
{
?a ?b ?c.
}}
}

 All   Comments   Change History      Sort Order:
Comment by Jeen Broekstra [15/Apr/12 03:45 AM]
I can not reproduce the error. When I execute this exact query on an (empty) repository, I get back a normal result.

Can you provide additional details about the error? What do the server logs say? Any stacktraces that might help me pinpoint the problem?

Can you attach a file with a minimal test dataset with which I can reproduce the error?

Comment by Paul Hermans [15/Apr/12 12:34 PM]
This is in the main.log file:

[ERROR] 2012-04-15 12:30:52,442 [repositories/OWMS] Error while handling request
java.lang.NullPointerException: null
at org.openrdf.query.algebra.UnaryValueOperator.hashCode(UnaryValueOperator.java:99)
at org.openrdf.query.algebra.Count.hashCode(Count.java:35)
at org.openrdf.query.algebra.GroupElem.hashCode(GroupElem.java:92)
at java.util.AbstractList.hashCode(AbstractList.java:527)
at org.openrdf.query.algebra.Group.hashCode(Group.java:150)
at org.openrdf.query.algebra.UnaryTupleOperator.hashCode(UnaryTupleOperator.java:104)
at org.openrdf.query.algebra.Extension.hashCode(Extension.java:123)
at org.openrdf.query.algebra.UnaryTupleOperator.hashCode(UnaryTupleOperator.java:104)
at org.openrdf.query.algebra.Projection.hashCode(Projection.java:98)
at java.util.HashMap.put(HashMap.java:372)
at org.openrdf.query.algebra.evaluation.impl.QueryJoinOptimizer$JoinVisitor.meet(QueryJoinOptimizer.java:89)
at org.openrdf.query.algebra.Join.visit(Join.java:51)
at org.openrdf.query.algebra.UnaryTupleOperator.visitChildren(UnaryTupleOperator.java:79)
at org.openrdf.query.algebra.Extension.visitChildren(Extension.java:101)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meetNode(QueryModelVisitorBase.java:633)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meetUnaryTupleOperator(QueryModelVisitorBase.java:534)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meet(QueryModelVisitorBase.java:194)
at org.openrdf.query.algebra.Extension.visit(Extension.java:90)
at org.openrdf.query.algebra.UnaryTupleOperator.visitChildren(UnaryTupleOperator.java:79)
at org.openrdf.query.algebra.MultiProjection.visitChildren(MultiProjection.java:112)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meetNode(QueryModelVisitorBase.java:633)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meetUnaryTupleOperator(QueryModelVisitorBase.java:534)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meet(QueryModelVisitorBase.java:344)
at org.openrdf.query.algebra.MultiProjection.visit(MultiProjection.java:101)
at org.openrdf.query.algebra.UnaryTupleOperator.visitChildren(UnaryTupleOperator.java:79)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meetNode(QueryModelVisitorBase.java:633)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meetUnaryTupleOperator(QueryModelVisitorBase.java:534)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meet(QueryModelVisitorBase.java:404)
at org.openrdf.query.algebra.Reduced.visit(Reduced.java:28)
at org.openrdf.query.algebra.UnaryTupleOperator.visitChildren(UnaryTupleOperator.java:79)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meetNode(QueryModelVisitorBase.java:633)
at org.openrdf.query.algebra.helpers.QueryModelVisitorBase.meet(QueryModelVisitorBase.java:398)
at org.openrdf.query.algebra.QueryRoot.visit(QueryRoot.java:34)
at org.openrdf.query.algebra.evaluation.impl.QueryJoinOptimizer.optimize(QueryJoinOptimizer.java:52)
at org.openrdf.sail.memory.MemoryStoreConnection.evaluateInternal(MemoryStoreConnection.java:136)
at org.openrdf.sail.helpers.SailConnectionBase.evaluate(SailConnectionBase.java:230)
at org.openrdf.sail.helpers.SailConnectionWrapper.evaluate(SailConnectionWrapper.java:82)
at org.openrdf.sail.inferencer.InferencerConnectionWrapper.evaluate(InferencerConnectionWrapper.java:103)
at org.openrdf.repository.sail.SailGraphQuery.evaluate(SailGraphQuery.java:53)
at org.openrdf.http.server.repository.RepositoryController.handleRequestInternal(RepositoryController.java:124)
at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:680)

Comment by Paul Hermans [15/Apr/12 12:35 PM]
The small dataset with which it happens.

Comment by Paul Hermans [15/Apr/12 12:38 PM]
The enviornment in which it is happening:

- apache-tomcat-6.0.32 on Mac OS X Lion 10.7.3 (11D50)
apache-tomcat started with -XX:MaxPermSize=1024m

Comment by Paul Hermans [15/Apr/12 12:46 PM]
OWMS is an in-memory store (RDF schema) with persistence.

Comment by Paul Hermans [15/Apr/12 02:09 PM]
It works in tomcat apache-tomcat-7.0.14.


Comment by Jeen Broekstra [17/Apr/12 05:16 AM]
Thanks, that stacktrace is helpful. Rather peculiar that the Apache Tomcat version seems to matter.

Comment by Jeen Broekstra [09/May/12 07:52 AM]
Query works when executed directly on an in memory store directly from console, so indeed the issue seems to be server or workbench application.

Comment by Jeen Broekstra [09/May/12 08:10 AM]
Slight amendment. Query seems to have significant performance issues when executed on an inferencing store. this occurs in the console when run directly on a store as well as when running from workbench against a server running on tomcat.

Will try to simplify query to isolate the cause.

Comment by Jeen Broekstra [09/May/12 09:40 AM]
I have noticed that the query which you execute seems to have a lot of redundancies in it. Especially given that you are executing this on a repository which supports RDFS inferencing.

For example, this subquery:

 SELECT ( count(?a) as ?geos ) WHERE { ?a a ?type. ?type rdfs:subClassOf+ dcterms:Location. }

can be simplified to:

 SELECT ( count(?a) as ?geos ) WHERE { ?a a dcterms:Location. }

The original subquery makes sense on a store that does not support RDFS inferencing, since we then need the property path to explore the transitive subclassof relation. But in an RDFS-inferencing store, transitivity is supported out of the box, so we don't need to explicitly query for it.

Of course, regardless, the query should perform better than it does, I will continue to look into the problem, but I thought I'd point this out.

Comment by Jeen Broekstra [09/May/12 11:03 PM]
The cause of the observed performance issue was a bug in the cycle detection for property paths having a fixed start and end value and iterating over a path that is both transitive and reflexive (e.g. a subclassof hierarchy). See SES-993. This bug has been fixed and the query now seems to evaluate quickly and correctly, at least from the command line. Will doublecheck behavior in Tomcat setting.

Comment by Jeen Broekstra [09/May/12 11:12 PM]
Query now executes quickly and correctly in reported environment (Tomcat 6, in memory store with inferencing).

The fix will be included in the upcoming 2.6.6 release and in the next snapshot build.