[#SES-724] OutOfMemoryError in NativeStore with large values.hash files on 32-bit platforms - openRDF.org Issue Tracker

Issue Details [XML]

Key:	SES-724
Type:	Bug
Status:	Open
Priority:	Major
Assignee:	Arjohn Kampman
Reporter:	Arjohn Kampman
Votes:	0
Watchers:	0

Operations

If you were logged in you would be able to see more operations.

Sesame

OutOfMemoryError in NativeStore with large values.hash files on 32-bit platforms

Created: 13/Dec/10 10:32 AM Updated: 24/Aug/11 09:00 AM

Component/s:

Native Sail

Affects Version/s:

2.3.2

Fix Version/s:

None

Description

Due to http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6822107 the HashFile used by Sesame's NativeStore can run into an OutOfMemoryError when it needs to grow. This happens when there's insufficient memory address space to store the hash file's overflow buckets. This problem has been seen on 32-bit platforms where a considerable part of the ~2GB of address space is used for the java heap. The relevant part of the stack trace looks like this:

java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:761)
at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:450)
at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:523)
at org.openrdf.sail.nativerdf.datastore.HashFile.increaseHashTable(HashFile.java:405)
at org.openrdf.sail.nativerdf.datastore.HashFile.storeID(HashFile.java:201)
at org.openrdf.sail.nativerdf.datastore.DataStore.storeData(DataStore.java:158)
at org.openrdf.sail.nativerdf.ValueStore.storeValue(ValueStore.java:319)
at org.openrdf.sail.nativerdf.NativeStoreConnection.addStatement(NativeStoreConnection.java:381)
... 24 common frames omitted
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:758)
... 31 common frames omitted

All

Comments

Change History

Sort Order:

Change by Arjohn Kampman [07/Apr/11 01:46 PM]
Field	Original Value	New Value
Fix Version/s		2.4.0 [ 10500 ]
Fix Version/s	2.3.3 [ 10470 ]

Change by Arjohn Kampman [03/May/11 08:47 PM]
Description	Due to http://bugs.sun.com/view_bug.do?bug_id=6431344 the HashFile used by Sesame's NativeStore can run into an OutOfMemoryError when it needs to grow. This happens when there's insufficient memory address space to store the hash file's overflow buckets. This problem has been seen on 32-bit platforms where a considerable part of the ~2GB of address space is used for the java heap. The relevant part of the stack trace looks like this: java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:761) at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:450) at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:523) at org.openrdf.sail.nativerdf.datastore.HashFile.increaseHashTable(HashFile.java:405) at org.openrdf.sail.nativerdf.datastore.HashFile.storeID(HashFile.java:201) at org.openrdf.sail.nativerdf.datastore.DataStore.storeData(DataStore.java:158) at org.openrdf.sail.nativerdf.ValueStore.storeValue(ValueStore.java:319) at org.openrdf.sail.nativerdf.NativeStoreConnection.addStatement(NativeStoreConnection.java:381) ... 24 common frames omitted Caused by: java.lang.OutOfMemoryError: Map failed at sun.nio.ch.FileChannelImpl.map0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:758) ... 31 common frames omitted	Due to http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6822107 the HashFile used by Sesame's NativeStore can run into an OutOfMemoryError when it needs to grow. This happens when there's insufficient memory address space to store the hash file's overflow buckets. This problem has been seen on 32-bit platforms where a considerable part of the ~2GB of address space is used for the java heap. The relevant part of the stack trace looks like this: java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:761) at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:450) at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:523) at org.openrdf.sail.nativerdf.datastore.HashFile.increaseHashTable(HashFile.java:405) at org.openrdf.sail.nativerdf.datastore.HashFile.storeID(HashFile.java:201) at org.openrdf.sail.nativerdf.datastore.DataStore.storeData(DataStore.java:158) at org.openrdf.sail.nativerdf.ValueStore.storeValue(ValueStore.java:319) at org.openrdf.sail.nativerdf.NativeStoreConnection.addStatement(NativeStoreConnection.java:381) ... 24 common frames omitted Caused by: java.lang.OutOfMemoryError: Map failed at sun.nio.ch.FileChannelImpl.map0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:758) ... 31 common frames omitted

Change by Arjohn Kampman [13/May/11 08:45 PM]
Fix Version/s		2.4.1 [ 10540 ]
Fix Version/s	2.4.0 [ 10500 ]

Change by Arjohn Kampman [12/Jul/11 10:43 AM]
Fix Version/s		2.4.2 [ 10570 ]
Fix Version/s	2.4.1 [ 10540 ]

Change by Arjohn Kampman [13/Jul/11 02:18 PM]
Fix Version/s		2.4.3 [ 10580 ]
Fix Version/s	2.4.2 [ 10570 ]

Change by Arjohn Kampman [24/Aug/11 09:00 AM]
Fix Version/s	2.4.3 [ 10580 ]