[#SES-638] MemoryStore's clean-up process takes a very long time

Issue Details [XML]

Key:	SES-638
Type:	Bug
Status:	Closed
Resolution:	Fixed
Priority:	Major
Assignee:	Arjohn Kampman
Reporter:	Arjohn Kampman
Votes:	0
Watchers:	0

Operations

If you were logged in you would be able to see more operations.

Sesame

MemoryStore's clean-up process takes a very long time

Created: 18/Nov/08 12:15 PM Updated: 17/Dec/08 02:05 PM

Component/s:

Memory Sail

Affects Version/s:

2.2.1, 2.1.4, 2.2, 2.1.3, 2.1.2, 2.1.1, 2.1, 2.0.1, 2.0

Fix Version/s:

2.2.2

Description

The MemoryStore's clean-up process takes a long time to complete when some 20% of 2.5 million statements are removed. The time to complete this operation is in the order of 10 minutes. During this operation, the clean-up thread holds a write lock so that no other read or write operations are possible during this time. To remedy this, the clean-up process should either be sped up considerably, or it should release the write lock on a regular basis.

All

Comments

Change History

Sort Order:

[ Permlink ]

Comment by Arjohn Kampman [20/Nov/08 01:27 PM]

Performance has further improved. When removing 20% from store with 2.5m triples, it takes some 10s to clean-up the deprecated statements, where this used to be 11 minutes.

[ Permlink ]

Comment by Arjohn Kampman [19/Nov/08 07:46 PM]

Performance still isn't good in all cases.

[ Permlink ]

Comment by Arjohn Kampman [18/Nov/08 04:40 PM]

The clean-up process had a huge bottle neck that came down to random removals from potentially long lists, which needed to be scanned in sequence for matchings statements. The algorithm has been improved to prevent these random removals.