Re: [infinispan-dev] LevelDBStore's expiryEntryQueue
On Fri, Jul 4, 2014 at 6:09 PM, Pedro Ruivo <pedro(a)infinispan.org> wrote:
Hi guys,
Is there a way to replace the expiryEntryQueue with a non-blocking
structure?
Did you try configuring levelDBStore.expiryQueueSize = MAX_INT?
Long history:
In high throughput systems, this queue gets full very often and it is
blocking all the writes (that throws timeout exceptions everywhere).
Also, I don't full understand why this queue exists. It is drain in
purgeExpired and it deletes the keys expired, but the not-expires keys
are never tested (or I am missing something)
Can someone explains?
AFAICT the idea is to store the expiration time of each key in a separate
DB, so that purging the expired entries doesn't require reading all the
values from the regular DB. The regular DB still contains the expiration
time, so the regular expiration check still works.
Note that the purge method iterates over all the entries in the expired DB,
not just the entries in the expiration queue.
The expiration queue is needed so that we don't have 2 sync LevelDB writes
for every Infinispan write. But I don't see any reason for it to be tied to
the purge() method - instead, another thread should constantly read entries
out of the expiration queue and write them to the expiration DB.
Cheers
Dan
Attachments:
- attachment.html (text/html — 2.2 KB)