On Fri, Jul 4, 2014 at 6:09 PM, Pedro Ruivo <pedro@infinispan.org> wrote:

Hi guys,

Is there a way to replace the expiryEntryQueue with a non-blocking
structure?

Did you try configuring levelDBStore.expiryQueueSize = MAX_INT?

Long history:

In high throughput systems, this queue gets full very often and it is
blocking all the writes (that throws timeout exceptions everywhere).

Also, I don't full understand why this queue exists. It is drain in
purgeExpired and it deletes the keys expired, but the not-expires keys
are never tested (or I am missing something)

Can someone explains?

AFAICT the idea is to store the expiration time of each key in a separate DB, so that purging the expired entries doesn't require reading all the values from the regular DB. The regular DB still contains the expiration time, so the regular expiration check still works.

Note that the purge method iterates over all the entries in the expired DB, not just the entries in the expiration queue.

The expiration queue is needed so that we don't have 2 sync LevelDB writes for every Infinispan write. But I don't see any reason for it to be tied to the purge() method - instead, another thread should constantly read entries out of the expiration queue and write them to the expiration DB.

Cheers

Dan