[infinispan-dev] user concerned about lack of "high performance" CacheLoaders

[Sanne Grinovero]

On 24 October 2011 11:55, Mircea Markus <mircea.markus at jboss.com> wrote:
>
> On 20 Oct 2011, at 16:18, Sanne Grinovero wrote:
>
>> Via twitter and Hibernate forums, @astralbodies seems to be doing some
>> interesting high performance *writing* stuff with Hibernate Search,
>> but in his experience Hibernate Search on disk is pretty fast, not so
>> the Search + Infinispan combination.
>>
>> http://twitter.com/#!/practicallyUX/statuses/126774913884360704
>>
>> In my own tests, we are quite faster than Lucene's FSDirectory even
>> while this uses MMap and NIO tricks, but indeed writing to a sync
>> CacheLoader will slow it down significantly, and since the performance
>> battle was won by a small margin only (close to 3X);
> 3X doesn't sound like a "small margin" to me :-)

No not a very small marin in absolute terms, but smaller than what I
would hope for when moving my stuff from a rotating disk to an
in-memory grid.
It's a good enough improvement to support some decisions to migrate to
Infinispan, but considering all the added work, complexity and risks
some might not agree. I hope we can reach for some more impressive
differences at the next round of Lucene developments.

> As a performance note, versions before and including 5.0 forced a long-lasting* WL acquisition in CacheLoaderInterceptor on keys that were not in memory. This was forced even for *reads*, and this might have a big impact on performance. This locking was not needed and dropped in 5.1.BETA1. It would be interesting to see the results with 5.1.

That's very good news! You should advertise such changes at least
dropping a line here; indeed you might remember me adding SKIP_LOCK in
many places to prevent this, not sure if I was skipping the ones which
where needed for cacheloaders consistency.. we'll see.

Cheers,
Sanne