[infinispan-dev] Appending to file

[Galder Zamarreño]

On Jul 3, 2013, at 10:20 AM, Radim Vansa <rvansa at redhat.com> wrote:

> | > 
> | > I've used three implementations: one simply synchronizing the access and
> | > calling force(false) after each write (by default 1kB). Second with
> | > threads cooperating - every thread puts its data into queue and waits for
> | > a short period of time - if then its data are still in the queue, it
> | > writes whole queue to disk, flushes it and wakes up other waiting threads.
> | > Third implementation (actually three flavours) used one spooler thread
> | > which polls the queue, writes as much as it can to disk, flushes and
> | > notifies waiting threads.
> | 
> | ^ Hmmm, aren't option 2 and 3 different flavours of the async store rather
> | than the file cache store? IOW, we want the file cache store to behave in
> | such way that it provides "reasonable" guarantees where store() finishes. We
> | are aware of different guarantees been given depending on whether force is
> | called or not, but adding queue/threads is something that's responsibility
> | of the async store. If you really wanna compare apples with apples, you
> | should do be comaring the performance of:
> | 
> | A) async store enabled + option 1 (sync force call)
> | B) option 2 (threads cooperating)
> | C) option 3 (spooler)
> | 
> | We've done a lot of work to improve the performance of the async store and
> | we're happy with its current performance numbers. Karsten helped hugely with
> | that, so I really have doubts any file cache store implementation should be
> | reimplementing async store-like logic.
> | 
> 
> Maybe I am missing something, but async store (aka write-behind) does not wait until the entry is written to the disk (and possibly flushed). The aim of these two strategies is to reduce number of flushes by collating stores into batch and flushing only once after several stores, but the storing threads are not released until the flush happens.

Also, how do the numbers you got with your implementations compared with running the same tests with Karten's cache stores, or LevelDB JNI?

> 
> Radim
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarreño
galder at redhat.com
twitter.com/galderz

Project Lead, Escalante
http://escalante.io

Engineer, Infinispan
http://infinispan.org