[infinispan-dev] [infinispan-internal] LevelDB performance testing

[Mircea Markus]

On 24 Jun 2013, at 13:56, Radim Vansa <rvansa at redhat.com> wrote:

> ----- Original Message -----
> | From: "Galder Zamarreño" <galder at redhat.com>
> | To: "Radim Vansa" <rvansa at redhat.com>
> | Cc: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> | Sent: Monday, June 24, 2013 12:46:52 PM
> | Subject: Re: [infinispan-internal] LevelDB performance testing
> | 
> | Putting Infinispan-Development list again to get others' thoughts...
> | 
> | On Jun 24, 2013, at 11:57 AM, Radim Vansa <rvansa at redhat.com> wrote:
> | 
> | > 
> | > 
> | > ----- Original Message -----
> | > | From: "Galder Zamarreño" <galder at redhat.com>
> | > | To: "Radim Vansa" <rvansa at redhat.com>
> | > | Sent: Monday, June 24, 2013 10:45:32 AM
> | > | Subject: Re: [infinispan-internal] LevelDB performance testing
> | > | 
> | > | Hey Radim,
> | > | 
> | > | Thanks a lot for running these tests. Comments inline...
> | > | 
> | > | On Jun 21, 2013, at 3:33 PM, Radim Vansa <rvansa at redhat.com> wrote:
> | > | 
> | > | > Hi all,
> | > | > 
> | > | > I've got some results from LevelDB performance testing.
> | > | > 
> | > | > Use case description:
> | > | > - writes with unique keys about 100 bytes in size, no value.
> | > | > - reads rare
> | > | > 
> | > | > Infinispan configuration:
> | > | > - eviction enabled (no passivation), 10000 entries is memory allowed
> | > | > (we
> | > | > don't need to occupy the memory if almost all operations are writes to
> | > | > unique keys)
> | > | > - synchronous mode (test have not revealed any significant performance
> | > | > gain
> | > | > using async mode) - distributed cache with 2 owners and 40 segments
> | > | > - write-behind cache store with 4 threads
> | > | 
> | > | ^ Hmmmm, why write-behind? If you really wanna test the performance of
> | > | LevelDB, and you want to check how fast it writes or deletes, how are you
> | > | gonna measure that if you're using an async store? Read operations,
> | > | assuming
> | > | that data is evicted, will go all the way to the cache store, and that's
> | > | really the only numbers you can get out this test…
> | > 
> | > Because I want to handle stuff ASAP. I don't really care how long each
> | > operation takes, I am just interested in the throughput I can reach. And
> | > this should be best with write-behind, because we're not waiting for the
> | > data to be written down, right?
> | 
> | ^ Indeed, but then not sure you're really getting throughput numbers based on
> | what the cache store implementation. In other words, with this test you're
> | testing the async store logic, to see how much throughput you can handle
> | queuing up those modifications… unless you have a validation phase that
> | times how long it takes for the cache store to contain all the data that is
> | supposed to have. Then yes, you'd be measuring how long it takes for all
> | that queued up modifications to be applied, which is a factor or many
> | things, one of which, is the speed of the cache store implementation to
> | apply those changes at the persistence layer.
> 
> I may be wrong, but isn't the async-processing queue somehow bounded?
the working queue of the AsyncStore is an unbounded LinkedBlockingQueue.
> I've assumed that if we write too fast, the requests start to be discarded eventually, showing some errors in the error log.
yep, I think the underlaying store cannot keep the pace with the amount of writes it receives.
> In a long test, this would prove as the validation that the data have really been written to the persistence layer. Or is this queue simply unbounded?

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)