On 24 Jun 2013, at 13:56, Radim Vansa <rvansa(a)redhat.com> wrote:
----- Original Message -----
| From: "Galder Zamarreño" <galder(a)redhat.com>
| To: "Radim Vansa" <rvansa(a)redhat.com>
| Cc: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org>
| Sent: Monday, June 24, 2013 12:46:52 PM
| Subject: Re: [infinispan-internal] LevelDB performance testing
|
| Putting Infinispan-Development list again to get others' thoughts...
|
| On Jun 24, 2013, at 11:57 AM, Radim Vansa <rvansa(a)redhat.com> wrote:
|
| >
| >
| > ----- Original Message -----
| > | From: "Galder Zamarreño" <galder(a)redhat.com>
| > | To: "Radim Vansa" <rvansa(a)redhat.com>
| > | Sent: Monday, June 24, 2013 10:45:32 AM
| > | Subject: Re: [infinispan-internal] LevelDB performance testing
| > |
| > | Hey Radim,
| > |
| > | Thanks a lot for running these tests. Comments inline...
| > |
| > | On Jun 21, 2013, at 3:33 PM, Radim Vansa <rvansa(a)redhat.com> wrote:
| > |
| > | > Hi all,
| > | >
| > | > I've got some results from LevelDB performance testing.
| > | >
| > | > Use case description:
| > | > - writes with unique keys about 100 bytes in size, no value.
| > | > - reads rare
| > | >
| > | > Infinispan configuration:
| > | > - eviction enabled (no passivation), 10000 entries is memory allowed
| > | > (we
| > | > don't need to occupy the memory if almost all operations are writes to
| > | > unique keys)
| > | > - synchronous mode (test have not revealed any significant performance
| > | > gain
| > | > using async mode) - distributed cache with 2 owners and 40 segments
| > | > - write-behind cache store with 4 threads
| > |
| > | ^ Hmmmm, why write-behind? If you really wanna test the performance of
| > | LevelDB, and you want to check how fast it writes or deletes, how are you
| > | gonna measure that if you're using an async store? Read operations,
| > | assuming
| > | that data is evicted, will go all the way to the cache store, and that's
| > | really the only numbers you can get out this test…
| >
| > Because I want to handle stuff ASAP. I don't really care how long each
| > operation takes, I am just interested in the throughput I can reach. And
| > this should be best with write-behind, because we're not waiting for the
| > data to be written down, right?
|
| ^ Indeed, but then not sure you're really getting throughput numbers based on
| what the cache store implementation. In other words, with this test you're
| testing the async store logic, to see how much throughput you can handle
| queuing up those modifications… unless you have a validation phase that
| times how long it takes for the cache store to contain all the data that is
| supposed to have. Then yes, you'd be measuring how long it takes for all
| that queued up modifications to be applied, which is a factor or many
| things, one of which, is the speed of the cache store implementation to
| apply those changes at the persistence layer.
I may be wrong, but isn't the async-processing queue somehow bounded?
the
working queue of the AsyncStore is an unbounded LinkedBlockingQueue.
I've assumed that if we write too fast, the requests start to be
discarded eventually, showing some errors in the error log.
yep, I think the
underlaying store cannot keep the pace with the amount of writes it receives.
In a long test, this would prove as the validation that the data have
really been written to the persistence layer. Or is this queue simply unbounded?
Cheers,
--
Mircea Markus
Infinispan lead (
www.infinispan.org)