FYI - here's a summary of EAP 6.0 vs 6.1 performance: https://bugzilla.redhat.com/attachment.cgi?id=741896 and here's the relevant BZ: https://bugzilla.redhat.com/show_bug.cgi?id=956988 On Wed, 2013-05-01 at 15:55 +0100, Mircea Markus wrote: > Hi Martin, > > Paul mentioned a severe degradation in performance between 5.1 and 5.2 for replicated + embedded + transactional + pessimistic caches, for write-heavy access. Do we have any tests we can run to check this? > > Cheers,

Hi, I've checked it on 4 nodes, stress test 15 minutes with 80% writes. Both reads and writes have improved in 5.2 READS/sec 247k -> 263k WRITES/sec 4547 -> 4771 (note: this is average, coordinator has about 2x more writes as it is the lock owner for all locks in replicated mode) that makes Paul, how many operations within transaction do you use? The results above are from single operation (put/get) per transaction (we have improvement in TRANSACTIONS/sec 5655 -> 5936 with this setting), I'll try to rerun with more ops/transaction as well. Radim ----- Original Message ----- | From: "Paul Ferraro" <paul.ferraro(a)redhat.com&gt; | To: "Mircea Markus" <mmarkus(a)redhat.com&gt; | Cc: "infinispan" <infinispan-dev(a)lists.jboss.org&gt;, "Paul Ferraro" <pferraro(a)redhat.com&gt; | Sent: Wednesday, May 1, 2013 5:05:39 PM | Subject: Re: [infinispan-dev] write-heavy performance degradation between 5.1 and 5.2 | | FYI - here's a summary of EAP 6.0 vs 6.1 performance: | https://bugzilla.redhat.com/attachment.cgi?id=741896 | | and here's the relevant BZ: | https://bugzilla.redhat.com/show_bug.cgi?id=956988 | | On Wed, 2013-05-01 at 15:55 +0100, Mircea Markus wrote: | > Hi Martin, | > | > Paul mentioned a severe degradation in performance between 5.1 and 5.2 for | > replicated + embedded + transactional + pessimistic caches, for | > write-heavy access. Do we have any tests we can run to check this? | > | > Cheers, | | | _______________________________________________ | infinispan-dev mailing list | infinispan-dev(a)lists.jboss.org | https://lists.jboss.org/mailman/listinfo/infinispan-dev | _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Kind of....their configuration is replicated & embedded cache + async replication + batching + file store + passivation off + write-heavy We, indeed, do *not* test this configuration for regression (especially the batching + file store + write-heavy). And even if we did, we wouldn't spot the problem easily. The performance regression was revealed in a client stress test where the number of clients is gradually increased. The peak throughput in their tests was higher for 5.2 than for 5.1 - this was OK. They complained about the max. number of clients that can be handled and the throughput after the peak (not before it). And this has to do with JGroups tuning (thread pools etc.) This is a very specific test case and issue and can be revealed only by the client stress tests which are the most time consuming. Anyway, we'll try to incorporate this scenario into the regular regression performance testing we're working on so that we catch these problems sooner than the respective version of ISPN gets to AS. Martin Dne 2.5.2013 19:43, Galder Zamarreño napsal(a): > My gut feeling is: AS uses some cache configuration that's affecting performance and it's not currently in the test plan? > > Cheers, > > On May 2, 2013, at 12:41 PM, Radim Vansa <rvansa(a)redhat.com&gt; wrote: > >> Hi, I've checked it on 4 nodes, stress test 15 minutes with 80% writes. Both reads and writes have improved in 5.2 >> >> READS/sec 247k -> 263k >> WRITES/sec 4547 -> 4771 (note: this is average, coordinator has about 2x more writes as it is the lock owner for all locks in replicated mode) >> that makes >> >> Paul, how many operations within transaction do you use? The results above are from single operation (put/get) per transaction (we have improvement in TRANSACTIONS/sec 5655 -> 5936 with this setting), I'll try to rerun with more ops/transaction as well. >> >> Radim >> >> ----- Original Message ----- >> | From: "Paul Ferraro" <paul.ferraro(a)redhat.com&gt; >> | To: "Mircea Markus" <mmarkus(a)redhat.com&gt; >> | Cc: "infinispan" <infinispan-dev(a)lists.jboss.org&gt;, "Paul Ferraro" <pferraro(a)redhat.com&gt; >> | Sent: Wednesday, May 1, 2013 5:05:39 PM >> | Subject: Re: [infinispan-dev] write-heavy performance degradation between 5.1 and 5.2 >> | >> | FYI - here's a summary of EAP 6.0 vs 6.1 performance: >> | https://bugzilla.redhat.com/attachment.cgi?id=741896 >> | >> | and here's the relevant BZ: >> | https://bugzilla.redhat.com/show_bug.cgi?id=956988 >> | >> | On Wed, 2013-05-01 at 15:55 +0100, Mircea Markus wrote: >> | > Hi Martin, >> | > >> | > Paul mentioned a severe degradation in performance between 5.1 and 5.2 for >> | > replicated + embedded + transactional + pessimistic caches, for >> | > write-heavy access. Do we have any tests we can run to check this? >> | > >> | > Cheers, >> | >> | >> | _______________________________________________ >> | infinispan-dev mailing list >> | infinispan-dev(a)lists.jboss.org >> | https://lists.jboss.org/mailman/listinfo/infinispan-dev >> | >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Galder Zamarreño > galder(a)redhat.com > twitter.com/galderz > > Project Lead, Escalante > http://escalante.io > > Engineer, Infinispan > http://infinispan.org > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev