From dan.berindei at gmail.com Sat Oct 1 05:37:41 2016 From: dan.berindei at gmail.com (Dan Berindei) Date: Sat, 1 Oct 2016 12:37:41 +0300 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: <57EE8FF3.9020303@redhat.com> References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> Message-ID: Dennis, this has been done in Infinispan ever since 4.0, so feel free to open a BZ :) Cheers Dan On Fri, Sep 30, 2016 at 7:16 PM, Dennis Reed wrote: > As Wolf noted, caching the trace flag is bad when trying to debug issues. > > Don't do it! It's not worth breaking the logging semantics for a > nano-second level performance difference. (if your trace is being > called enough for that tiny impact to make any real difference, that > trace logging is going to be WAY too verbose to be of any use anyways). > > If I see it done, I'm going to open a BZ. > > -Dennis > > > On 09/30/2016 08:23 AM, Sanne Grinovero wrote: >> this discussion appears on this mailing list approximately every 2 years :) >> >> On 30 September 2016 at 13:41, Dan Berindei wrote: >>> I should stress that we only cache `isTraceEnabled()` in a static >>> field. Debug logging can still be enabled or disabled on the fly. >>> >>> >>> Cheers >>> Dan >>> >>> >>> On Fri, Sep 30, 2016 at 3:14 PM, Wolf Fink wrote: >>>> Ok, >>>> >>>> thanks for clarifying it. >>>> >>>> So there is a factor of 3 for the test if no trace is enabled, just for >>>> checking. >>>> It makes sense to use it. >>>> But my concern is still that it is sometimes good to have the possibility to >>>> enable debug for some important information in production just on the fly >>>> and switch it of to prevent from throtteling the server by that log >>>> statements or restart the server. >>>> We have the same issue in EAP but here a restart is not that bad as here you >>>> don't have to load the cache or rebalance the cluster for stop/start. >>>> >>>> - Wolf >>>> >>>> On Fri, Sep 30, 2016 at 1:53 PM, David M. Lloyd >>>> wrote: >>>>> >>>>> On 09/30/2016 01:53 AM, Sebastian Laskawiec wrote: >>>>>> Hey! >>>>>> >>>>>> A while ago I asked Radim and Dan about these kind of constructs [1]: >>>>>> >>>>>> private boolean trace = logger.isTraceEnabled(); //stored in a field >>>>>> >>>>>> ... called in some method ... >>>>>> if(trace) >>>>>> logger.tracef(...); >>>>>> ... >>>>>> >>>>>> At first they seemed wrong to me, because if one changes logging level >>>>>> (using JMX for example), the code won't notice it. I also though it's >>>>>> quite ok to use tracef directly, because JIT will inline and optimize >>>>>> it. >>>>>> >>>>>> Unfortunately my benchmarks [2] show that I was wrong. Logger#tracef >>>>>> indeed checks if the logging level is enabled but since JBoss Logging >>>>>> may use different backends, the check is not trivial and is not inlined >>>>>> (at least with default settings). >>>>> >>>>> What backend where you using with your test? >>>>> >>>>>> The performance results look like this: >>>>>> Benchmark Mode Cnt Score Error >>>>>> Units >>>>>> MyBenchmark.noVariable thrpt 20 *717252060.124* ? 13420522.229 >>>>>> ops/s >>>>>> MyBenchmark.withVariable thrpt 20 *2358360244.627* ? 50214969.572 >>>>>> ops/s >>>>>> >>>>>> So if you even see a construct like this: logger.debuf or logger.tracef >>>>>> - make sure you check if the logging level is enabled (and the check >>>>>> result is stored in a field). >>>>>> >>>>>> That was a bit surprising and interesting lesson :D >>>>>> >>>>>> Thanks >>>>>> Sebastian >>>>>> >>>>>> [1] >>>>>> https://github.com/infinispan/infinispan/pull/4538#discussion_r80666086 >>>>>> [2] https://github.com/slaskawi/jboss-logging-perf-test >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> infinispan-dev mailing list >>>>>> infinispan-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> >>>>> >>>>> -- >>>>> - DML >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From dan.berindei at gmail.com Sat Oct 1 05:53:25 2016 From: dan.berindei at gmail.com (Dan Berindei) Date: Sat, 1 Oct 2016 12:53:25 +0300 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> Message-ID: On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd wrote: > The performance problem that this trick is meant to resolve is really a > problem in the logging backend. It *should* be faster inside of > WildFly, where JBoss LogManager is used, because that project just > checks a single volatile field for the level check... and the path to > that code *should* be inline-friendly. > Indeed, we started using this trick because of log4j 1.2, which needs to walk the logger hierarchy in order to check the level, and it made a significant difference there. Nowadays I think it's pretty close to optimal in all logging frameworks. The only nitpick is that they all use enums for the levels, and the JIT can't inline Level.TRACE.value as it would with a Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going to be more or less equivalent to using a volatile "trace" field in each class, so it should be suitable even for local read operations that take < 200ns. We'd probably still need to weed out some of the trace messages, as we probably have more than 10 of them during such a read operation. I confess that I added way too many trace logs myself, precisely because I knew we are using a static final field and the JIT compiler doesn't even generate code for that branch. Cheers Dan From slaskawi at redhat.com Mon Oct 3 01:32:49 2016 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 3 Oct 2016 07:32:49 +0200 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> Message-ID: Hey! Adding James to this thread. @Dennis - I think Dan has a point here. The trick with checking a field in a class is 3 times faster. Most of the checks are done in core so they are executed multiple times per operation. Changing all those places is probably not an option. @David - Let me run a test with JBoss Log Manager and get back to you with some results. But if Dan is right, and the problem is with enum mapping, I will get similar results. Thanks Sebastian On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei wrote: > On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd > wrote: > > The performance problem that this trick is meant to resolve is really a > > problem in the logging backend. It *should* be faster inside of > > WildFly, where JBoss LogManager is used, because that project just > > checks a single volatile field for the level check... and the path to > > that code *should* be inline-friendly. > > > > Indeed, we started using this trick because of log4j 1.2, which needs > to walk the logger hierarchy in order to check the level, and it made > a significant difference there. > > Nowadays I think it's pretty close to optimal in all logging > frameworks. The only nitpick is that they all use enums for the > levels, and the JIT can't inline Level.TRACE.value as it would with a > Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going to > be more or less equivalent to using a volatile "trace" field in each > class, so it should be suitable even for local read operations that > take < 200ns. > > We'd probably still need to weed out some of the trace messages, as we > probably have more than 10 of them during such a read operation. I > confess that I added way too many trace logs myself, precisely because > I knew we are using a static final field and the JIT compiler doesn't > even generate code for that branch. > > Cheers > Dan > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161003/ac48b09e/attachment.html From ttarrant at redhat.com Mon Oct 3 02:36:50 2016 From: ttarrant at redhat.com (Tristan Tarrant) Date: Mon, 3 Oct 2016 08:36:50 +0200 Subject: [infinispan-dev] Documentation mini-sprint Message-ID: <3f989e99-0bd6-adcb-57f3-34d49cff0d44@redhat.com> Hi all, between now and Wednesday please work on [1] Waiting for your PRs :) Tristan [1] https://github.com/infinispan/infinispan/blob/master/documentation/TODO.md -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From slaskawi at redhat.com Mon Oct 3 04:28:49 2016 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 3 Oct 2016 10:28:49 +0200 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> Message-ID: Hey! Please have a look at the latest perf test results [1][2]: Benchmark Mode Cnt Score Error Units MyBenchmark.noVariable thrpt 20 *681131269.875* ? 3961932.923 ops/s MyBenchmark.withIsTraceEnabledCheck thrpt 20 *676307984.921* ? 14305970.393 ops/s MyBenchmark.withVariable thrpt 20 *2411000894.582* ? 17382438.635 ops/s I think there is no surprise here.. using a field, which stores the result of `logger.isTraceEnabled()` evaluation is 3 times faster than other options. If anyone is interested in printing out JIT stuff, I also ran it with "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and "-XX:+PrintInlining" here [3]. I'm not a performance expert but it seems that JIT could not inline the "translate" method because of its size (see line 1861). However it tried several times with different optimizations (and some of them were thrown away - "made not entrant" messages [4]). Let's wait for James' opinion on this, but I think we should address this issue on JBoss Logging/LogManager side (so I agree with David here) and make those parts inlinable (wow, does this word even exist? :D). Once this is done, we could experiment further in Infinispan codebase and see how this relates to some real world benchmarks... Thanks Sebastian [1] https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a53c [2] Test repository: https://github.com/slaskawi/jboss-logging-perf-test [3] https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924bae0 [4] https://www.safaribooksonline.com/library/view/java-performance-the/9781449363512/ch04.html On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec wrote: > Hey! > > Adding James to this thread. > > @Dennis - I think Dan has a point here. The trick with checking a field in > a class is 3 times faster. Most of the checks are done in core so they are > executed multiple times per operation. Changing all those places is > probably not an option. > > @David - Let me run a test with JBoss Log Manager and get back to you with > some results. But if Dan is right, and the problem is with enum mapping, I > will get similar results. > > Thanks > Sebastian > > On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei > wrote: > >> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd >> wrote: >> > The performance problem that this trick is meant to resolve is really a >> > problem in the logging backend. It *should* be faster inside of >> > WildFly, where JBoss LogManager is used, because that project just >> > checks a single volatile field for the level check... and the path to >> > that code *should* be inline-friendly. >> > >> >> Indeed, we started using this trick because of log4j 1.2, which needs >> to walk the logger hierarchy in order to check the level, and it made >> a significant difference there. >> >> Nowadays I think it's pretty close to optimal in all logging >> frameworks. The only nitpick is that they all use enums for the >> levels, and the JIT can't inline Level.TRACE.value as it would with a >> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going to >> be more or less equivalent to using a volatile "trace" field in each >> class, so it should be suitable even for local read operations that >> take < 200ns. >> >> We'd probably still need to weed out some of the trace messages, as we >> probably have more than 10 of them during such a read operation. I >> confess that I added way too many trace logs myself, precisely because >> I knew we are using a static final field and the JIT compiler doesn't >> even generate code for that branch. >> >> Cheers >> Dan >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161003/5b3bad5c/attachment-0001.html From dan.berindei at gmail.com Mon Oct 3 08:40:49 2016 From: dan.berindei at gmail.com (Dan Berindei) Date: Mon, 3 Oct 2016 15:40:49 +0300 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> Message-ID: Hi Sebastian I modified your benchmark so that the logger and the trace field are static final and looked at the generated assembly with JITWatch [1]. Based on the attached assembly listings, caching isTraceEnabled() in a constant field is "infinitely faster", because there are no assembly instructions that can be traced back to the test method. JBossLogManagerLogger::translate() is inlined in this listing, but it still goes through the switch machinery, I'm guessing because the ordinal of an Enum is not considered a constant. Cheers Dan [1]: https://github.com/AdoptOpenJDK/jitwatch On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec wrote: > Hey! > > Please have a look at the latest perf test results [1][2]: > > Benchmark Mode Cnt Score > Error Units > MyBenchmark.noVariable thrpt 20 681131269.875 ? > 3961932.923 ops/s > MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 ? > 14305970.393 ops/s > MyBenchmark.withVariable thrpt 20 2411000894.582 ? > 17382438.635 ops/s > > I think there is no surprise here.. using a field, which stores the result > of `logger.isTraceEnabled()` evaluation is 3 times faster than other > options. > > If anyone is interested in printing out JIT stuff, I also ran it with > "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and "-XX:+PrintInlining" > here [3]. > > I'm not a performance expert but it seems that JIT could not inline the > "translate" method because of its size (see line 1861). However it tried > several times with different optimizations (and some of them were thrown > away - "made not entrant" messages [4]). > > Let's wait for James' opinion on this, but I think we should address this > issue on JBoss Logging/LogManager side (so I agree with David here) and make > those parts inlinable (wow, does this word even exist? :D). Once this is > done, we could experiment further in Infinispan codebase and see how this > relates to some real world benchmarks... > > Thanks > Sebastian > > [1] https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a53c > [2] Test repository: https://github.com/slaskawi/jboss-logging-perf-test > [3] https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924bae0 > [4] > https://www.safaribooksonline.com/library/view/java-performance-the/9781449363512/ch04.html > > On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec > wrote: >> >> Hey! >> >> Adding James to this thread. >> >> @Dennis - I think Dan has a point here. The trick with checking a field in >> a class is 3 times faster. Most of the checks are done in core so they are >> executed multiple times per operation. Changing all those places is probably >> not an option. >> >> @David - Let me run a test with JBoss Log Manager and get back to you with >> some results. But if Dan is right, and the problem is with enum mapping, I >> will get similar results. >> >> Thanks >> Sebastian >> >> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei >> wrote: >>> >>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd >>> wrote: >>> > The performance problem that this trick is meant to resolve is really a >>> > problem in the logging backend. It *should* be faster inside of >>> > WildFly, where JBoss LogManager is used, because that project just >>> > checks a single volatile field for the level check... and the path to >>> > that code *should* be inline-friendly. >>> > >>> >>> Indeed, we started using this trick because of log4j 1.2, which needs >>> to walk the logger hierarchy in order to check the level, and it made >>> a significant difference there. >>> >>> Nowadays I think it's pretty close to optimal in all logging >>> frameworks. The only nitpick is that they all use enums for the >>> levels, and the JIT can't inline Level.TRACE.value as it would with a >>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going to >>> be more or less equivalent to using a volatile "trace" field in each >>> class, so it should be suitable even for local read operations that >>> take < 200ns. >>> >>> We'd probably still need to weed out some of the trace messages, as we >>> probably have more than 10 of them during such a read operation. I >>> confess that I added way too many trace logs myself, precisely because >>> I knew we are using a static final field and the JIT compiler doesn't >>> even generate code for that branch. >>> >>> Cheers >>> Dan >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -------------- next part -------------- # {method} {0x00007f0033478a50} 'constantIsTraceEnabled_thrpt_jmhStub' '(Lorg/openjdk/jmh/runner/InfraControl;Lorg/openjdk/jmh/results/RawResults;Lorg/openjdk/jmh/infra/BenchmarkParams;Lorg/openjdk/jmh/infra/IterationParams;Lorg/openjdk/jmh/infra/ThreadParams;Lorg/openjdk/jmh/infra/Blackhole;Lorg/openjdk/jmh/infra/Control;ILorg/infinispan/generated/MyBenchmark_jmhType;)V' in 'org/infinispan/generated/MyBenchmark_constantIsTraceEnabled_jmhTest' 0x00007f00491d9160: call 0x00007f005e4cc0b0 ; {runtime_call} 0x00007f00491d9165: nop 0x00007f00491d9170: mov 0x00007f00491d9177: push rbp 0x00007f00491d9178: sub rsp,0x40 0x00007f00491d917c: mov rbx 0x00007f00491d917f: mov r13 0x00007f00491d9183: mov r10 0x00007f00491d9187: mov 0x00007f00491d918b: mov rbp 0x00007f00491d918f: mov r14 0x00007f00491d9193: mov rdi,rsi 0x00007f00491d9196: mov r10,0x7f005e56e9e0 0x00007f00491d91a0: call r10 0x00007f00491d91a3: mov r10 0x00007f00491d91a7: test r10,r10 0x00007f00491d91aa: je L0003 0x00007f00491d91b0: mov r11d 0x00007f00491d91b4: cmp r11d,0xf8013e3b ; {metadata('org/openjdk/jmh/runner/InfraControl')} 0x00007f00491d91bb: jne L0006 0x00007f00491d91c1: mov r11,r10 L0000: test rbp,rbp 0x00007f00491d91c7: je L0004 0x00007f00491d91cd: mov r8d 0x00007f00491d91d1: cmp r8d,0xf801403e ; {metadata('org/openjdk/jmh/results/RawResults')} 0x00007f00491d91d8: jne L0006 0x00007f00491d91de: mov r9,rbp L0001: mov r8d 0x00007f00491d91e5: cmp r8d,0xf801446d ; {metadata('org/infinispan/generated/MyBenchmark_jmhType')} 0x00007f00491d91ec: jne L0006 0x00007f00491d91f2: test r11,r11 0x00007f00491d91f5: je L0005 0x00007f00491d91f7: nop ; - org.infinispan.generated.MyBenchmark_constantIsTraceEnabled_jmhTest::constantIsTraceEnabled_thrpt_jmhStub at 15 (line 113) L0002: movzx r10d ; - org.infinispan.generated.MyBenchmark_constantIsTraceEnabled_jmhTest::constantIsTraceEnabled_thrpt_jmhStub at 25 (line 115) 0x00007f00491d9208: add r13,0x1 ; OopMap{r11=Oop r9=Oop r14=Oop off=172} ;*ifeq ; - org.infinispan.generated.MyBenchmark_constantIsTraceEnabled_jmhTest::constantIsTraceEnabled_thrpt_jmhStub at 28 (line 115) 0x00007f00491d920c: test ; {poll} *** SAFEPOINT POLL *** 0x00007f00491d9212: test r10d,r10d 0x00007f00491d9215: je L0002 ;*ifeq ; - org.infinispan.generated.MyBenchmark_constantIsTraceEnabled_jmhTest::constantIsTraceEnabled_thrpt_jmhStub at 28 (line 115) 0x00007f00491d9217: mov esi,0xffffff65 0x00007f00491d921c: mov rbp,r11 0x00007f00491d921f: mov 0x00007f00491d9223: mov 0x00007f00491d9228: mov 0x00007f00491d922d: mov 0x00007f00491d9232: mov 0x00007f00491d9237: call 0x00007f00490051a0 ; OopMap{rbp=Oop [0]=Oop [8]=Oop off=220} ;*ifeq ; - org.infinispan.generated.MyBenchmark_constantIsTraceEnabled_jmhTest::constantIsTraceEnabled_thrpt_jmhStub at 28 (line 115) ; {runtime_call} 0x00007f00491d923c: call 0x00007f005e4cc0b0 ;*ifeq ; - org.infinispan.generated.MyBenchmark_constantIsTraceEnabled_jmhTest::constantIsTraceEnabled_thrpt_jmhStub at 28 (line 115) ; {runtime_call} L0003: xor r11d,r11d 0x00007f00491d9244: jmp L0000 L0004: xor r9d,r9d 0x00007f00491d924c: jmp L0001 0x00007f00491d924e: xor r14d,r14d L0005: mov esi,0xffffff86 0x00007f00491d9256: mov rbp,r11 0x00007f00491d9259: mov 0x00007f00491d925d: mov 0x00007f00491d9262: mov 0x00007f00491d9267: mov 0x00007f00491d926c: xchg ax,ax 0x00007f00491d926f: call 0x00007f00490051a0 ; OopMap{rbp=Oop [0]=Oop [8]=Oop off=276} ;*aload ; - org.infinispan.generated.MyBenchmark_constantIsTraceEnabled_jmhTest::constantIsTraceEnabled_thrpt_jmhStub at 13 (line 113) ; {runtime_call} 0x00007f00491d9274: call 0x00007f005e4cc0b0 ; {runtime_call} L0006: mov esi,0xffffff9d 0x00007f00491d927e: mov 0x00007f00491d9282: mov 0x00007f00491d9287: mov 0x00007f00491d928c: mov 0x00007f00491d9291: xchg ax,ax 0x00007f00491d9293: call 0x00007f00490051a0 ; OopMap{rbp=Oop [0]=Oop [24]=Oop off=312} ;*aload ; - org.infinispan.generated.MyBenchmark_constantIsTraceEnabled_jmhTest::constantIsTraceEnabled_thrpt_jmhStub at 13 (line 113) ; {runtime_call} 0x00007f00491d9298: call 0x00007f005e4cc0b0 ;*aload ; - org.infinispan.generated.MyBenchmark_constantIsTraceEnabled_jmhTest::constantIsTraceEnabled_thrpt_jmhStub at 13 (line 113) ; {runtime_call} 0x00007f00491d929d: hlt 0x00007f00491d929e: hlt 0x00007f00491d929f: hlt [Exception Handler] [Stub Code] 0x00007f00491d92a0: jmp 0x00007f004906c760 ; {no_reloc} [Deopt Handler Code] 0x00007f00491d92a5: call 0x00007f00491d92aa 0x00007f00491d92aa: sub 0x00007f00491d92af: jmp 0x00007f00490473c0 ; {runtime_call} 0x00007f00491d92b4: hlt 0x00007f00491d92b5: hlt 0x00007f00491d92b6: hlt 0x00007f00491d92b7: hlt -------------- next part -------------- # {method} {0x00007f937f478a50} 'tracef_thrpt_jmhStub' '(Lorg/openjdk/jmh/runner/InfraControl;Lorg/openjdk/jmh/results/RawResults;Lorg/openjdk/jmh/infra/BenchmarkParams;Lorg/openjdk/jmh/infra/IterationParams;Lorg/openjdk/jmh/infra/ThreadParams;Lorg/openjdk/jmh/infra/Blackhole;Lorg/openjdk/jmh/infra/Control;ILorg/infinispan/generated/MyBenchmark_jmhType;)V' in 'org/infinispan/generated/MyBenchmark_tracef_jmhTest' # parm0: rsi:rsi = 'org/openjdk/jmh/runner/InfraControl' # parm1: rdx:rdx = 'org/openjdk/jmh/results/RawResults' # parm2: rcx:rcx = 'org/openjdk/jmh/infra/BenchmarkParams' # parm3: r8:r8 = 'org/openjdk/jmh/infra/IterationParams' # parm4: r9:r9 = 'org/openjdk/jmh/infra/ThreadParams' # parm5: rdi:rdi = 'org/openjdk/jmh/infra/Blackhole' # parm6: [sp+0x30] = 'org/openjdk/jmh/infra/Control' (sp of caller) # parm7: [sp+0x38] = int # parm8: [sp+0x40] = 'org/infinispan/generated/MyBenchmark_jmhType' 0x00007f93951e9fa0: mov 0x00007f93951e9fa7: push rbp 0x00007f93951e9fa8: sub rsp,0x20 ;*synchronization entry ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at -1 (line 109) 0x00007f93951e9fac: mov r13,rsi 0x00007f93951e9faf: mov rbx,rdx 0x00007f93951e9fb2: mov r10,0x7f93aa365ad0 0x00007f93951e9fbc: call r10 ;*invokestatic nanoTime ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 7 (line 111) 0x00007f93951e9fbf: mov ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 10 (line 111) ; implicit exception: dispatches to 0x00007f93951ea261 0x00007f93951e9fc3: mov r9 0x00007f93951e9fc8: test r9,r9 0x00007f93951e9fcb: je L0012 ;*invokevirtual tracef ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951e9fd1: mov rdx,0x76d4e9898 ; {oop(a 'org/jboss/logging/JBossLogManagerLogger')} 0x00007f93951e9fdb: mov r10d ;*getfield logger ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 1 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951e9fdf: mov rdi,0x76d4e6d48 ; {oop(a 'org/jboss/logging/Logger$Level')} 0x00007f93951e9fe9: mov ebp ; - java.lang.Enum::ordinal at 1 (line 103) ; - org.jboss.logging.JBossLogManagerLogger::translate at 8 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951e9fec: cmp ebp,0x6 0x00007f93951e9fef: jae L0013 0x00007f93951e9ff5: mov r11,0x76d4e9010 ; {oop([I)} 0x00007f93951e9fff: mov ecx ;*iaload ; - org.jboss.logging.JBossLogManagerLogger::translate at 11 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea004: cmp ecx,0x4 0x00007f93951ea007: je L0005 0x00007f93951ea009: cmp ecx,0x4 0x00007f93951ea00c: jle L0002 0x00007f93951ea00e: cmp ecx,0x6 0x00007f93951ea011: jne L0000 0x00007f93951ea013: mov r8,0x76d4bbc68 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea01d: jmp L0006 L0000: cmp ecx,0x6 0x00007f93951ea022: jg L0001 ;*tableswitch ; - org.jboss.logging.JBossLogManagerLogger::translate at 12 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea024: mov r8,0x76d4bbac8 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea02e: jmp L0006 ;*getstatic ALL ; - org.jboss.logging.JBossLogManagerLogger::translate at 76 (line 66) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) L0001: mov r8,0x76d444030 ; {oop(a 'java/util/logging/Level')} 0x00007f93951ea03a: jmp L0006 L0002: cmp ecx,0x2 0x00007f93951ea03f: je L0004 0x00007f93951ea041: cmp ecx,0x2 0x00007f93951ea044: jle L0003 0x00007f93951ea046: mov r8,0x76d4bb958 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea050: jmp L0006 L0003: cmp ecx,0x1 0x00007f93951ea055: jne L0001 ;*tableswitch ; - org.jboss.logging.JBossLogManagerLogger::translate at 12 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea057: mov r8,0x76d4bb6b8 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea061: jmp L0006 L0004: mov r8,0x76d4bb848 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea06d: jmp L0006 L0005: mov r8,0x76d4bba60 ;*invokestatic translate ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {oop(a 'org/jboss/logmanager/Level')} L0006: mov r10d ;*getfield loggerNode ; - org.jboss.logmanager.Logger::isLoggable at 1 (line 153) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; implicit exception: dispatches to 0x00007f93951ea275 0x00007f93951ea07e: mov r10d ;*getfield effectiveLevel ; - org.jboss.logmanager.LoggerNode::getEffectiveLevel at 1 (line 251) ; - org.jboss.logmanager.Logger::isLoggable at 4 (line 153) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; implicit exception: dispatches to 0x00007f93951ea289 0x00007f93951ea083: mov ebp ; - java.util.logging.Level::intValue at 1 (line 398) ; - org.jboss.logmanager.Logger::isLoggable at 9 (line 154) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea087: cmp ebp,r10d 0x00007f93951ea08a: jge L0014 ;*if_icmplt ; - org.jboss.logmanager.Logger::isLoggable at 13 (line 154) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea090: mov rcx,r13 0x00007f93951ea093: movzx r10d ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 25 (line 115) ; implicit exception: dispatches to 0x00007f93951ea299 0x00007f93951ea09b: mov ebp,0x1 0x00007f93951ea0a0: test r10d,r10d 0x00007f93951ea0a3: je L000a ;*aload_1 ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 31 (line 116) L0007: mov r10,0x7f93aa365ad0 0x00007f93951ea0af: call r10 ;*invokestatic nanoTime ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 32 (line 116) 0x00007f93951ea0b2: mov ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 47 (line 118) 0x00007f93951ea0b6: mov ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 35 (line 116) 0x00007f93951ea0ba: mov ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea0be: add rsp,0x20 0x00007f93951ea0c2: pop rbp 0x00007f93951ea0c3: test ; {poll_return} *** SAFEPOINT POLL *** 0x00007f93951ea0c9: ret 0x00007f93951ea0ca: nop L0008: mov r8,0x76d4bba60 ;*invokestatic translate ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {oop(a 'org/jboss/logmanager/Level')} L0009: mov r10d ;*getfield loggerNode ; - org.jboss.logmanager.Logger::isLoggable at 1 (line 153) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; implicit exception: dispatches to 0x00007f93951ea23d 0x00007f93951ea0df: mov r10d ;*getfield effectiveLevel ; - org.jboss.logmanager.LoggerNode::getEffectiveLevel at 1 (line 251) ; - org.jboss.logmanager.Logger::isLoggable at 4 (line 153) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; implicit exception: dispatches to 0x00007f93951ea251 0x00007f93951ea0e4: mov r8d ; - java.util.logging.Level::intValue at 1 (line 398) ; - org.jboss.logmanager.Logger::isLoggable at 9 (line 154) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea0e8: cmp r8d,r10d 0x00007f93951ea0eb: jge L0011 ;*if_icmplt ; - org.jboss.logmanager.Logger::isLoggable at 13 (line 154) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea0f1: movzx r10d ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 25 (line 115) 0x00007f93951ea0f9: add rbp,0x1 ; OopMap{r11=Oop r9=Oop rcx=Oop rbx=Oop rdi=Oop rdx=Oop off=349} ;*ifeq ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 28 (line 115) 0x00007f93951ea0fd: test ; {poll} *** SAFEPOINT POLL *** 0x00007f93951ea103: test r10d,r10d 0x00007f93951ea106: jne L0007 ;*aload ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 13 (line 113) L000a: mov r10d ;*getfield logger ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 1 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea10c: mov r8d ; - java.lang.Enum::ordinal at 1 (line 103) ; - org.jboss.logging.JBossLogManagerLogger::translate at 8 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea110: cmp r8d,0x6 0x00007f93951ea114: jae L0010 0x00007f93951ea11a: mov r8d ;*iaload ; - org.jboss.logging.JBossLogManagerLogger::translate at 11 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea11f: cmp r8d,0x4 0x00007f93951ea123: je L0008 0x00007f93951ea125: cmp r8d,0x4 0x00007f93951ea129: jle L000d 0x00007f93951ea12b: cmp r8d,0x6 0x00007f93951ea12f: jne L000b 0x00007f93951ea131: mov r8,0x76d4bbc68 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea13b: jmp L0009 L000b: cmp r8d,0x6 0x00007f93951ea141: jg L000c ;*tableswitch ; - org.jboss.logging.JBossLogManagerLogger::translate at 12 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea143: mov r8,0x76d4bbac8 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea14d: jmp L0009 ;*getstatic ALL ; - org.jboss.logging.JBossLogManagerLogger::translate at 76 (line 66) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) L000c: mov r8,0x76d444030 ; {oop(a 'java/util/logging/Level')} 0x00007f93951ea159: jmp L0009 L000d: cmp r8d,0x2 0x00007f93951ea162: je L000f 0x00007f93951ea164: cmp r8d,0x2 0x00007f93951ea168: jle L000e 0x00007f93951ea16a: mov r8,0x76d4bb958 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea174: jmp L0009 L000e: cmp r8d,0x1 0x00007f93951ea17d: jne L000c ;*tableswitch ; - org.jboss.logging.JBossLogManagerLogger::translate at 12 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) 0x00007f93951ea17f: mov r8,0x76d4bb6b8 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea189: jmp L0009 L000f: mov r8,0x76d4bb848 ; {oop(a 'org/jboss/logmanager/Level')} 0x00007f93951ea198: jmp L0009 L0010: mov esi,0xffffffe4 0x00007f93951ea1a2: mov 0x00007f93951ea1a7: mov 0x00007f93951ea1ac: mov 0x00007f93951ea1b1: mov 0x00007f93951ea1b5: mov 0x00007f93951ea1ba: nop 0x00007f93951ea1bb: call 0x00007f93950051a0 ; OopMap{[48]=Oop [56]=Oop [64]=Oop [0]=NarrowOop off=544} ;*iaload ; - org.jboss.logging.JBossLogManagerLogger::translate at 11 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea1c0: call 0x00007f93aa3650b0 ;*iaload ; - org.jboss.logging.JBossLogManagerLogger::translate at 11 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} L0011: mov esi,0xffffff65 0x00007f93951ea1ca: mov 0x00007f93951ea1cf: mov 0x00007f93951ea1d4: mov 0x00007f93951ea1d9: mov 0x00007f93951ea1de: mov 0x00007f93951ea1e3: call 0x00007f93950051a0 ; OopMap{[48]=Oop [56]=Oop [64]=Oop off=584} ;*if_icmplt ; - org.jboss.logmanager.Logger::isLoggable at 13 (line 154) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea1e8: call 0x00007f93aa3650b0 ; {runtime_call} L0012: mov esi,0xfffffff6 0x00007f93951ea1f2: nop 0x00007f93951ea1f3: call 0x00007f93950051a0 ; OopMap{off=600} ;*invokevirtual tracef ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea1f8: call 0x00007f93aa3650b0 ; {runtime_call} L0013: mov esi,0xffffffe4 0x00007f93951ea202: mov 0x00007f93951ea207: mov 0x00007f93951ea20c: mov 0x00007f93951ea210: xchg ax,ax 0x00007f93951ea213: call 0x00007f93950051a0 ; OopMap{[48]=Oop [56]=Oop [64]=Oop [0]=NarrowOop off=632} ;*iaload ; - org.jboss.logging.JBossLogManagerLogger::translate at 11 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea218: call 0x00007f93aa3650b0 ;*iaload ; - org.jboss.logging.JBossLogManagerLogger::translate at 11 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} L0014: mov esi,0xffffff65 0x00007f93951ea222: mov 0x00007f93951ea227: mov 0x00007f93951ea22c: mov 0x00007f93951ea231: xchg ax,ax 0x00007f93951ea233: call 0x00007f93950051a0 ; OopMap{[48]=Oop [56]=Oop [64]=Oop off=664} ;*if_icmplt ; - org.jboss.logmanager.Logger::isLoggable at 13 (line 154) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea238: call 0x00007f93aa3650b0 ;*if_icmplt ; - org.jboss.logmanager.Logger::isLoggable at 13 (line 154) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea23d: mov esi,0xfffffff6 0x00007f93951ea242: mov rbp,r8 0x00007f93951ea245: xchg ax,ax 0x00007f93951ea247: call 0x00007f93950051a0 ; OopMap{rbp=Oop off=684} ;*invokevirtual isLoggable ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea24c: call 0x00007f93aa3650b0 ;*invokevirtual isLoggable ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea251: mov esi,0xfffffff6 0x00007f93951ea256: nop 0x00007f93951ea257: call 0x00007f93950051a0 ; OopMap{off=700} ;*invokevirtual getEffectiveLevel ; - org.jboss.logmanager.Logger::isLoggable at 4 (line 153) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea25c: call 0x00007f93aa3650b0 ;*invokevirtual getEffectiveLevel ; - org.jboss.logmanager.Logger::isLoggable at 4 (line 153) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea261: mov esi,0xfffffff6 0x00007f93951ea266: mov rbp,rax 0x00007f93951ea269: xchg ax,ax 0x00007f93951ea26b: call 0x00007f93950051a0 ; OopMap{off=720} ;*putfield startTime ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 10 (line 111) ; {runtime_call} 0x00007f93951ea270: call 0x00007f93aa3650b0 ; {runtime_call} 0x00007f93951ea275: mov esi,0xfffffff6 0x00007f93951ea27a: mov rbp,r8 0x00007f93951ea27d: xchg ax,ax 0x00007f93951ea27f: call 0x00007f93950051a0 ; OopMap{rbp=Oop off=740} ;*invokevirtual isLoggable ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea284: call 0x00007f93aa3650b0 ; {runtime_call} 0x00007f93951ea289: mov esi,0xfffffff6 0x00007f93951ea28e: nop 0x00007f93951ea28f: call 0x00007f93950051a0 ; OopMap{off=756} ;*invokevirtual getEffectiveLevel ; - org.jboss.logmanager.Logger::isLoggable at 4 (line 153) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea294: call 0x00007f93aa3650b0 ;*invokevirtual getEffectiveLevel ; - org.jboss.logmanager.Logger::isLoggable at 4 (line 153) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 8 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) ; {runtime_call} 0x00007f93951ea299: mov esi,0xfffffff6 0x00007f93951ea29e: nop 0x00007f93951ea29f: call 0x00007f93950051a0 ; OopMap{off=772} ;*getfield isDone ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 25 (line 115) ; {runtime_call} 0x00007f93951ea2a4: call 0x00007f93aa3650b0 ;*getfield isDone ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 25 (line 115) ; {runtime_call} 0x00007f93951ea2a9: hlt 0x00007f93951ea2aa: hlt 0x00007f93951ea2ab: hlt 0x00007f93951ea2ac: hlt 0x00007f93951ea2ad: hlt 0x00007f93951ea2ae: hlt 0x00007f93951ea2af: hlt 0x00007f93951ea2b0: hlt 0x00007f93951ea2b1: hlt 0x00007f93951ea2b2: hlt 0x00007f93951ea2b3: hlt 0x00007f93951ea2b4: hlt 0x00007f93951ea2b5: hlt 0x00007f93951ea2b6: hlt 0x00007f93951ea2b7: hlt 0x00007f93951ea2b8: hlt 0x00007f93951ea2b9: hlt 0x00007f93951ea2ba: hlt 0x00007f93951ea2bb: hlt 0x00007f93951ea2bc: hlt 0x00007f93951ea2bd: hlt 0x00007f93951ea2be: hlt 0x00007f93951ea2bf: hlt [Exception Handler] [Stub Code] 0x00007f93951ea2c0: jmp 0x00007f93950f94a0 ; {no_reloc} [Deopt Handler Code] 0x00007f93951ea2c5: call 0x00007f93951ea2ca 0x00007f93951ea2ca: sub 0x00007f93951ea2cf: jmp 0x00007f93950473c0 ; {runtime_call} 0x00007f93951ea2d4: hlt 0x00007f93951ea2d5: hlt 0x00007f93951ea2d6: hlt 0x00007f93951ea2d7: hlt From slaskawi at redhat.com Mon Oct 3 08:43:58 2016 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 3 Oct 2016 14:43:58 +0200 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> Message-ID: Awesome Dan! May I ask for a Pull Request: https://github.com/slaskawi/jboss-logging-perf-test? On Mon, Oct 3, 2016 at 2:40 PM, Dan Berindei wrote: > Hi Sebastian > > I modified your benchmark so that the logger and the trace field are > static final and looked at the generated assembly with JITWatch [1]. > Based on the attached assembly listings, caching isTraceEnabled() in a > constant field is "infinitely faster", because there are no assembly > instructions that can be traced back to the test method. > > JBossLogManagerLogger::translate() is inlined in this listing, but it > still goes through the switch machinery, I'm guessing because the > ordinal of an Enum is not considered a constant. > > Cheers > Dan > > > [1]: https://github.com/AdoptOpenJDK/jitwatch > > On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec > wrote: > > Hey! > > > > Please have a look at the latest perf test results [1][2]: > > > > Benchmark Mode Cnt Score > > Error Units > > MyBenchmark.noVariable thrpt 20 681131269.875 ? > > 3961932.923 ops/s > > MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 ? > > 14305970.393 ops/s > > MyBenchmark.withVariable thrpt 20 2411000894.582 ? > > 17382438.635 ops/s > > > > I think there is no surprise here.. using a field, which stores the > result > > of `logger.isTraceEnabled()` evaluation is 3 times faster than other > > options. > > > > If anyone is interested in printing out JIT stuff, I also ran it with > > "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and > "-XX:+PrintInlining" > > here [3]. > > > > I'm not a performance expert but it seems that JIT could not inline the > > "translate" method because of its size (see line 1861). However it tried > > several times with different optimizations (and some of them were thrown > > away - "made not entrant" messages [4]). > > > > Let's wait for James' opinion on this, but I think we should address this > > issue on JBoss Logging/LogManager side (so I agree with David here) and > make > > those parts inlinable (wow, does this word even exist? :D). Once this is > > done, we could experiment further in Infinispan codebase and see how this > > relates to some real world benchmarks... > > > > Thanks > > Sebastian > > > > [1] https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a53c > > [2] Test repository: https://github.com/slaskawi/jboss-logging-perf-test > > [3] https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924bae0 > > [4] > > https://www.safaribooksonline.com/library/view/java- > performance-the/9781449363512/ch04.html > > > > On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec > > > wrote: > >> > >> Hey! > >> > >> Adding James to this thread. > >> > >> @Dennis - I think Dan has a point here. The trick with checking a field > in > >> a class is 3 times faster. Most of the checks are done in core so they > are > >> executed multiple times per operation. Changing all those places is > probably > >> not an option. > >> > >> @David - Let me run a test with JBoss Log Manager and get back to you > with > >> some results. But if Dan is right, and the problem is with enum > mapping, I > >> will get similar results. > >> > >> Thanks > >> Sebastian > >> > >> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei > >> wrote: > >>> > >>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd < > david.lloyd at redhat.com> > >>> wrote: > >>> > The performance problem that this trick is meant to resolve is > really a > >>> > problem in the logging backend. It *should* be faster inside of > >>> > WildFly, where JBoss LogManager is used, because that project just > >>> > checks a single volatile field for the level check... and the path to > >>> > that code *should* be inline-friendly. > >>> > > >>> > >>> Indeed, we started using this trick because of log4j 1.2, which needs > >>> to walk the logger hierarchy in order to check the level, and it made > >>> a significant difference there. > >>> > >>> Nowadays I think it's pretty close to optimal in all logging > >>> frameworks. The only nitpick is that they all use enums for the > >>> levels, and the JIT can't inline Level.TRACE.value as it would with a > >>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going to > >>> be more or less equivalent to using a volatile "trace" field in each > >>> class, so it should be suitable even for local read operations that > >>> take < 200ns. > >>> > >>> We'd probably still need to weed out some of the trace messages, as we > >>> probably have more than 10 of them during such a read operation. I > >>> confess that I added way too many trace logs myself, precisely because > >>> I knew we are using a static final field and the JIT compiler doesn't > >>> even generate code for that branch. > >>> > >>> Cheers > >>> Dan > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161003/a2990334/attachment.html From sanne at infinispan.org Mon Oct 3 08:58:53 2016 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 03 Oct 2016 12:58:53 +0000 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> Message-ID: I've not looked at any such details but I always assumed that since the logger frameworks (any of them) need to be able to change the logging level, there must be some way of switching state - at least a volatile read? And while that could be very cheap, I doubt it would be as easily optimised as a constant. Since we might not be interested only in the logger method to be inlined, we also want most of the client methods to be considered for inlining.. some of these can be extremely sensitive so such a decision requires a full Infinispan performance test rather than a microbenchmark. (which is useful too but more for the sake of optimising JBoss Logging, not to proof if it's OK to switch all of Infinispan 's internal conventions) On Mon, 3 Oct 2016, 13:45 Sebastian Laskawiec, wrote: > Awesome Dan! May I ask for a Pull Request: > https://github.com/slaskawi/jboss-logging-perf-test? > > > > On Mon, Oct 3, 2016 at 2:40 PM, Dan Berindei > wrote: > > Hi Sebastian > > I modified your benchmark so that the logger and the trace field are > static final and looked at the generated assembly with JITWatch [1]. > Based on the attached assembly listings, caching isTraceEnabled() in a > constant field is "infinitely faster", because there are no assembly > instructions that can be traced back to the test method. > > JBossLogManagerLogger::translate() is inlined in this listing, but it > still goes through the switch machinery, I'm guessing because the > ordinal of an Enum is not considered a constant. > > Cheers > Dan > > > [1]: https://github.com/AdoptOpenJDK/jitwatch > > On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec > wrote: > > Hey! > > > > Please have a look at the latest perf test results [1][2]: > > > > Benchmark Mode Cnt Score > > Error Units > > MyBenchmark.noVariable thrpt 20 681131269.875 ? > > 3961932.923 ops/s > > MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 ? > > 14305970.393 ops/s > > MyBenchmark.withVariable thrpt 20 2411000894.582 ? > > 17382438.635 ops/s > > > > I think there is no surprise here.. using a field, which stores the > result > > of `logger.isTraceEnabled()` evaluation is 3 times faster than other > > options. > > > > If anyone is interested in printing out JIT stuff, I also ran it with > > "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and > "-XX:+PrintInlining" > > here [3]. > > > > I'm not a performance expert but it seems that JIT could not inline the > > "translate" method because of its size (see line 1861). However it tried > > several times with different optimizations (and some of them were thrown > > away - "made not entrant" messages [4]). > > > > Let's wait for James' opinion on this, but I think we should address this > > issue on JBoss Logging/LogManager side (so I agree with David here) and > make > > those parts inlinable (wow, does this word even exist? :D). Once this is > > done, we could experiment further in Infinispan codebase and see how this > > relates to some real world benchmarks... > > > > Thanks > > Sebastian > > > > [1] https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a53c > > [2] Test repository: https://github.com/slaskawi/jboss-logging-perf-test > > [3] https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924bae0 > > [4] > > > https://www.safaribooksonline.com/library/view/java-performance-the/9781449363512/ch04.html > > > > On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec > > > wrote: > >> > >> Hey! > >> > >> Adding James to this thread. > >> > >> @Dennis - I think Dan has a point here. The trick with checking a field > in > >> a class is 3 times faster. Most of the checks are done in core so they > are > >> executed multiple times per operation. Changing all those places is > probably > >> not an option. > >> > >> @David - Let me run a test with JBoss Log Manager and get back to you > with > >> some results. But if Dan is right, and the problem is with enum > mapping, I > >> will get similar results. > >> > >> Thanks > >> Sebastian > >> > >> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei > >> wrote: > >>> > >>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd < > david.lloyd at redhat.com> > >>> wrote: > >>> > The performance problem that this trick is meant to resolve is > really a > >>> > problem in the logging backend. It *should* be faster inside of > >>> > WildFly, where JBoss LogManager is used, because that project just > >>> > checks a single volatile field for the level check... and the path to > >>> > that code *should* be inline-friendly. > >>> > > >>> > >>> Indeed, we started using this trick because of log4j 1.2, which needs > >>> to walk the logger hierarchy in order to check the level, and it made > >>> a significant difference there. > >>> > >>> Nowadays I think it's pretty close to optimal in all logging > >>> frameworks. The only nitpick is that they all use enums for the > >>> levels, and the JIT can't inline Level.TRACE.value as it would with a > >>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going to > >>> be more or less equivalent to using a volatile "trace" field in each > >>> class, so it should be suitable even for local read operations that > >>> take < 200ns. > >>> > >>> We'd probably still need to weed out some of the trace messages, as we > >>> probably have more than 10 of them during such a read operation. I > >>> confess that I added way too many trace logs myself, precisely because > >>> I knew we are using a static final field and the JIT compiler doesn't > >>> even generate code for that branch. > >>> > >>> Cheers > >>> Dan > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161003/7ae9a273/attachment-0001.html From slaskawi at redhat.com Mon Oct 3 09:04:35 2016 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 3 Oct 2016 15:04:35 +0200 Subject: [infinispan-dev] WF Swarm - first approach Message-ID: Hey! I've created very short demo of WF Swarm and Infinispan Embedded mode: https://github.com/slaskawi/infinispan-wf-swarm-example/tree/master I have a couple of thoughts after doing this demo: - Wildfly Swarm generator [2] allows to add Infinispan integration module. Unfortunately it doesn't provide Infinispan classes to the classpath (one needs to add infinispan-core in provided scope for example). I created JIRA to fix this [3]. - We should probably make JPA (Hibernate) an optional dependency. Created JIRA for it [4]. - Injecting beans into JAX-RS classes require using org.wildfly.swarm:jaxrs-cdi. This will be addressed at WF Swarm side [5]. I must admit, implementing this demo was a really nice experience (there were some problems, but with Martin's and Tomas' help I managed to work them out (thanks a lot guys!)). If you have some time, please experiment with the demo code and tell me what you think. If there won't be any comments, I plan to write a small blog entry about WF Swarm and Infinispan. Thanks Sebastian [2] http://wildfly-swarm.io/generator/ [3] https://issues.jboss.org/browse/ISPN-7071 [4] https://issues.jboss.org/browse/ISPN-7072 [5] https://issues.jboss.org/browse/SWARM-714 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161003/eb34f879/attachment.html From dan.berindei at gmail.com Mon Oct 3 09:25:16 2016 From: dan.berindei at gmail.com (Dan Berindei) Date: Mon, 3 Oct 2016 16:25:16 +0300 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> Message-ID: On Mon, Oct 3, 2016 at 3:58 PM, Sanne Grinovero wrote: > I've not looked at any such details but I always assumed that since the > logger frameworks (any of them) need to be able to change the logging level, > there must be some way of switching state - at least a volatile read? > > And while that could be very cheap, I doubt it would be as easily optimised > as a constant. > True, at some point you need a volatile read -- you can only avoid that by caching isTraceEnabled(), like we do in Infinispan. My point was related to this assembly instruction, which reads the ordinal of Level.TRACE: 0x00007f93951e9fdf: mov rdi,0x76d4e6d48 ; {oop(a 'org/jboss/logging/Logger$Level')} 0x00007f93951e9fe9: mov ebp << missing an operand here >>; - java.lang.Enum::ordinal at 1 (line 103) ; - org.jboss.logging.JBossLogManagerLogger::translate at 8 (line 58) ; - org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) ; - org.jboss.logging.Logger::tracef at 4 (line 287) ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) ; - org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 (line 113) In an ideal world, the JIT would replace translate(Level.TRACE) with a constant. But here, even though `0x76d4e6d48` is clearly a constant, the ordinal is still loaded and checked every time. > Since we might not be interested only in the logger method to be inlined, we > also want most of the client methods to be considered for inlining.. some of > these can be extremely sensitive so such a decision requires a full > Infinispan performance test rather than a microbenchmark. (which is useful > too but more for the sake of optimising JBoss Logging, not to proof if it's > OK to switch all of Infinispan 's internal conventions) > Based on the testing both myself and Radim did WRT inlining, it doesn't make a huge difference. But I agree 100% that before removing the isTraceEnabled() caching we'd have to benchmark the whole of Infinispan, and not just JBoss Logging in isolation. Cheers Dan > > On Mon, 3 Oct 2016, 13:45 Sebastian Laskawiec, wrote: >> >> Awesome Dan! May I ask for a Pull Request: >> https://github.com/slaskawi/jboss-logging-perf-test? >> >> >> >> On Mon, Oct 3, 2016 at 2:40 PM, Dan Berindei >> wrote: >>> >>> Hi Sebastian >>> >>> I modified your benchmark so that the logger and the trace field are >>> static final and looked at the generated assembly with JITWatch [1]. >>> Based on the attached assembly listings, caching isTraceEnabled() in a >>> constant field is "infinitely faster", because there are no assembly >>> instructions that can be traced back to the test method. >>> >>> JBossLogManagerLogger::translate() is inlined in this listing, but it >>> still goes through the switch machinery, I'm guessing because the >>> ordinal of an Enum is not considered a constant. >>> >>> Cheers >>> Dan >>> >>> >>> [1]: https://github.com/AdoptOpenJDK/jitwatch >>> >>> On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec >>> wrote: >>> > Hey! >>> > >>> > Please have a look at the latest perf test results [1][2]: >>> > >>> > Benchmark Mode Cnt Score >>> > Error Units >>> > MyBenchmark.noVariable thrpt 20 681131269.875 ? >>> > 3961932.923 ops/s >>> > MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 ? >>> > 14305970.393 ops/s >>> > MyBenchmark.withVariable thrpt 20 2411000894.582 ? >>> > 17382438.635 ops/s >>> > >>> > I think there is no surprise here.. using a field, which stores the >>> > result >>> > of `logger.isTraceEnabled()` evaluation is 3 times faster than other >>> > options. >>> > >>> > If anyone is interested in printing out JIT stuff, I also ran it with >>> > "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and >>> > "-XX:+PrintInlining" >>> > here [3]. >>> > >>> > I'm not a performance expert but it seems that JIT could not inline the >>> > "translate" method because of its size (see line 1861). However it >>> > tried >>> > several times with different optimizations (and some of them were >>> > thrown >>> > away - "made not entrant" messages [4]). >>> > >>> > Let's wait for James' opinion on this, but I think we should address >>> > this >>> > issue on JBoss Logging/LogManager side (so I agree with David here) and >>> > make >>> > those parts inlinable (wow, does this word even exist? :D). Once this >>> > is >>> > done, we could experiment further in Infinispan codebase and see how >>> > this >>> > relates to some real world benchmarks... >>> > >>> > Thanks >>> > Sebastian >>> > >>> > [1] https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a53c >>> > [2] Test repository: >>> > https://github.com/slaskawi/jboss-logging-perf-test >>> > [3] https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924bae0 >>> > [4] >>> > >>> > https://www.safaribooksonline.com/library/view/java-performance-the/9781449363512/ch04.html >>> > >>> > On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec >>> > >>> > wrote: >>> >> >>> >> Hey! >>> >> >>> >> Adding James to this thread. >>> >> >>> >> @Dennis - I think Dan has a point here. The trick with checking a >>> >> field in >>> >> a class is 3 times faster. Most of the checks are done in core so they >>> >> are >>> >> executed multiple times per operation. Changing all those places is >>> >> probably >>> >> not an option. >>> >> >>> >> @David - Let me run a test with JBoss Log Manager and get back to you >>> >> with >>> >> some results. But if Dan is right, and the problem is with enum >>> >> mapping, I >>> >> will get similar results. >>> >> >>> >> Thanks >>> >> Sebastian >>> >> >>> >> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei >>> >> wrote: >>> >>> >>> >>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd >>> >>> >>> >>> wrote: >>> >>> > The performance problem that this trick is meant to resolve is >>> >>> > really a >>> >>> > problem in the logging backend. It *should* be faster inside of >>> >>> > WildFly, where JBoss LogManager is used, because that project just >>> >>> > checks a single volatile field for the level check... and the path >>> >>> > to >>> >>> > that code *should* be inline-friendly. >>> >>> > >>> >>> >>> >>> Indeed, we started using this trick because of log4j 1.2, which needs >>> >>> to walk the logger hierarchy in order to check the level, and it made >>> >>> a significant difference there. >>> >>> >>> >>> Nowadays I think it's pretty close to optimal in all logging >>> >>> frameworks. The only nitpick is that they all use enums for the >>> >>> levels, and the JIT can't inline Level.TRACE.value as it would with a >>> >>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going >>> >>> to >>> >>> be more or less equivalent to using a volatile "trace" field in each >>> >>> class, so it should be suitable even for local read operations that >>> >>> take < 200ns. >>> >>> >>> >>> We'd probably still need to weed out some of the trace messages, as >>> >>> we >>> >>> probably have more than 10 of them during such a read operation. I >>> >>> confess that I added way too many trace logs myself, precisely >>> >>> because >>> >>> I knew we are using a static final field and the JIT compiler doesn't >>> >>> even generate code for that branch. >>> >>> >>> >>> Cheers >>> >>> Dan >>> >>> _______________________________________________ >>> >>> infinispan-dev mailing list >>> >>> infinispan-dev at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> >>> >> >>> > >>> > >>> > _______________________________________________ >>> > infinispan-dev mailing list >>> > infinispan-dev at lists.jboss.org >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From rvansa at redhat.com Mon Oct 3 09:43:46 2016 From: rvansa at redhat.com (Radim Vansa) Date: Mon, 3 Oct 2016 15:43:46 +0200 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> Message-ID: <57F26092.3050705@redhat.com> Aren't we investing a bit too much time to this? The trace level was cached for ages, has anyone ever complained? Turning on trace level is completely impractical in production, but that info (or, I'd say 80% of that) is very useful when debugging race conditions etc., so it makes sense to keep it there and reduce the cost to minimum. My 2c Radim On 10/03/2016 03:25 PM, Dan Berindei wrote: > On Mon, Oct 3, 2016 at 3:58 PM, Sanne Grinovero wrote: >> I've not looked at any such details but I always assumed that since the >> logger frameworks (any of them) need to be able to change the logging level, >> there must be some way of switching state - at least a volatile read? >> >> And while that could be very cheap, I doubt it would be as easily optimised >> as a constant. >> > True, at some point you need a volatile read -- you can only avoid > that by caching isTraceEnabled(), like we do in Infinispan. > > My point was related to this assembly instruction, which reads the > ordinal of Level.TRACE: > > 0x00007f93951e9fdf: mov rdi,0x76d4e6d48 ; {oop(a > 'org/jboss/logging/Logger$Level')} > 0x00007f93951e9fe9: mov ebp << missing an operand here >>; - > java.lang.Enum::ordinal at 1 (line 103) > ; - > org.jboss.logging.JBossLogManagerLogger::translate at 8 (line 58) > ; - > org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) > ; - org.jboss.logging.Logger::tracef at 4 (line 287) > ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) > ; - > org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 > (line 113) > > In an ideal world, the JIT would replace translate(Level.TRACE) with a > constant. But here, even though `0x76d4e6d48` is clearly a constant, > the ordinal is still loaded and checked every time. > >> Since we might not be interested only in the logger method to be inlined, we >> also want most of the client methods to be considered for inlining.. some of >> these can be extremely sensitive so such a decision requires a full >> Infinispan performance test rather than a microbenchmark. (which is useful >> too but more for the sake of optimising JBoss Logging, not to proof if it's >> OK to switch all of Infinispan 's internal conventions) >> > Based on the testing both myself and Radim did WRT inlining, it > doesn't make a huge difference. But I agree 100% that before removing > the isTraceEnabled() caching we'd have to benchmark the whole of > Infinispan, and not just JBoss Logging in isolation. > > Cheers > Dan > >> On Mon, 3 Oct 2016, 13:45 Sebastian Laskawiec, wrote: >>> Awesome Dan! May I ask for a Pull Request: >>> https://github.com/slaskawi/jboss-logging-perf-test? >>> >>> >>> >>> On Mon, Oct 3, 2016 at 2:40 PM, Dan Berindei >>> wrote: >>>> Hi Sebastian >>>> >>>> I modified your benchmark so that the logger and the trace field are >>>> static final and looked at the generated assembly with JITWatch [1]. >>>> Based on the attached assembly listings, caching isTraceEnabled() in a >>>> constant field is "infinitely faster", because there are no assembly >>>> instructions that can be traced back to the test method. >>>> >>>> JBossLogManagerLogger::translate() is inlined in this listing, but it >>>> still goes through the switch machinery, I'm guessing because the >>>> ordinal of an Enum is not considered a constant. >>>> >>>> Cheers >>>> Dan >>>> >>>> >>>> [1]: https://github.com/AdoptOpenJDK/jitwatch >>>> >>>> On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec >>>> wrote: >>>>> Hey! >>>>> >>>>> Please have a look at the latest perf test results [1][2]: >>>>> >>>>> Benchmark Mode Cnt Score >>>>> Error Units >>>>> MyBenchmark.noVariable thrpt 20 681131269.875 ? >>>>> 3961932.923 ops/s >>>>> MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 ? >>>>> 14305970.393 ops/s >>>>> MyBenchmark.withVariable thrpt 20 2411000894.582 ? >>>>> 17382438.635 ops/s >>>>> >>>>> I think there is no surprise here.. using a field, which stores the >>>>> result >>>>> of `logger.isTraceEnabled()` evaluation is 3 times faster than other >>>>> options. >>>>> >>>>> If anyone is interested in printing out JIT stuff, I also ran it with >>>>> "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and >>>>> "-XX:+PrintInlining" >>>>> here [3]. >>>>> >>>>> I'm not a performance expert but it seems that JIT could not inline the >>>>> "translate" method because of its size (see line 1861). However it >>>>> tried >>>>> several times with different optimizations (and some of them were >>>>> thrown >>>>> away - "made not entrant" messages [4]). >>>>> >>>>> Let's wait for James' opinion on this, but I think we should address >>>>> this >>>>> issue on JBoss Logging/LogManager side (so I agree with David here) and >>>>> make >>>>> those parts inlinable (wow, does this word even exist? :D). Once this >>>>> is >>>>> done, we could experiment further in Infinispan codebase and see how >>>>> this >>>>> relates to some real world benchmarks... >>>>> >>>>> Thanks >>>>> Sebastian >>>>> >>>>> [1] https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a53c >>>>> [2] Test repository: >>>>> https://github.com/slaskawi/jboss-logging-perf-test >>>>> [3] https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924bae0 >>>>> [4] >>>>> >>>>> https://www.safaribooksonline.com/library/view/java-performance-the/9781449363512/ch04.html >>>>> >>>>> On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec >>>>> >>>>> wrote: >>>>>> Hey! >>>>>> >>>>>> Adding James to this thread. >>>>>> >>>>>> @Dennis - I think Dan has a point here. The trick with checking a >>>>>> field in >>>>>> a class is 3 times faster. Most of the checks are done in core so they >>>>>> are >>>>>> executed multiple times per operation. Changing all those places is >>>>>> probably >>>>>> not an option. >>>>>> >>>>>> @David - Let me run a test with JBoss Log Manager and get back to you >>>>>> with >>>>>> some results. But if Dan is right, and the problem is with enum >>>>>> mapping, I >>>>>> will get similar results. >>>>>> >>>>>> Thanks >>>>>> Sebastian >>>>>> >>>>>> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei >>>>>> wrote: >>>>>>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd >>>>>>> >>>>>>> wrote: >>>>>>>> The performance problem that this trick is meant to resolve is >>>>>>>> really a >>>>>>>> problem in the logging backend. It *should* be faster inside of >>>>>>>> WildFly, where JBoss LogManager is used, because that project just >>>>>>>> checks a single volatile field for the level check... and the path >>>>>>>> to >>>>>>>> that code *should* be inline-friendly. >>>>>>>> >>>>>>> Indeed, we started using this trick because of log4j 1.2, which needs >>>>>>> to walk the logger hierarchy in order to check the level, and it made >>>>>>> a significant difference there. >>>>>>> >>>>>>> Nowadays I think it's pretty close to optimal in all logging >>>>>>> frameworks. The only nitpick is that they all use enums for the >>>>>>> levels, and the JIT can't inline Level.TRACE.value as it would with a >>>>>>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going >>>>>>> to >>>>>>> be more or less equivalent to using a volatile "trace" field in each >>>>>>> class, so it should be suitable even for local read operations that >>>>>>> take < 200ns. >>>>>>> >>>>>>> We'd probably still need to weed out some of the trace messages, as >>>>>>> we >>>>>>> probably have more than 10 of them during such a read operation. I >>>>>>> confess that I added way too many trace logs myself, precisely >>>>>>> because >>>>>>> I knew we are using a static final field and the JIT compiler doesn't >>>>>>> even generate code for that branch. >>>>>>> >>>>>>> Cheers >>>>>>> Dan >>>>>>> _______________________________________________ >>>>>>> infinispan-dev mailing list >>>>>>> infinispan-dev at lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From dereed at redhat.com Mon Oct 3 13:31:56 2016 From: dereed at redhat.com (Dennis Reed) Date: Mon, 3 Oct 2016 12:31:56 -0500 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: <57F26092.3050705@redhat.com> References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> <57F26092.3050705@redhat.com> Message-ID: <57F2960C.6000705@redhat.com> On 10/03/2016 08:43 AM, Radim Vansa wrote: > Aren't we investing a bit too much time to this? The trace level was > cached for ages, has anyone ever complained? I have. https://issues.jboss.org/browse/JBCACHE-1625 Looks like I just never copied it over to the ISPN JIRA. :) > Turning on trace level is completely impractical in production, We do it all the time when there's no better option to debug an issue. (generally for as short a period as possible if it's production) -Dennis > but that info (or, I'd say 80% of > that) is very useful when debugging race conditions etc., so it makes > sense to keep it there and reduce the cost to minimum. > > My 2c > > Radim > > On 10/03/2016 03:25 PM, Dan Berindei wrote: >> On Mon, Oct 3, 2016 at 3:58 PM, Sanne Grinovero wrote: >>> I've not looked at any such details but I always assumed that since the >>> logger frameworks (any of them) need to be able to change the logging level, >>> there must be some way of switching state - at least a volatile read? >>> >>> And while that could be very cheap, I doubt it would be as easily optimised >>> as a constant. >>> >> True, at some point you need a volatile read -- you can only avoid >> that by caching isTraceEnabled(), like we do in Infinispan. >> >> My point was related to this assembly instruction, which reads the >> ordinal of Level.TRACE: >> >> 0x00007f93951e9fdf: mov rdi,0x76d4e6d48 ; {oop(a >> 'org/jboss/logging/Logger$Level')} >> 0x00007f93951e9fe9: mov ebp << missing an operand here >>; - >> java.lang.Enum::ordinal at 1 (line 103) >> ; - >> org.jboss.logging.JBossLogManagerLogger::translate at 8 (line 58) >> ; - >> org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) >> ; - org.jboss.logging.Logger::tracef at 4 (line 287) >> ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) >> ; - >> org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 >> (line 113) >> >> In an ideal world, the JIT would replace translate(Level.TRACE) with a >> constant. But here, even though `0x76d4e6d48` is clearly a constant, >> the ordinal is still loaded and checked every time. >> >>> Since we might not be interested only in the logger method to be inlined, we >>> also want most of the client methods to be considered for inlining.. some of >>> these can be extremely sensitive so such a decision requires a full >>> Infinispan performance test rather than a microbenchmark. (which is useful >>> too but more for the sake of optimising JBoss Logging, not to proof if it's >>> OK to switch all of Infinispan 's internal conventions) >>> >> Based on the testing both myself and Radim did WRT inlining, it >> doesn't make a huge difference. But I agree 100% that before removing >> the isTraceEnabled() caching we'd have to benchmark the whole of >> Infinispan, and not just JBoss Logging in isolation. >> >> Cheers >> Dan >> >>> On Mon, 3 Oct 2016, 13:45 Sebastian Laskawiec, wrote: >>>> Awesome Dan! May I ask for a Pull Request: >>>> https://github.com/slaskawi/jboss-logging-perf-test? >>>> >>>> >>>> >>>> On Mon, Oct 3, 2016 at 2:40 PM, Dan Berindei >>>> wrote: >>>>> Hi Sebastian >>>>> >>>>> I modified your benchmark so that the logger and the trace field are >>>>> static final and looked at the generated assembly with JITWatch [1]. >>>>> Based on the attached assembly listings, caching isTraceEnabled() in a >>>>> constant field is "infinitely faster", because there are no assembly >>>>> instructions that can be traced back to the test method. >>>>> >>>>> JBossLogManagerLogger::translate() is inlined in this listing, but it >>>>> still goes through the switch machinery, I'm guessing because the >>>>> ordinal of an Enum is not considered a constant. >>>>> >>>>> Cheers >>>>> Dan >>>>> >>>>> >>>>> [1]: https://github.com/AdoptOpenJDK/jitwatch >>>>> >>>>> On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec >>>>> wrote: >>>>>> Hey! >>>>>> >>>>>> Please have a look at the latest perf test results [1][2]: >>>>>> >>>>>> Benchmark Mode Cnt Score >>>>>> Error Units >>>>>> MyBenchmark.noVariable thrpt 20 681131269.875 ? >>>>>> 3961932.923 ops/s >>>>>> MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 ? >>>>>> 14305970.393 ops/s >>>>>> MyBenchmark.withVariable thrpt 20 2411000894.582 ? >>>>>> 17382438.635 ops/s >>>>>> >>>>>> I think there is no surprise here.. using a field, which stores the >>>>>> result >>>>>> of `logger.isTraceEnabled()` evaluation is 3 times faster than other >>>>>> options. >>>>>> >>>>>> If anyone is interested in printing out JIT stuff, I also ran it with >>>>>> "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and >>>>>> "-XX:+PrintInlining" >>>>>> here [3]. >>>>>> >>>>>> I'm not a performance expert but it seems that JIT could not inline the >>>>>> "translate" method because of its size (see line 1861). However it >>>>>> tried >>>>>> several times with different optimizations (and some of them were >>>>>> thrown >>>>>> away - "made not entrant" messages [4]). >>>>>> >>>>>> Let's wait for James' opinion on this, but I think we should address >>>>>> this >>>>>> issue on JBoss Logging/LogManager side (so I agree with David here) and >>>>>> make >>>>>> those parts inlinable (wow, does this word even exist? :D). Once this >>>>>> is >>>>>> done, we could experiment further in Infinispan codebase and see how >>>>>> this >>>>>> relates to some real world benchmarks... >>>>>> >>>>>> Thanks >>>>>> Sebastian >>>>>> >>>>>> [1] https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a53c >>>>>> [2] Test repository: >>>>>> https://github.com/slaskawi/jboss-logging-perf-test >>>>>> [3] https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924bae0 >>>>>> [4] >>>>>> >>>>>> https://www.safaribooksonline.com/library/view/java-performance-the/9781449363512/ch04.html >>>>>> >>>>>> On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec >>>>>> >>>>>> wrote: >>>>>>> Hey! >>>>>>> >>>>>>> Adding James to this thread. >>>>>>> >>>>>>> @Dennis - I think Dan has a point here. The trick with checking a >>>>>>> field in >>>>>>> a class is 3 times faster. Most of the checks are done in core so they >>>>>>> are >>>>>>> executed multiple times per operation. Changing all those places is >>>>>>> probably >>>>>>> not an option. >>>>>>> >>>>>>> @David - Let me run a test with JBoss Log Manager and get back to you >>>>>>> with >>>>>>> some results. But if Dan is right, and the problem is with enum >>>>>>> mapping, I >>>>>>> will get similar results. >>>>>>> >>>>>>> Thanks >>>>>>> Sebastian >>>>>>> >>>>>>> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei >>>>>>> wrote: >>>>>>>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd >>>>>>>> >>>>>>>> wrote: >>>>>>>>> The performance problem that this trick is meant to resolve is >>>>>>>>> really a >>>>>>>>> problem in the logging backend. It *should* be faster inside of >>>>>>>>> WildFly, where JBoss LogManager is used, because that project just >>>>>>>>> checks a single volatile field for the level check... and the path >>>>>>>>> to >>>>>>>>> that code *should* be inline-friendly. >>>>>>>>> >>>>>>>> Indeed, we started using this trick because of log4j 1.2, which needs >>>>>>>> to walk the logger hierarchy in order to check the level, and it made >>>>>>>> a significant difference there. >>>>>>>> >>>>>>>> Nowadays I think it's pretty close to optimal in all logging >>>>>>>> frameworks. The only nitpick is that they all use enums for the >>>>>>>> levels, and the JIT can't inline Level.TRACE.value as it would with a >>>>>>>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going >>>>>>>> to >>>>>>>> be more or less equivalent to using a volatile "trace" field in each >>>>>>>> class, so it should be suitable even for local read operations that >>>>>>>> take < 200ns. >>>>>>>> >>>>>>>> We'd probably still need to weed out some of the trace messages, as >>>>>>>> we >>>>>>>> probably have more than 10 of them during such a read operation. I >>>>>>>> confess that I added way too many trace logs myself, precisely >>>>>>>> because >>>>>>>> I knew we are using a static final field and the JIT compiler doesn't >>>>>>>> even generate code for that branch. >>>>>>>> >>>>>>>> Cheers >>>>>>>> Dan >>>>>>>> _______________________________________________ >>>>>>>> infinispan-dev mailing list >>>>>>>> infinispan-dev at lists.jboss.org >>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> infinispan-dev mailing list >>>>>> infinispan-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > From rvansa at redhat.com Tue Oct 4 03:31:25 2016 From: rvansa at redhat.com (Radim Vansa) Date: Tue, 4 Oct 2016 09:31:25 +0200 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: <57F2960C.6000705@redhat.com> References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> <57F26092.3050705@redhat.com> <57F2960C.6000705@redhat.com> Message-ID: <57F35ACD.1090605@redhat.com> On 10/03/2016 07:31 PM, Dennis Reed wrote: > On 10/03/2016 08:43 AM, Radim Vansa wrote: >> Aren't we investing a bit too much time to this? The trace level was >> cached for ages, has anyone ever complained? > I have. https://issues.jboss.org/browse/JBCACHE-1625 > > Looks like I just never copied it over to the ISPN JIRA. :) Is this based on customer/user scenario, or have you just noticed that it is not what it should be? (I agree that not caching is definitely nicer) > >> Turning on trace level is completely impractical in production, > We do it all the time when there's no better option to debug an issue. > (generally for as short a period as possible if it's production) I was speaking specifically about Infinispan. I understand that other products have different level of verbosity and it may be of use. However during my time in QA I was running stress tests on trace level and I've often experienced problems caused by contention in log4j (log4j2 has somewhat improved the situation), breaking the cluster apart etc. That's why I am deep in doubt. Btw., workaround is to start the cluster with tracing on, let the static fields resolve and then switch it off until you really want to use it. Radim > > -Dennis > >> but that info (or, I'd say 80% of >> that) is very useful when debugging race conditions etc., so it makes >> sense to keep it there and reduce the cost to minimum. >> >> My 2c >> >> Radim >> >> On 10/03/2016 03:25 PM, Dan Berindei wrote: >>> On Mon, Oct 3, 2016 at 3:58 PM, Sanne Grinovero wrote: >>>> I've not looked at any such details but I always assumed that since the >>>> logger frameworks (any of them) need to be able to change the logging level, >>>> there must be some way of switching state - at least a volatile read? >>>> >>>> And while that could be very cheap, I doubt it would be as easily optimised >>>> as a constant. >>>> >>> True, at some point you need a volatile read -- you can only avoid >>> that by caching isTraceEnabled(), like we do in Infinispan. >>> >>> My point was related to this assembly instruction, which reads the >>> ordinal of Level.TRACE: >>> >>> 0x00007f93951e9fdf: mov rdi,0x76d4e6d48 ; {oop(a >>> 'org/jboss/logging/Logger$Level')} >>> 0x00007f93951e9fe9: mov ebp << missing an operand here >>; - >>> java.lang.Enum::ordinal at 1 (line 103) >>> ; - >>> org.jboss.logging.JBossLogManagerLogger::translate at 8 (line 58) >>> ; - >>> org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) >>> ; - org.jboss.logging.Logger::tracef at 4 (line 287) >>> ; - org.infinispan.MyBenchmark::tracef at 7 (line 26) >>> ; - >>> org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 >>> (line 113) >>> >>> In an ideal world, the JIT would replace translate(Level.TRACE) with a >>> constant. But here, even though `0x76d4e6d48` is clearly a constant, >>> the ordinal is still loaded and checked every time. >>> >>>> Since we might not be interested only in the logger method to be inlined, we >>>> also want most of the client methods to be considered for inlining.. some of >>>> these can be extremely sensitive so such a decision requires a full >>>> Infinispan performance test rather than a microbenchmark. (which is useful >>>> too but more for the sake of optimising JBoss Logging, not to proof if it's >>>> OK to switch all of Infinispan 's internal conventions) >>>> >>> Based on the testing both myself and Radim did WRT inlining, it >>> doesn't make a huge difference. But I agree 100% that before removing >>> the isTraceEnabled() caching we'd have to benchmark the whole of >>> Infinispan, and not just JBoss Logging in isolation. >>> >>> Cheers >>> Dan >>> >>>> On Mon, 3 Oct 2016, 13:45 Sebastian Laskawiec, wrote: >>>>> Awesome Dan! May I ask for a Pull Request: >>>>> https://github.com/slaskawi/jboss-logging-perf-test? >>>>> >>>>> >>>>> >>>>> On Mon, Oct 3, 2016 at 2:40 PM, Dan Berindei >>>>> wrote: >>>>>> Hi Sebastian >>>>>> >>>>>> I modified your benchmark so that the logger and the trace field are >>>>>> static final and looked at the generated assembly with JITWatch [1]. >>>>>> Based on the attached assembly listings, caching isTraceEnabled() in a >>>>>> constant field is "infinitely faster", because there are no assembly >>>>>> instructions that can be traced back to the test method. >>>>>> >>>>>> JBossLogManagerLogger::translate() is inlined in this listing, but it >>>>>> still goes through the switch machinery, I'm guessing because the >>>>>> ordinal of an Enum is not considered a constant. >>>>>> >>>>>> Cheers >>>>>> Dan >>>>>> >>>>>> >>>>>> [1]: https://github.com/AdoptOpenJDK/jitwatch >>>>>> >>>>>> On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec >>>>>> wrote: >>>>>>> Hey! >>>>>>> >>>>>>> Please have a look at the latest perf test results [1][2]: >>>>>>> >>>>>>> Benchmark Mode Cnt Score >>>>>>> Error Units >>>>>>> MyBenchmark.noVariable thrpt 20 681131269.875 ? >>>>>>> 3961932.923 ops/s >>>>>>> MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 ? >>>>>>> 14305970.393 ops/s >>>>>>> MyBenchmark.withVariable thrpt 20 2411000894.582 ? >>>>>>> 17382438.635 ops/s >>>>>>> >>>>>>> I think there is no surprise here.. using a field, which stores the >>>>>>> result >>>>>>> of `logger.isTraceEnabled()` evaluation is 3 times faster than other >>>>>>> options. >>>>>>> >>>>>>> If anyone is interested in printing out JIT stuff, I also ran it with >>>>>>> "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and >>>>>>> "-XX:+PrintInlining" >>>>>>> here [3]. >>>>>>> >>>>>>> I'm not a performance expert but it seems that JIT could not inline the >>>>>>> "translate" method because of its size (see line 1861). However it >>>>>>> tried >>>>>>> several times with different optimizations (and some of them were >>>>>>> thrown >>>>>>> away - "made not entrant" messages [4]). >>>>>>> >>>>>>> Let's wait for James' opinion on this, but I think we should address >>>>>>> this >>>>>>> issue on JBoss Logging/LogManager side (so I agree with David here) and >>>>>>> make >>>>>>> those parts inlinable (wow, does this word even exist? :D). Once this >>>>>>> is >>>>>>> done, we could experiment further in Infinispan codebase and see how >>>>>>> this >>>>>>> relates to some real world benchmarks... >>>>>>> >>>>>>> Thanks >>>>>>> Sebastian >>>>>>> >>>>>>> [1] https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a53c >>>>>>> [2] Test repository: >>>>>>> https://github.com/slaskawi/jboss-logging-perf-test >>>>>>> [3] https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924bae0 >>>>>>> [4] >>>>>>> >>>>>>> https://www.safaribooksonline.com/library/view/java-performance-the/9781449363512/ch04.html >>>>>>> >>>>>>> On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec >>>>>>> >>>>>>> wrote: >>>>>>>> Hey! >>>>>>>> >>>>>>>> Adding James to this thread. >>>>>>>> >>>>>>>> @Dennis - I think Dan has a point here. The trick with checking a >>>>>>>> field in >>>>>>>> a class is 3 times faster. Most of the checks are done in core so they >>>>>>>> are >>>>>>>> executed multiple times per operation. Changing all those places is >>>>>>>> probably >>>>>>>> not an option. >>>>>>>> >>>>>>>> @David - Let me run a test with JBoss Log Manager and get back to you >>>>>>>> with >>>>>>>> some results. But if Dan is right, and the problem is with enum >>>>>>>> mapping, I >>>>>>>> will get similar results. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Sebastian >>>>>>>> >>>>>>>> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei >>>>>>>> wrote: >>>>>>>>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>>> The performance problem that this trick is meant to resolve is >>>>>>>>>> really a >>>>>>>>>> problem in the logging backend. It *should* be faster inside of >>>>>>>>>> WildFly, where JBoss LogManager is used, because that project just >>>>>>>>>> checks a single volatile field for the level check... and the path >>>>>>>>>> to >>>>>>>>>> that code *should* be inline-friendly. >>>>>>>>>> >>>>>>>>> Indeed, we started using this trick because of log4j 1.2, which needs >>>>>>>>> to walk the logger hierarchy in order to check the level, and it made >>>>>>>>> a significant difference there. >>>>>>>>> >>>>>>>>> Nowadays I think it's pretty close to optimal in all logging >>>>>>>>> frameworks. The only nitpick is that they all use enums for the >>>>>>>>> levels, and the JIT can't inline Level.TRACE.value as it would with a >>>>>>>>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's going >>>>>>>>> to >>>>>>>>> be more or less equivalent to using a volatile "trace" field in each >>>>>>>>> class, so it should be suitable even for local read operations that >>>>>>>>> take < 200ns. >>>>>>>>> >>>>>>>>> We'd probably still need to weed out some of the trace messages, as >>>>>>>>> we >>>>>>>>> probably have more than 10 of them during such a read operation. I >>>>>>>>> confess that I added way too many trace logs myself, precisely >>>>>>>>> because >>>>>>>>> I knew we are using a static final field and the JIT compiler doesn't >>>>>>>>> even generate code for that branch. >>>>>>>>> >>>>>>>>> Cheers >>>>>>>>> Dan >>>>>>>>> _______________________________________________ >>>>>>>>> infinispan-dev mailing list >>>>>>>>> infinispan-dev at lists.jboss.org >>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>> _______________________________________________ >>>>>>> infinispan-dev mailing list >>>>>>> infinispan-dev at lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> _______________________________________________ >>>>>> infinispan-dev mailing list >>>>>> infinispan-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From slaskawi at redhat.com Tue Oct 4 03:44:24 2016 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Tue, 4 Oct 2016 09:44:24 +0200 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: <57F2960C.6000705@redhat.com> References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> <57F26092.3050705@redhat.com> <57F2960C.6000705@redhat.com> Message-ID: I think we reached the point where we mention the same arguments over and over again... So let me try to sum it up and possibly iron an action plan out: - The main reason we cache the "isTraceEnabled" value is speed. It's more than 3 times faster then checking it before each trace invocation. - This decreases maintainability since we can't change the logging level in runtime. Dennis filed a JIRA for it: https://issues.jboss.org/ browse/JBCACHE-1625 - It doesn't matter which logging backend we use (JDK, JBoss etc). - The main problem is the "translate" method in logger, which translates JBoss Logging levels into backend's native levels. The "translate" method is used for all logging levels (since "doLogf" method checks "isEnabled" method result, which in turn calls "translate"). - As shown on benchmarks with JIT activity, there are two reasons why "translate" method is not inlined - it's too big and it contains a switch statement. If you agree with the above, I would like to propose the following action plan: - @Dennis - please move the JBCACHE-1625 to Infinispan project and assign it to me. - @James - I would like to ask you to experiment with optimizing these methods. It would be great if both "translate" and "isEnabled" methods could be inlined. I would be more than happy to verify if it works in both my private benchmarks and if that goes well, with Infinispan stress tests. Thanks Sebastian On Mon, Oct 3, 2016 at 7:31 PM, Dennis Reed wrote: > On 10/03/2016 08:43 AM, Radim Vansa wrote: > > Aren't we investing a bit too much time to this? The trace level was > > cached for ages, has anyone ever complained? > > I have. https://issues.jboss.org/browse/JBCACHE-1625 > > Looks like I just never copied it over to the ISPN JIRA. :) > > > Turning on trace level is completely impractical in production, > > We do it all the time when there's no better option to debug an issue. > (generally for as short a period as possible if it's production) > > -Dennis > > > but that info (or, I'd say 80% of > > that) is very useful when debugging race conditions etc., so it makes > > sense to keep it there and reduce the cost to minimum. > > > > My 2c > > > > Radim > > > > On 10/03/2016 03:25 PM, Dan Berindei wrote: > >> On Mon, Oct 3, 2016 at 3:58 PM, Sanne Grinovero > wrote: > >>> I've not looked at any such details but I always assumed that since > the > >>> logger frameworks (any of them) need to be able to change the logging > level, > >>> there must be some way of switching state - at least a volatile read? > >>> > >>> And while that could be very cheap, I doubt it would be as easily > optimised > >>> as a constant. > >>> > >> True, at some point you need a volatile read -- you can only avoid > >> that by caching isTraceEnabled(), like we do in Infinispan. > >> > >> My point was related to this assembly instruction, which reads the > >> ordinal of Level.TRACE: > >> > >> 0x00007f93951e9fdf: mov rdi,0x76d4e6d48 ; {oop(a > >> 'org/jboss/logging/Logger$Level')} > >> 0x00007f93951e9fe9: mov ebp << missing an operand here >>; - > >> java.lang.Enum::ordinal at 1 (line 103) > >> ; - > >> org.jboss.logging.JBossLogManagerLogger::translate at 8 (line 58) > >> ; - > >> org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) > >> ; - org.jboss.logging.Logger::tracef at 4 > (line 287) > >> ; - org.infinispan.MyBenchmark::tracef at 7 > (line 26) > >> ; - > >> org.infinispan.generated.MyBenchmark_tracef_jmhTest:: > tracef_thrpt_jmhStub at 15 > >> (line 113) > >> > >> In an ideal world, the JIT would replace translate(Level.TRACE) with a > >> constant. But here, even though `0x76d4e6d48` is clearly a constant, > >> the ordinal is still loaded and checked every time. > >> > >>> Since we might not be interested only in the logger method to be > inlined, we > >>> also want most of the client methods to be considered for inlining.. > some of > >>> these can be extremely sensitive so such a decision requires a full > >>> Infinispan performance test rather than a microbenchmark. (which is > useful > >>> too but more for the sake of optimising JBoss Logging, not to proof if > it's > >>> OK to switch all of Infinispan 's internal conventions) > >>> > >> Based on the testing both myself and Radim did WRT inlining, it > >> doesn't make a huge difference. But I agree 100% that before removing > >> the isTraceEnabled() caching we'd have to benchmark the whole of > >> Infinispan, and not just JBoss Logging in isolation. > >> > >> Cheers > >> Dan > >> > >>> On Mon, 3 Oct 2016, 13:45 Sebastian Laskawiec, > wrote: > >>>> Awesome Dan! May I ask for a Pull Request: > >>>> https://github.com/slaskawi/jboss-logging-perf-test? > >>>> > >>>> > >>>> > >>>> On Mon, Oct 3, 2016 at 2:40 PM, Dan Berindei > >>>> wrote: > >>>>> Hi Sebastian > >>>>> > >>>>> I modified your benchmark so that the logger and the trace field are > >>>>> static final and looked at the generated assembly with JITWatch [1]. > >>>>> Based on the attached assembly listings, caching isTraceEnabled() in > a > >>>>> constant field is "infinitely faster", because there are no assembly > >>>>> instructions that can be traced back to the test method. > >>>>> > >>>>> JBossLogManagerLogger::translate() is inlined in this listing, but > it > >>>>> still goes through the switch machinery, I'm guessing because the > >>>>> ordinal of an Enum is not considered a constant. > >>>>> > >>>>> Cheers > >>>>> Dan > >>>>> > >>>>> > >>>>> [1]: https://github.com/AdoptOpenJDK/jitwatch > >>>>> > >>>>> On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec > >>>>> wrote: > >>>>>> Hey! > >>>>>> > >>>>>> Please have a look at the latest perf test results [1][2]: > >>>>>> > >>>>>> Benchmark Mode Cnt Score > >>>>>> Error Units > >>>>>> MyBenchmark.noVariable thrpt 20 681131269.875 ? > >>>>>> 3961932.923 ops/s > >>>>>> MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 ? > >>>>>> 14305970.393 ops/s > >>>>>> MyBenchmark.withVariable thrpt 20 2411000894.582 ? > >>>>>> 17382438.635 ops/s > >>>>>> > >>>>>> I think there is no surprise here.. using a field, which stores the > >>>>>> result > >>>>>> of `logger.isTraceEnabled()` evaluation is 3 times faster than other > >>>>>> options. > >>>>>> > >>>>>> If anyone is interested in printing out JIT stuff, I also ran it > with > >>>>>> "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and > >>>>>> "-XX:+PrintInlining" > >>>>>> here [3]. > >>>>>> > >>>>>> I'm not a performance expert but it seems that JIT could not inline > the > >>>>>> "translate" method because of its size (see line 1861). However it > >>>>>> tried > >>>>>> several times with different optimizations (and some of them were > >>>>>> thrown > >>>>>> away - "made not entrant" messages [4]). > >>>>>> > >>>>>> Let's wait for James' opinion on this, but I think we should address > >>>>>> this > >>>>>> issue on JBoss Logging/LogManager side (so I agree with David here) > and > >>>>>> make > >>>>>> those parts inlinable (wow, does this word even exist? :D). Once > this > >>>>>> is > >>>>>> done, we could experiment further in Infinispan codebase and see how > >>>>>> this > >>>>>> relates to some real world benchmarks... > >>>>>> > >>>>>> Thanks > >>>>>> Sebastian > >>>>>> > >>>>>> [1] https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a5 > 3c > >>>>>> [2] Test repository: > >>>>>> https://github.com/slaskawi/jboss-logging-perf-test > >>>>>> [3] https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924ba > e0 > >>>>>> [4] > >>>>>> > >>>>>> https://www.safaribooksonline.com/library/view/java- > performance-the/9781449363512/ch04.html > >>>>>> > >>>>>> On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec > >>>>>> > >>>>>> wrote: > >>>>>>> Hey! > >>>>>>> > >>>>>>> Adding James to this thread. > >>>>>>> > >>>>>>> @Dennis - I think Dan has a point here. The trick with checking a > >>>>>>> field in > >>>>>>> a class is 3 times faster. Most of the checks are done in core so > they > >>>>>>> are > >>>>>>> executed multiple times per operation. Changing all those places is > >>>>>>> probably > >>>>>>> not an option. > >>>>>>> > >>>>>>> @David - Let me run a test with JBoss Log Manager and get back to > you > >>>>>>> with > >>>>>>> some results. But if Dan is right, and the problem is with enum > >>>>>>> mapping, I > >>>>>>> will get similar results. > >>>>>>> > >>>>>>> Thanks > >>>>>>> Sebastian > >>>>>>> > >>>>>>> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei < > dan.berindei at gmail.com> > >>>>>>> wrote: > >>>>>>>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd > >>>>>>>> > >>>>>>>> wrote: > >>>>>>>>> The performance problem that this trick is meant to resolve is > >>>>>>>>> really a > >>>>>>>>> problem in the logging backend. It *should* be faster inside of > >>>>>>>>> WildFly, where JBoss LogManager is used, because that project > just > >>>>>>>>> checks a single volatile field for the level check... and the > path > >>>>>>>>> to > >>>>>>>>> that code *should* be inline-friendly. > >>>>>>>>> > >>>>>>>> Indeed, we started using this trick because of log4j 1.2, which > needs > >>>>>>>> to walk the logger hierarchy in order to check the level, and it > made > >>>>>>>> a significant difference there. > >>>>>>>> > >>>>>>>> Nowadays I think it's pretty close to optimal in all logging > >>>>>>>> frameworks. The only nitpick is that they all use enums for the > >>>>>>>> levels, and the JIT can't inline Level.TRACE.value as it would > with a > >>>>>>>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's > going > >>>>>>>> to > >>>>>>>> be more or less equivalent to using a volatile "trace" field in > each > >>>>>>>> class, so it should be suitable even for local read operations > that > >>>>>>>> take < 200ns. > >>>>>>>> > >>>>>>>> We'd probably still need to weed out some of the trace messages, > as > >>>>>>>> we > >>>>>>>> probably have more than 10 of them during such a read operation. I > >>>>>>>> confess that I added way too many trace logs myself, precisely > >>>>>>>> because > >>>>>>>> I knew we are using a static final field and the JIT compiler > doesn't > >>>>>>>> even generate code for that branch. > >>>>>>>> > >>>>>>>> Cheers > >>>>>>>> Dan > >>>>>>>> _______________________________________________ > >>>>>>>> infinispan-dev mailing list > >>>>>>>> infinispan-dev at lists.jboss.org > >>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>>>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>>> infinispan-dev mailing list > >>>>>> infinispan-dev at lists.jboss.org > >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>>>> _______________________________________________ > >>>>> infinispan-dev mailing list > >>>>> infinispan-dev at lists.jboss.org > >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>>> > >>>> _______________________________________________ > >>>> infinispan-dev mailing list > >>>> infinispan-dev at lists.jboss.org > >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161004/e8f64765/attachment-0001.html From dan.berindei at gmail.com Tue Oct 4 06:24:59 2016 From: dan.berindei at gmail.com (Dan Berindei) Date: Tue, 4 Oct 2016 13:24:59 +0300 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> <57F26092.3050705@redhat.com> <57F2960C.6000705@redhat.com> Message-ID: On Tue, Oct 4, 2016 at 10:44 AM, Sebastian Laskawiec wrote: > I think we reached the point where we mention the same arguments over and > over again... > > So let me try to sum it up and possibly iron an action plan out: > > The main reason we cache the "isTraceEnabled" value is speed. It's more than > 3 times faster then checking it before each trace invocation. Actually, if you look at the the assembly code in const.txt, all the assembly instructions are from JMH itself, not from the test method. So caching in a static final is "infinitely faster", not just 3 times faster. > This decreases maintainability since we can't change the logging level in > runtime. Dennis filed a JIRA for it: > https://issues.jboss.org/browse/JBCACHE-1625 We can change the logging level to DEBUG at runtime, which I would argue is enough in most cases. > It doesn't matter which logging backend we use (JDK, JBoss etc). > The main problem is the "translate" method in logger, which translates JBoss > Logging levels into backend's native levels. The "translate" method is used > for all logging levels (since "doLogf" method checks "isEnabled" method > result, which in turn calls "translate"). > As shown on benchmarks with JIT activity, there are two reasons why > "translate" method is not inlined - it's too big and it contains a switch > statement. > Hmmm, I thought the assembly listing in tracef.txt showed that it *is* inlined :) > If you agree with the above, I would like to propose the following action > plan: > > @Dennis - please move the JBCACHE-1625 to Infinispan project and assign it > to me. > @James - I would like to ask you to experiment with optimizing these > methods. It would be great if both "translate" and "isEnabled" methods could > be inlined. I would be more than happy to verify if it works in both my > private benchmarks and if that goes well, with Infinispan stress tests. > IMO there are way too many TRACE logs in the core, and I don't think we can avoid affecting performance without removing 80% of them. And I also think the time spent arguing about this would have been better spent trying to fix some of the random failures in the test suite, i.e. the thing that the excessive TRACE logs are supposed to help us with :) Cheers Dan > Thanks > Sebastian > > On Mon, Oct 3, 2016 at 7:31 PM, Dennis Reed wrote: >> >> On 10/03/2016 08:43 AM, Radim Vansa wrote: >> > Aren't we investing a bit too much time to this? The trace level was >> > cached for ages, has anyone ever complained? >> >> I have. https://issues.jboss.org/browse/JBCACHE-1625 >> >> Looks like I just never copied it over to the ISPN JIRA. :) >> >> > Turning on trace level is completely impractical in production, >> >> We do it all the time when there's no better option to debug an issue. >> (generally for as short a period as possible if it's production) >> >> -Dennis >> >> > but that info (or, I'd say 80% of >> > that) is very useful when debugging race conditions etc., so it makes >> > sense to keep it there and reduce the cost to minimum. >> > >> > My 2c >> > >> > Radim >> > >> > On 10/03/2016 03:25 PM, Dan Berindei wrote: >> >> On Mon, Oct 3, 2016 at 3:58 PM, Sanne Grinovero >> >> wrote: >> >>> I've not looked at any such details but I always assumed that since >> >>> the >> >>> logger frameworks (any of them) need to be able to change the logging >> >>> level, >> >>> there must be some way of switching state - at least a volatile read? >> >>> >> >>> And while that could be very cheap, I doubt it would be as easily >> >>> optimised >> >>> as a constant. >> >>> >> >> True, at some point you need a volatile read -- you can only avoid >> >> that by caching isTraceEnabled(), like we do in Infinispan. >> >> >> >> My point was related to this assembly instruction, which reads the >> >> ordinal of Level.TRACE: >> >> >> >> 0x00007f93951e9fdf: mov rdi,0x76d4e6d48 ; {oop(a >> >> 'org/jboss/logging/Logger$Level')} >> >> 0x00007f93951e9fe9: mov ebp << missing an operand here >>; - >> >> java.lang.Enum::ordinal at 1 (line 103) >> >> ; - >> >> org.jboss.logging.JBossLogManagerLogger::translate at 8 (line 58) >> >> ; - >> >> org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) >> >> ; - org.jboss.logging.Logger::tracef at 4 >> >> (line 287) >> >> ; - org.infinispan.MyBenchmark::tracef at 7 >> >> (line 26) >> >> ; - >> >> >> >> org.infinispan.generated.MyBenchmark_tracef_jmhTest::tracef_thrpt_jmhStub at 15 >> >> (line 113) >> >> >> >> In an ideal world, the JIT would replace translate(Level.TRACE) with a >> >> constant. But here, even though `0x76d4e6d48` is clearly a constant, >> >> the ordinal is still loaded and checked every time. >> >> >> >>> Since we might not be interested only in the logger method to be >> >>> inlined, we >> >>> also want most of the client methods to be considered for inlining.. >> >>> some of >> >>> these can be extremely sensitive so such a decision requires a full >> >>> Infinispan performance test rather than a microbenchmark. (which is >> >>> useful >> >>> too but more for the sake of optimising JBoss Logging, not to proof if >> >>> it's >> >>> OK to switch all of Infinispan 's internal conventions) >> >>> >> >> Based on the testing both myself and Radim did WRT inlining, it >> >> doesn't make a huge difference. But I agree 100% that before removing >> >> the isTraceEnabled() caching we'd have to benchmark the whole of >> >> Infinispan, and not just JBoss Logging in isolation. >> >> >> >> Cheers >> >> Dan >> >> >> >>> On Mon, 3 Oct 2016, 13:45 Sebastian Laskawiec, >> >>> wrote: >> >>>> Awesome Dan! May I ask for a Pull Request: >> >>>> https://github.com/slaskawi/jboss-logging-perf-test? >> >>>> >> >>>> >> >>>> >> >>>> On Mon, Oct 3, 2016 at 2:40 PM, Dan Berindei >> >>>> wrote: >> >>>>> Hi Sebastian >> >>>>> >> >>>>> I modified your benchmark so that the logger and the trace field are >> >>>>> static final and looked at the generated assembly with JITWatch [1]. >> >>>>> Based on the attached assembly listings, caching isTraceEnabled() in >> >>>>> a >> >>>>> constant field is "infinitely faster", because there are no assembly >> >>>>> instructions that can be traced back to the test method. >> >>>>> >> >>>>> JBossLogManagerLogger::translate() is inlined in this listing, but >> >>>>> it >> >>>>> still goes through the switch machinery, I'm guessing because the >> >>>>> ordinal of an Enum is not considered a constant. >> >>>>> >> >>>>> Cheers >> >>>>> Dan >> >>>>> >> >>>>> >> >>>>> [1]: https://github.com/AdoptOpenJDK/jitwatch >> >>>>> >> >>>>> On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec >> >>>>> wrote: >> >>>>>> Hey! >> >>>>>> >> >>>>>> Please have a look at the latest perf test results [1][2]: >> >>>>>> >> >>>>>> Benchmark Mode Cnt Score >> >>>>>> Error Units >> >>>>>> MyBenchmark.noVariable thrpt 20 681131269.875 ? >> >>>>>> 3961932.923 ops/s >> >>>>>> MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 ? >> >>>>>> 14305970.393 ops/s >> >>>>>> MyBenchmark.withVariable thrpt 20 2411000894.582 ? >> >>>>>> 17382438.635 ops/s >> >>>>>> >> >>>>>> I think there is no surprise here.. using a field, which stores the >> >>>>>> result >> >>>>>> of `logger.isTraceEnabled()` evaluation is 3 times faster than >> >>>>>> other >> >>>>>> options. >> >>>>>> >> >>>>>> If anyone is interested in printing out JIT stuff, I also ran it >> >>>>>> with >> >>>>>> "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and >> >>>>>> "-XX:+PrintInlining" >> >>>>>> here [3]. >> >>>>>> >> >>>>>> I'm not a performance expert but it seems that JIT could not inline >> >>>>>> the >> >>>>>> "translate" method because of its size (see line 1861). However it >> >>>>>> tried >> >>>>>> several times with different optimizations (and some of them were >> >>>>>> thrown >> >>>>>> away - "made not entrant" messages [4]). >> >>>>>> >> >>>>>> Let's wait for James' opinion on this, but I think we should >> >>>>>> address >> >>>>>> this >> >>>>>> issue on JBoss Logging/LogManager side (so I agree with David here) >> >>>>>> and >> >>>>>> make >> >>>>>> those parts inlinable (wow, does this word even exist? :D). Once >> >>>>>> this >> >>>>>> is >> >>>>>> done, we could experiment further in Infinispan codebase and see >> >>>>>> how >> >>>>>> this >> >>>>>> relates to some real world benchmarks... >> >>>>>> >> >>>>>> Thanks >> >>>>>> Sebastian >> >>>>>> >> >>>>>> [1] >> >>>>>> https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a53c >> >>>>>> [2] Test repository: >> >>>>>> https://github.com/slaskawi/jboss-logging-perf-test >> >>>>>> [3] >> >>>>>> https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924bae0 >> >>>>>> [4] >> >>>>>> >> >>>>>> >> >>>>>> https://www.safaribooksonline.com/library/view/java-performance-the/9781449363512/ch04.html >> >>>>>> >> >>>>>> On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec >> >>>>>> >> >>>>>> wrote: >> >>>>>>> Hey! >> >>>>>>> >> >>>>>>> Adding James to this thread. >> >>>>>>> >> >>>>>>> @Dennis - I think Dan has a point here. The trick with checking a >> >>>>>>> field in >> >>>>>>> a class is 3 times faster. Most of the checks are done in core so >> >>>>>>> they >> >>>>>>> are >> >>>>>>> executed multiple times per operation. Changing all those places >> >>>>>>> is >> >>>>>>> probably >> >>>>>>> not an option. >> >>>>>>> >> >>>>>>> @David - Let me run a test with JBoss Log Manager and get back to >> >>>>>>> you >> >>>>>>> with >> >>>>>>> some results. But if Dan is right, and the problem is with enum >> >>>>>>> mapping, I >> >>>>>>> will get similar results. >> >>>>>>> >> >>>>>>> Thanks >> >>>>>>> Sebastian >> >>>>>>> >> >>>>>>> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei >> >>>>>>> >> >>>>>>> wrote: >> >>>>>>>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd >> >>>>>>>> >> >>>>>>>> wrote: >> >>>>>>>>> The performance problem that this trick is meant to resolve is >> >>>>>>>>> really a >> >>>>>>>>> problem in the logging backend. It *should* be faster inside of >> >>>>>>>>> WildFly, where JBoss LogManager is used, because that project >> >>>>>>>>> just >> >>>>>>>>> checks a single volatile field for the level check... and the >> >>>>>>>>> path >> >>>>>>>>> to >> >>>>>>>>> that code *should* be inline-friendly. >> >>>>>>>>> >> >>>>>>>> Indeed, we started using this trick because of log4j 1.2, which >> >>>>>>>> needs >> >>>>>>>> to walk the logger hierarchy in order to check the level, and it >> >>>>>>>> made >> >>>>>>>> a significant difference there. >> >>>>>>>> >> >>>>>>>> Nowadays I think it's pretty close to optimal in all logging >> >>>>>>>> frameworks. The only nitpick is that they all use enums for the >> >>>>>>>> levels, and the JIT can't inline Level.TRACE.value as it would >> >>>>>>>> with a >> >>>>>>>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's >> >>>>>>>> going >> >>>>>>>> to >> >>>>>>>> be more or less equivalent to using a volatile "trace" field in >> >>>>>>>> each >> >>>>>>>> class, so it should be suitable even for local read operations >> >>>>>>>> that >> >>>>>>>> take < 200ns. >> >>>>>>>> >> >>>>>>>> We'd probably still need to weed out some of the trace messages, >> >>>>>>>> as >> >>>>>>>> we >> >>>>>>>> probably have more than 10 of them during such a read operation. >> >>>>>>>> I >> >>>>>>>> confess that I added way too many trace logs myself, precisely >> >>>>>>>> because >> >>>>>>>> I knew we are using a static final field and the JIT compiler >> >>>>>>>> doesn't >> >>>>>>>> even generate code for that branch. >> >>>>>>>> >> >>>>>>>> Cheers >> >>>>>>>> Dan >> >>>>>>>> _______________________________________________ >> >>>>>>>> infinispan-dev mailing list >> >>>>>>>> infinispan-dev at lists.jboss.org >> >>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >>>>>>> >> >>>>>> >> >>>>>> _______________________________________________ >> >>>>>> infinispan-dev mailing list >> >>>>>> infinispan-dev at lists.jboss.org >> >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >>>>> _______________________________________________ >> >>>>> infinispan-dev mailing list >> >>>>> infinispan-dev at lists.jboss.org >> >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >>>> >> >>>> _______________________________________________ >> >>>> infinispan-dev mailing list >> >>>> infinispan-dev at lists.jboss.org >> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >>> >> >>> _______________________________________________ >> >>> infinispan-dev mailing list >> >>> infinispan-dev at lists.jboss.org >> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From slaskawi at redhat.com Tue Oct 4 07:08:38 2016 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Tue, 4 Oct 2016 13:08:38 +0200 Subject: [infinispan-dev] if (trace) logger.tracef - it makes sense In-Reply-To: References: <50e73fd2-49b8-55ae-a7b4-91345b44a7e9@redhat.com> <57EE8FF3.9020303@redhat.com> <57F26092.3050705@redhat.com> <57F2960C.6000705@redhat.com> Message-ID: ahhh, for some reason I missed that. Sorry guys... In that case there's probably not much we can do. The only missing bit is what to do with JBCACHE-1625? On Tue, Oct 4, 2016 at 12:24 PM, Dan Berindei wrote: > On Tue, Oct 4, 2016 at 10:44 AM, Sebastian Laskawiec > wrote: > > I think we reached the point where we mention the same arguments over and > > over again... > > > > So let me try to sum it up and possibly iron an action plan out: > > > > The main reason we cache the "isTraceEnabled" value is speed. It's more > than > > 3 times faster then checking it before each trace invocation. > > Actually, if you look at the the assembly code in const.txt, all the > assembly instructions are from JMH itself, not from the test method. > So caching in a static final is "infinitely faster", not just 3 times > faster. > > > This decreases maintainability since we can't change the logging level in > > runtime. Dennis filed a JIRA for it: > > https://issues.jboss.org/browse/JBCACHE-1625 > > We can change the logging level to DEBUG at runtime, which I would > argue is enough in most cases. > > > It doesn't matter which logging backend we use (JDK, JBoss etc). > > The main problem is the "translate" method in logger, which translates > JBoss > > Logging levels into backend's native levels. The "translate" method is > used > > for all logging levels (since "doLogf" method checks "isEnabled" method > > result, which in turn calls "translate"). > > As shown on benchmarks with JIT activity, there are two reasons why > > "translate" method is not inlined - it's too big and it contains a switch > > statement. > > > > Hmmm, I thought the assembly listing in tracef.txt showed that it *is* > inlined :) > > > If you agree with the above, I would like to propose the following action > > plan: > > > > @Dennis - please move the JBCACHE-1625 to Infinispan project and assign > it > > to me. > > @James - I would like to ask you to experiment with optimizing these > > methods. It would be great if both "translate" and "isEnabled" methods > could > > be inlined. I would be more than happy to verify if it works in both my > > private benchmarks and if that goes well, with Infinispan stress tests. > > > > IMO there are way too many TRACE logs in the core, and I don't think > we can avoid affecting performance without removing 80% of them. And I > also think the time spent arguing about this would have been better > spent trying to fix some of the random failures in the test suite, > i.e. the thing that the excessive TRACE logs are supposed to help us > with :) > > Cheers > Dan > > > > Thanks > > Sebastian > > > > On Mon, Oct 3, 2016 at 7:31 PM, Dennis Reed wrote: > >> > >> On 10/03/2016 08:43 AM, Radim Vansa wrote: > >> > Aren't we investing a bit too much time to this? The trace level was > >> > cached for ages, has anyone ever complained? > >> > >> I have. https://issues.jboss.org/browse/JBCACHE-1625 > >> > >> Looks like I just never copied it over to the ISPN JIRA. :) > >> > >> > Turning on trace level is completely impractical in production, > >> > >> We do it all the time when there's no better option to debug an issue. > >> (generally for as short a period as possible if it's production) > >> > >> -Dennis > >> > >> > but that info (or, I'd say 80% of > >> > that) is very useful when debugging race conditions etc., so it makes > >> > sense to keep it there and reduce the cost to minimum. > >> > > >> > My 2c > >> > > >> > Radim > >> > > >> > On 10/03/2016 03:25 PM, Dan Berindei wrote: > >> >> On Mon, Oct 3, 2016 at 3:58 PM, Sanne Grinovero < > sanne at infinispan.org> > >> >> wrote: > >> >>> I've not looked at any such details but I always assumed that since > >> >>> the > >> >>> logger frameworks (any of them) need to be able to change the > logging > >> >>> level, > >> >>> there must be some way of switching state - at least a volatile > read? > >> >>> > >> >>> And while that could be very cheap, I doubt it would be as easily > >> >>> optimised > >> >>> as a constant. > >> >>> > >> >> True, at some point you need a volatile read -- you can only avoid > >> >> that by caching isTraceEnabled(), like we do in Infinispan. > >> >> > >> >> My point was related to this assembly instruction, which reads the > >> >> ordinal of Level.TRACE: > >> >> > >> >> 0x00007f93951e9fdf: mov rdi,0x76d4e6d48 ; {oop(a > >> >> 'org/jboss/logging/Logger$Level')} > >> >> 0x00007f93951e9fe9: mov ebp << missing an operand here >>; - > >> >> java.lang.Enum::ordinal at 1 (line 103) > >> >> ; - > >> >> org.jboss.logging.JBossLogManagerLogger::translate at 8 (line 58) > >> >> ; - > >> >> org.jboss.logging.JBossLogManagerLogger::isEnabled at 5 (line 35) > >> >> ; - org.jboss.logging.Logger::tracef at 4 > >> >> (line 287) > >> >> ; - org.infinispan.MyBenchmark:: > tracef at 7 > >> >> (line 26) > >> >> ; - > >> >> > >> >> org.infinispan.generated.MyBenchmark_tracef_jmhTest:: > tracef_thrpt_jmhStub at 15 > >> >> (line 113) > >> >> > >> >> In an ideal world, the JIT would replace translate(Level.TRACE) with > a > >> >> constant. But here, even though `0x76d4e6d48` is clearly a constant, > >> >> the ordinal is still loaded and checked every time. > >> >> > >> >>> Since we might not be interested only in the logger method to be > >> >>> inlined, we > >> >>> also want most of the client methods to be considered for inlining.. > >> >>> some of > >> >>> these can be extremely sensitive so such a decision requires a full > >> >>> Infinispan performance test rather than a microbenchmark. (which is > >> >>> useful > >> >>> too but more for the sake of optimising JBoss Logging, not to proof > if > >> >>> it's > >> >>> OK to switch all of Infinispan 's internal conventions) > >> >>> > >> >> Based on the testing both myself and Radim did WRT inlining, it > >> >> doesn't make a huge difference. But I agree 100% that before removing > >> >> the isTraceEnabled() caching we'd have to benchmark the whole of > >> >> Infinispan, and not just JBoss Logging in isolation. > >> >> > >> >> Cheers > >> >> Dan > >> >> > >> >>> On Mon, 3 Oct 2016, 13:45 Sebastian Laskawiec, > > >> >>> wrote: > >> >>>> Awesome Dan! May I ask for a Pull Request: > >> >>>> https://github.com/slaskawi/jboss-logging-perf-test? > >> >>>> > >> >>>> > >> >>>> > >> >>>> On Mon, Oct 3, 2016 at 2:40 PM, Dan Berindei < > dan.berindei at gmail.com> > >> >>>> wrote: > >> >>>>> Hi Sebastian > >> >>>>> > >> >>>>> I modified your benchmark so that the logger and the trace field > are > >> >>>>> static final and looked at the generated assembly with JITWatch > [1]. > >> >>>>> Based on the attached assembly listings, caching isTraceEnabled() > in > >> >>>>> a > >> >>>>> constant field is "infinitely faster", because there are no > assembly > >> >>>>> instructions that can be traced back to the test method. > >> >>>>> > >> >>>>> JBossLogManagerLogger::translate() is inlined in this listing, > but > >> >>>>> it > >> >>>>> still goes through the switch machinery, I'm guessing because the > >> >>>>> ordinal of an Enum is not considered a constant. > >> >>>>> > >> >>>>> Cheers > >> >>>>> Dan > >> >>>>> > >> >>>>> > >> >>>>> [1]: https://github.com/AdoptOpenJDK/jitwatch > >> >>>>> > >> >>>>> On Mon, Oct 3, 2016 at 11:28 AM, Sebastian Laskawiec > >> >>>>> wrote: > >> >>>>>> Hey! > >> >>>>>> > >> >>>>>> Please have a look at the latest perf test results [1][2]: > >> >>>>>> > >> >>>>>> Benchmark Mode Cnt Score > >> >>>>>> Error Units > >> >>>>>> MyBenchmark.noVariable thrpt 20 681131269.875 ? > >> >>>>>> 3961932.923 ops/s > >> >>>>>> MyBenchmark.withIsTraceEnabledCheck thrpt 20 676307984.921 > ? > >> >>>>>> 14305970.393 ops/s > >> >>>>>> MyBenchmark.withVariable thrpt 20 2411000894.582 ? > >> >>>>>> 17382438.635 ops/s > >> >>>>>> > >> >>>>>> I think there is no surprise here.. using a field, which stores > the > >> >>>>>> result > >> >>>>>> of `logger.isTraceEnabled()` evaluation is 3 times faster than > >> >>>>>> other > >> >>>>>> options. > >> >>>>>> > >> >>>>>> If anyone is interested in printing out JIT stuff, I also ran it > >> >>>>>> with > >> >>>>>> "-XX:+PrintCompilation", "-XX:+PrintCompilation2" and > >> >>>>>> "-XX:+PrintInlining" > >> >>>>>> here [3]. > >> >>>>>> > >> >>>>>> I'm not a performance expert but it seems that JIT could not > inline > >> >>>>>> the > >> >>>>>> "translate" method because of its size (see line 1861). However > it > >> >>>>>> tried > >> >>>>>> several times with different optimizations (and some of them were > >> >>>>>> thrown > >> >>>>>> away - "made not entrant" messages [4]). > >> >>>>>> > >> >>>>>> Let's wait for James' opinion on this, but I think we should > >> >>>>>> address > >> >>>>>> this > >> >>>>>> issue on JBoss Logging/LogManager side (so I agree with David > here) > >> >>>>>> and > >> >>>>>> make > >> >>>>>> those parts inlinable (wow, does this word even exist? :D). Once > >> >>>>>> this > >> >>>>>> is > >> >>>>>> done, we could experiment further in Infinispan codebase and see > >> >>>>>> how > >> >>>>>> this > >> >>>>>> relates to some real world benchmarks... > >> >>>>>> > >> >>>>>> Thanks > >> >>>>>> Sebastian > >> >>>>>> > >> >>>>>> [1] > >> >>>>>> https://gist.github.com/slaskawi/6766b6e17c7a28ac8d8962293c48a5 > 3c > >> >>>>>> [2] Test repository: > >> >>>>>> https://github.com/slaskawi/jboss-logging-perf-test > >> >>>>>> [3] > >> >>>>>> https://gist.github.com/slaskawi/6f317bb05539611434bc91d66924ba > e0 > >> >>>>>> [4] > >> >>>>>> > >> >>>>>> > >> >>>>>> https://www.safaribooksonline.com/library/view/java- > performance-the/9781449363512/ch04.html > >> >>>>>> > >> >>>>>> On Mon, Oct 3, 2016 at 7:32 AM, Sebastian Laskawiec > >> >>>>>> > >> >>>>>> wrote: > >> >>>>>>> Hey! > >> >>>>>>> > >> >>>>>>> Adding James to this thread. > >> >>>>>>> > >> >>>>>>> @Dennis - I think Dan has a point here. The trick with checking > a > >> >>>>>>> field in > >> >>>>>>> a class is 3 times faster. Most of the checks are done in core > so > >> >>>>>>> they > >> >>>>>>> are > >> >>>>>>> executed multiple times per operation. Changing all those places > >> >>>>>>> is > >> >>>>>>> probably > >> >>>>>>> not an option. > >> >>>>>>> > >> >>>>>>> @David - Let me run a test with JBoss Log Manager and get back > to > >> >>>>>>> you > >> >>>>>>> with > >> >>>>>>> some results. But if Dan is right, and the problem is with enum > >> >>>>>>> mapping, I > >> >>>>>>> will get similar results. > >> >>>>>>> > >> >>>>>>> Thanks > >> >>>>>>> Sebastian > >> >>>>>>> > >> >>>>>>> On Sat, Oct 1, 2016 at 11:53 AM, Dan Berindei > >> >>>>>>> > >> >>>>>>> wrote: > >> >>>>>>>> On Fri, Sep 30, 2016 at 10:15 PM, David M. Lloyd > >> >>>>>>>> > >> >>>>>>>> wrote: > >> >>>>>>>>> The performance problem that this trick is meant to resolve is > >> >>>>>>>>> really a > >> >>>>>>>>> problem in the logging backend. It *should* be faster inside > of > >> >>>>>>>>> WildFly, where JBoss LogManager is used, because that project > >> >>>>>>>>> just > >> >>>>>>>>> checks a single volatile field for the level check... and the > >> >>>>>>>>> path > >> >>>>>>>>> to > >> >>>>>>>>> that code *should* be inline-friendly. > >> >>>>>>>>> > >> >>>>>>>> Indeed, we started using this trick because of log4j 1.2, which > >> >>>>>>>> needs > >> >>>>>>>> to walk the logger hierarchy in order to check the level, and > it > >> >>>>>>>> made > >> >>>>>>>> a significant difference there. > >> >>>>>>>> > >> >>>>>>>> Nowadays I think it's pretty close to optimal in all logging > >> >>>>>>>> frameworks. The only nitpick is that they all use enums for the > >> >>>>>>>> levels, and the JIT can't inline Level.TRACE.value as it would > >> >>>>>>>> with a > >> >>>>>>>> Level.TRACE_VALUE int constant. If JDK9 fixes that, then it's > >> >>>>>>>> going > >> >>>>>>>> to > >> >>>>>>>> be more or less equivalent to using a volatile "trace" field in > >> >>>>>>>> each > >> >>>>>>>> class, so it should be suitable even for local read operations > >> >>>>>>>> that > >> >>>>>>>> take < 200ns. > >> >>>>>>>> > >> >>>>>>>> We'd probably still need to weed out some of the trace > messages, > >> >>>>>>>> as > >> >>>>>>>> we > >> >>>>>>>> probably have more than 10 of them during such a read > operation. > >> >>>>>>>> I > >> >>>>>>>> confess that I added way too many trace logs myself, precisely > >> >>>>>>>> because > >> >>>>>>>> I knew we are using a static final field and the JIT compiler > >> >>>>>>>> doesn't > >> >>>>>>>> even generate code for that branch. > >> >>>>>>>> > >> >>>>>>>> Cheers > >> >>>>>>>> Dan > >> >>>>>>>> _______________________________________________ > >> >>>>>>>> infinispan-dev mailing list > >> >>>>>>>> infinispan-dev at lists.jboss.org > >> >>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> >>>>>>> > >> >>>>>> > >> >>>>>> _______________________________________________ > >> >>>>>> infinispan-dev mailing list > >> >>>>>> infinispan-dev at lists.jboss.org > >> >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> >>>>> _______________________________________________ > >> >>>>> infinispan-dev mailing list > >> >>>>> infinispan-dev at lists.jboss.org > >> >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> >>>> > >> >>>> _______________________________________________ > >> >>>> infinispan-dev mailing list > >> >>>> infinispan-dev at lists.jboss.org > >> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> >>> > >> >>> _______________________________________________ > >> >>> infinispan-dev mailing list > >> >>> infinispan-dev at lists.jboss.org > >> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> >> _______________________________________________ > >> >> infinispan-dev mailing list > >> >> infinispan-dev at lists.jboss.org > >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > > >> > > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161004/ae37396a/attachment-0001.html From ttarrant at redhat.com Wed Oct 5 04:18:31 2016 From: ttarrant at redhat.com (Tristan Tarrant) Date: Wed, 5 Oct 2016 10:18:31 +0200 Subject: [infinispan-dev] How to improve our PR queue In-Reply-To: <57F4AE2A.1050205@redhat.com> References: <57F4AE2A.1050205@redhat.com> Message-ID: <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> [Moving to infinispan-dev] I agree with all of you. It is not respectful towards the developer who has opened the PR to let it linger for longer than necessary, and also for the developer who has opened it not to react on the comments (that happens too!) Let's set some ground rules for Pull Requests (this should go in the Contribution guide): Rules for everybody - Each and everyone of the core engineers looks at the PR queue at least once per day. More frequently if possible, especially if engaging in an active collaborative review. - We have more than one project: cachestores, console, integrations, quickstarts, clients, website... Rules for PR owners: - PRs *must* be kept "applicable" (no conflicts) - non-trivial PRs *must* have a corresponding Jira and the link to it *must* be present in the PR description. Viceversa, the Jira's workflow must be advanced to "Pull Request Sent" and include the link to the PR - if the PR is for multiple branches and it cherry-picks cleanly to all of them, no need to open multiple PRs. Use the "Backport" label to indicate this - if there is a need for separate PRs for different branches, clearly indicate the non-master branch it needs to be applied to in the PR title using the [branch] notation - Ensure the correct labels are applied when opening a PR or when the status changes - The owner *must* react to comments / CI failures / requests for rebase within 24 hours. Reaction means acknowledgement, not necessarily being able to fix the issues - If the owner cannot fix the issues related to comments / CI within a week, the PR should be closed and reopened when they can be addressed - While we still have some intermittent CI failures, we're in a situation where it is quite reliable. Exceptions obviously happen. Rules for PR reviewers - A PR should be acted upon (commented, merged) within 24 hours of its opening - A PR can be commented upon even if CI hasn't finished its job - PRs opened by external developers won't have the appropriate labels for permission reasons. Add them. - If the owner has addressed all comments / CI failures, the PR should be merged within a week - Some effects of a PR cannot be detected by CI. This for example includes verifying that docs / PDFs / etc render correctly, that distribution packages contain the appropriate files, etc. Do your best to evaluate these. With that being said, the solution for the current situation is to divide the queue among ourselves and work through it using the sprint approach (but we need to be careful with stability: "commit storms" can be detrimental). Tristan On 05/10/16 09:39, Radim Vansa wrote: > I think that can be combined: sprint just explicitly prioritizes PRs > instead of normal development, so you won't be reviewing PRs only when > "you have a free spot" but "until you can't review anything else". So > once the PR count hits a threshold, Tristan schedules sprint. > > R. > > On 10/05/2016 09:17 AM, Sebastian Laskawiec wrote: >> That's an interesting idea... but after a month or two, the PR queue >> will pile up again and we will need another sprint... >> >> IMO we should have a day-to-day strategy for doing this. >> >> On Wed, Oct 5, 2016 at 9:07 AM, Gustavo Fernandes > > wrote: >> >> I propose doing with PRs the same we've been doing to areas that >> require some care, the so called 'sprints'. >> >> We are in more than 10, a couple of PRs integrated each and we can >> get >> rid of most of the queue. >> >> Gustavo >> >> On Wed, Oct 5, 2016 at 7:36 AM, Sebastian Laskawiec >> > wrote: >> > Hey guys! >> > >> > I'm not sure what do you think about our PR queue but for me it >> simply >> > sucks... >> > >> > Some of our PRs are sitting there since I remember (like this >> one [1] - Nov >> > 2015!!! I guess we should prepare an anniversary cake, November >> will is >> > really soon :D) and some of them have more than 150 comments [2] >> (maybe this >> > sounds weird since it's my PR, but if something is not good >> enough after 150 >> > comments, maybe we should throw it away and try to implement it >> again - >> > differently?). >> > >> > After thinking about this for a little while, I would like to >> propose one >> > rule for reviewing PRs - when you have a free spot and you'd >> like to >> > integrate a couple of PRs - always start with the oldest and go >> through all >> > "Ready for review" PRs. >> > >> > I know that integrating some new stuff will be tempting but >> please don't do >> > that. Even if the new PR modifies only one small line.. noooo. >> Go through >> > some old stuff instead. This way we should revisit old PRs much >> more >> > frequently than new ones and this will force us to make some >> tough decisions >> > that we avoided for quite some time (e.g. whether or not we >> should integrate >> > [1] or [2] - but those are only examples). >> > >> > And finally, I think we should have more courage to integrate >> PRs especially >> > into master branch... After all, we have all machinery in place >> - nightly CI >> > jobs, stress tests, performance tests - what bad can happen? We >> break the >> > build? So what? We can always fix it by git revert. >> > >> > What do you think? Maybe you have better ideas? >> > >> > Thanks >> > Seb >> > >> > [1] https://github.com/infinispan/infinispan/pull/3860 >> >> > [2] https://github.com/infinispan/infinispan/pull/4348 >> >> >> > > -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From sanne at infinispan.org Wed Oct 5 05:21:50 2016 From: sanne at infinispan.org (Sanne Grinovero) Date: Wed, 5 Oct 2016 10:21:50 +0100 Subject: [infinispan-dev] How to improve our PR queue In-Reply-To: <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> References: <57F4AE2A.1050205@redhat.com> <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> Message-ID: Nice suggestions. I would be careful in actually *not* creating dedicated "pr merge sprints" as during such a sprint one would have more frequent merges, and so making the keep-up-rebasing and conflict resolving even more painful ;) I think the only solution is - like Tristan suggests - to make it a discipline to regularly check for PRs to be merged. Slightly unrelated, but it might help: did you notice that Github recently evolved their PR merge button? For trivial changes, i.e. if you're confident to not need to re-run the testsuite after a rebase, you can now just press the green button to have it rebase & merge, without creating a merge point. I just enabled the feature on the Infinispan repo ;) Sanne On 5 October 2016 at 09:18, Tristan Tarrant wrote: > [Moving to infinispan-dev] > > I agree with all of you. > > It is not respectful towards the developer who has opened the PR to let > it linger for longer than necessary, and also for the developer who has > opened it not to react on the comments (that happens too!) > > Let's set some ground rules for Pull Requests (this should go in the > Contribution guide): > > Rules for everybody > - Each and everyone of the core engineers looks at the PR queue at least > once per day. More frequently if possible, especially if engaging in an > active collaborative review. > - We have more than one project: cachestores, console, integrations, > quickstarts, clients, website... > > Rules for PR owners: > - PRs *must* be kept "applicable" (no conflicts) > - non-trivial PRs *must* have a corresponding Jira and the link to it > *must* be present in the PR description. Viceversa, the Jira's workflow > must be advanced to "Pull Request Sent" and include the link to the PR > - if the PR is for multiple branches and it cherry-picks cleanly to all > of them, no need to open multiple PRs. Use the "Backport" label to > indicate this > - if there is a need for separate PRs for different branches, clearly > indicate the non-master branch it needs to be applied to in the PR title > using the [branch] notation > - Ensure the correct labels are applied when opening a PR or when the > status changes > - The owner *must* react to comments / CI failures / requests for rebase > within 24 hours. Reaction means acknowledgement, not necessarily being > able to fix the issues > - If the owner cannot fix the issues related to comments / CI within a > week, the PR should be closed and reopened when they can be addressed > - While we still have some intermittent CI failures, we're in a > situation where it is quite reliable. Exceptions obviously happen. > > Rules for PR reviewers > - A PR should be acted upon (commented, merged) within 24 hours of its > opening > - A PR can be commented upon even if CI hasn't finished its job > - PRs opened by external developers won't have the appropriate labels > for permission reasons. Add them. > - If the owner has addressed all comments / CI failures, the PR should > be merged within a week > - Some effects of a PR cannot be detected by CI. This for example > includes verifying that docs / PDFs / etc render correctly, that > distribution packages contain the appropriate files, etc. Do your best > to evaluate these. > > > With that being said, the solution for the current situation is to > divide the queue among ourselves and work through it using the sprint > approach (but we need to be careful with stability: "commit storms" can > be detrimental). > > Tristan > > On 05/10/16 09:39, Radim Vansa wrote: >> I think that can be combined: sprint just explicitly prioritizes PRs >> instead of normal development, so you won't be reviewing PRs only when >> "you have a free spot" but "until you can't review anything else". So >> once the PR count hits a threshold, Tristan schedules sprint. >> >> R. >> >> On 10/05/2016 09:17 AM, Sebastian Laskawiec wrote: >>> That's an interesting idea... but after a month or two, the PR queue >>> will pile up again and we will need another sprint... >>> >>> IMO we should have a day-to-day strategy for doing this. >>> >>> On Wed, Oct 5, 2016 at 9:07 AM, Gustavo Fernandes >> > wrote: >>> >>> I propose doing with PRs the same we've been doing to areas that >>> require some care, the so called 'sprints'. >>> >>> We are in more than 10, a couple of PRs integrated each and we can >>> get >>> rid of most of the queue. >>> >>> Gustavo >>> >>> On Wed, Oct 5, 2016 at 7:36 AM, Sebastian Laskawiec >>> > wrote: >>> > Hey guys! >>> > >>> > I'm not sure what do you think about our PR queue but for me it >>> simply >>> > sucks... >>> > >>> > Some of our PRs are sitting there since I remember (like this >>> one [1] - Nov >>> > 2015!!! I guess we should prepare an anniversary cake, November >>> will is >>> > really soon :D) and some of them have more than 150 comments [2] >>> (maybe this >>> > sounds weird since it's my PR, but if something is not good >>> enough after 150 >>> > comments, maybe we should throw it away and try to implement it >>> again - >>> > differently?). >>> > >>> > After thinking about this for a little while, I would like to >>> propose one >>> > rule for reviewing PRs - when you have a free spot and you'd >>> like to >>> > integrate a couple of PRs - always start with the oldest and go >>> through all >>> > "Ready for review" PRs. >>> > >>> > I know that integrating some new stuff will be tempting but >>> please don't do >>> > that. Even if the new PR modifies only one small line.. noooo. >>> Go through >>> > some old stuff instead. This way we should revisit old PRs much >>> more >>> > frequently than new ones and this will force us to make some >>> tough decisions >>> > that we avoided for quite some time (e.g. whether or not we >>> should integrate >>> > [1] or [2] - but those are only examples). >>> > >>> > And finally, I think we should have more courage to integrate >>> PRs especially >>> > into master branch... After all, we have all machinery in place >>> - nightly CI >>> > jobs, stress tests, performance tests - what bad can happen? We >>> break the >>> > build? So what? We can always fix it by git revert. >>> > >>> > What do you think? Maybe you have better ideas? >>> > >>> > Thanks >>> > Seb >>> > >>> > [1] https://github.com/infinispan/infinispan/pull/3860 >>> >>> > [2] https://github.com/infinispan/infinispan/pull/4348 >>> >>> >>> >> >> > > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From slaskawi at redhat.com Wed Oct 5 05:37:49 2016 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Wed, 5 Oct 2016 11:37:49 +0200 Subject: [infinispan-dev] How to improve our PR queue In-Reply-To: References: <57F4AE2A.1050205@redhat.com> <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> Message-ID: I agree with both Tristan and Sanne - they key point here is to make reviewing PRs a habit. Doing it once a day (or even more often as Tristan suggested) is a good place to start in my opinion. The "rebase and merge" can speed the things up but it doesn't solve the main problem. We simply need to be more courageous when integrating PRs and avoid blocking project's progress. Thanks Sebastian On Wed, Oct 5, 2016 at 11:21 AM, Sanne Grinovero wrote: > Nice suggestions. I would be careful in actually *not* creating > dedicated "pr merge sprints" as during such a sprint one would have > more frequent merges, and so making the keep-up-rebasing and conflict > resolving even more painful ;) > > I think the only solution is - like Tristan suggests - to make it a > discipline to regularly check for PRs to be merged. > > Slightly unrelated, but it might help: did you notice that Github > recently evolved their PR merge button? > For trivial changes, i.e. if you're confident to not need to re-run > the testsuite after a rebase, you can now just press the green button > to have it rebase & merge, without creating a merge point. > > I just enabled the feature on the Infinispan repo ;) > > Sanne > > > On 5 October 2016 at 09:18, Tristan Tarrant wrote: > > [Moving to infinispan-dev] > > > > I agree with all of you. > > > > It is not respectful towards the developer who has opened the PR to let > > it linger for longer than necessary, and also for the developer who has > > opened it not to react on the comments (that happens too!) > > > > Let's set some ground rules for Pull Requests (this should go in the > > Contribution guide): > > > > Rules for everybody > > - Each and everyone of the core engineers looks at the PR queue at least > > once per day. More frequently if possible, especially if engaging in an > > active collaborative review. > > - We have more than one project: cachestores, console, integrations, > > quickstarts, clients, website... > > > > Rules for PR owners: > > - PRs *must* be kept "applicable" (no conflicts) > > - non-trivial PRs *must* have a corresponding Jira and the link to it > > *must* be present in the PR description. Viceversa, the Jira's workflow > > must be advanced to "Pull Request Sent" and include the link to the PR > > - if the PR is for multiple branches and it cherry-picks cleanly to all > > of them, no need to open multiple PRs. Use the "Backport" label to > > indicate this > > - if there is a need for separate PRs for different branches, clearly > > indicate the non-master branch it needs to be applied to in the PR title > > using the [branch] notation > > - Ensure the correct labels are applied when opening a PR or when the > > status changes > > - The owner *must* react to comments / CI failures / requests for rebase > > within 24 hours. Reaction means acknowledgement, not necessarily being > > able to fix the issues > > - If the owner cannot fix the issues related to comments / CI within a > > week, the PR should be closed and reopened when they can be addressed > > - While we still have some intermittent CI failures, we're in a > > situation where it is quite reliable. Exceptions obviously happen. > > > > Rules for PR reviewers > > - A PR should be acted upon (commented, merged) within 24 hours of its > > opening > > - A PR can be commented upon even if CI hasn't finished its job > > - PRs opened by external developers won't have the appropriate labels > > for permission reasons. Add them. > > - If the owner has addressed all comments / CI failures, the PR should > > be merged within a week > > - Some effects of a PR cannot be detected by CI. This for example > > includes verifying that docs / PDFs / etc render correctly, that > > distribution packages contain the appropriate files, etc. Do your best > > to evaluate these. > > > > > > With that being said, the solution for the current situation is to > > divide the queue among ourselves and work through it using the sprint > > approach (but we need to be careful with stability: "commit storms" can > > be detrimental). > > > > Tristan > > > > On 05/10/16 09:39, Radim Vansa wrote: > >> I think that can be combined: sprint just explicitly prioritizes PRs > >> instead of normal development, so you won't be reviewing PRs only when > >> "you have a free spot" but "until you can't review anything else". So > >> once the PR count hits a threshold, Tristan schedules sprint. > >> > >> R. > >> > >> On 10/05/2016 09:17 AM, Sebastian Laskawiec wrote: > >>> That's an interesting idea... but after a month or two, the PR queue > >>> will pile up again and we will need another sprint... > >>> > >>> IMO we should have a day-to-day strategy for doing this. > >>> > >>> On Wed, Oct 5, 2016 at 9:07 AM, Gustavo Fernandes >>> > wrote: > >>> > >>> I propose doing with PRs the same we've been doing to areas that > >>> require some care, the so called 'sprints'. > >>> > >>> We are in more than 10, a couple of PRs integrated each and we can > >>> get > >>> rid of most of the queue. > >>> > >>> Gustavo > >>> > >>> On Wed, Oct 5, 2016 at 7:36 AM, Sebastian Laskawiec > >>> > wrote: > >>> > Hey guys! > >>> > > >>> > I'm not sure what do you think about our PR queue but for me it > >>> simply > >>> > sucks... > >>> > > >>> > Some of our PRs are sitting there since I remember (like this > >>> one [1] - Nov > >>> > 2015!!! I guess we should prepare an anniversary cake, November > >>> will is > >>> > really soon :D) and some of them have more than 150 comments [2] > >>> (maybe this > >>> > sounds weird since it's my PR, but if something is not good > >>> enough after 150 > >>> > comments, maybe we should throw it away and try to implement it > >>> again - > >>> > differently?). > >>> > > >>> > After thinking about this for a little while, I would like to > >>> propose one > >>> > rule for reviewing PRs - when you have a free spot and you'd > >>> like to > >>> > integrate a couple of PRs - always start with the oldest and go > >>> through all > >>> > "Ready for review" PRs. > >>> > > >>> > I know that integrating some new stuff will be tempting but > >>> please don't do > >>> > that. Even if the new PR modifies only one small line.. noooo. > >>> Go through > >>> > some old stuff instead. This way we should revisit old PRs much > >>> more > >>> > frequently than new ones and this will force us to make some > >>> tough decisions > >>> > that we avoided for quite some time (e.g. whether or not we > >>> should integrate > >>> > [1] or [2] - but those are only examples). > >>> > > >>> > And finally, I think we should have more courage to integrate > >>> PRs especially > >>> > into master branch... After all, we have all machinery in place > >>> - nightly CI > >>> > jobs, stress tests, performance tests - what bad can happen? We > >>> break the > >>> > build? So what? We can always fix it by git revert. > >>> > > >>> > What do you think? Maybe you have better ideas? > >>> > > >>> > Thanks > >>> > Seb > >>> > > >>> > [1] https://github.com/infinispan/infinispan/pull/3860 > >>> > >>> > [2] https://github.com/infinispan/infinispan/pull/4348 > >>> > >>> > >>> > >> > >> > > > > -- > > Tristan Tarrant > > Infinispan Lead > > JBoss, a division of Red Hat > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161005/1466af13/attachment-0001.html From dan.berindei at gmail.com Wed Oct 5 05:51:55 2016 From: dan.berindei at gmail.com (Dan Berindei) Date: Wed, 5 Oct 2016 12:51:55 +0300 Subject: [infinispan-dev] How to improve our PR queue In-Reply-To: References: <57F4AE2A.1050205@redhat.com> <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> Message-ID: I would also suggest avoiding +1/LGTM comments. It's enough to have the author and the integrator responsible in case something goes wrong, we don't need to "spread the blame" by having multiple +1 comments. "LGTM, but" is fine if there's a specific area that you have doubts about, and you're assigning the PR to someone else. Otherwise, if it looks good, then we should integrate it without superfluous comments :) Cheers Dan On Wed, Oct 5, 2016 at 12:37 PM, Sebastian Laskawiec wrote: > I agree with both Tristan and Sanne - they key point here is to make > reviewing PRs a habit. Doing it once a day (or even more often as Tristan > suggested) is a good place to start in my opinion. > > The "rebase and merge" can speed the things up but it doesn't solve the main > problem. We simply need to be more courageous when integrating PRs and avoid > blocking project's progress. > > Thanks > Sebastian > > > > On Wed, Oct 5, 2016 at 11:21 AM, Sanne Grinovero > wrote: >> >> Nice suggestions. I would be careful in actually *not* creating >> dedicated "pr merge sprints" as during such a sprint one would have >> more frequent merges, and so making the keep-up-rebasing and conflict >> resolving even more painful ;) >> >> I think the only solution is - like Tristan suggests - to make it a >> discipline to regularly check for PRs to be merged. >> >> Slightly unrelated, but it might help: did you notice that Github >> recently evolved their PR merge button? >> For trivial changes, i.e. if you're confident to not need to re-run >> the testsuite after a rebase, you can now just press the green button >> to have it rebase & merge, without creating a merge point. >> >> I just enabled the feature on the Infinispan repo ;) >> >> Sanne >> >> >> On 5 October 2016 at 09:18, Tristan Tarrant wrote: >> > [Moving to infinispan-dev] >> > >> > I agree with all of you. >> > >> > It is not respectful towards the developer who has opened the PR to let >> > it linger for longer than necessary, and also for the developer who has >> > opened it not to react on the comments (that happens too!) >> > >> > Let's set some ground rules for Pull Requests (this should go in the >> > Contribution guide): >> > >> > Rules for everybody >> > - Each and everyone of the core engineers looks at the PR queue at least >> > once per day. More frequently if possible, especially if engaging in an >> > active collaborative review. >> > - We have more than one project: cachestores, console, integrations, >> > quickstarts, clients, website... >> > >> > Rules for PR owners: >> > - PRs *must* be kept "applicable" (no conflicts) >> > - non-trivial PRs *must* have a corresponding Jira and the link to it >> > *must* be present in the PR description. Viceversa, the Jira's workflow >> > must be advanced to "Pull Request Sent" and include the link to the PR >> > - if the PR is for multiple branches and it cherry-picks cleanly to all >> > of them, no need to open multiple PRs. Use the "Backport" label to >> > indicate this >> > - if there is a need for separate PRs for different branches, clearly >> > indicate the non-master branch it needs to be applied to in the PR title >> > using the [branch] notation >> > - Ensure the correct labels are applied when opening a PR or when the >> > status changes >> > - The owner *must* react to comments / CI failures / requests for rebase >> > within 24 hours. Reaction means acknowledgement, not necessarily being >> > able to fix the issues >> > - If the owner cannot fix the issues related to comments / CI within a >> > week, the PR should be closed and reopened when they can be addressed >> > - While we still have some intermittent CI failures, we're in a >> > situation where it is quite reliable. Exceptions obviously happen. >> > >> > Rules for PR reviewers >> > - A PR should be acted upon (commented, merged) within 24 hours of its >> > opening >> > - A PR can be commented upon even if CI hasn't finished its job >> > - PRs opened by external developers won't have the appropriate labels >> > for permission reasons. Add them. >> > - If the owner has addressed all comments / CI failures, the PR should >> > be merged within a week >> > - Some effects of a PR cannot be detected by CI. This for example >> > includes verifying that docs / PDFs / etc render correctly, that >> > distribution packages contain the appropriate files, etc. Do your best >> > to evaluate these. >> > >> > >> > With that being said, the solution for the current situation is to >> > divide the queue among ourselves and work through it using the sprint >> > approach (but we need to be careful with stability: "commit storms" can >> > be detrimental). >> > >> > Tristan >> > >> > On 05/10/16 09:39, Radim Vansa wrote: >> >> I think that can be combined: sprint just explicitly prioritizes PRs >> >> instead of normal development, so you won't be reviewing PRs only when >> >> "you have a free spot" but "until you can't review anything else". So >> >> once the PR count hits a threshold, Tristan schedules sprint. >> >> >> >> R. >> >> >> >> On 10/05/2016 09:17 AM, Sebastian Laskawiec wrote: >> >>> That's an interesting idea... but after a month or two, the PR queue >> >>> will pile up again and we will need another sprint... >> >>> >> >>> IMO we should have a day-to-day strategy for doing this. >> >>> >> >>> On Wed, Oct 5, 2016 at 9:07 AM, Gustavo Fernandes > >>> > wrote: >> >>> >> >>> I propose doing with PRs the same we've been doing to areas that >> >>> require some care, the so called 'sprints'. >> >>> >> >>> We are in more than 10, a couple of PRs integrated each and we can >> >>> get >> >>> rid of most of the queue. >> >>> >> >>> Gustavo >> >>> >> >>> On Wed, Oct 5, 2016 at 7:36 AM, Sebastian Laskawiec >> >>> > wrote: >> >>> > Hey guys! >> >>> > >> >>> > I'm not sure what do you think about our PR queue but for me it >> >>> simply >> >>> > sucks... >> >>> > >> >>> > Some of our PRs are sitting there since I remember (like this >> >>> one [1] - Nov >> >>> > 2015!!! I guess we should prepare an anniversary cake, November >> >>> will is >> >>> > really soon :D) and some of them have more than 150 comments [2] >> >>> (maybe this >> >>> > sounds weird since it's my PR, but if something is not good >> >>> enough after 150 >> >>> > comments, maybe we should throw it away and try to implement it >> >>> again - >> >>> > differently?). >> >>> > >> >>> > After thinking about this for a little while, I would like to >> >>> propose one >> >>> > rule for reviewing PRs - when you have a free spot and you'd >> >>> like to >> >>> > integrate a couple of PRs - always start with the oldest and go >> >>> through all >> >>> > "Ready for review" PRs. >> >>> > >> >>> > I know that integrating some new stuff will be tempting but >> >>> please don't do >> >>> > that. Even if the new PR modifies only one small line.. noooo. >> >>> Go through >> >>> > some old stuff instead. This way we should revisit old PRs much >> >>> more >> >>> > frequently than new ones and this will force us to make some >> >>> tough decisions >> >>> > that we avoided for quite some time (e.g. whether or not we >> >>> should integrate >> >>> > [1] or [2] - but those are only examples). >> >>> > >> >>> > And finally, I think we should have more courage to integrate >> >>> PRs especially >> >>> > into master branch... After all, we have all machinery in place >> >>> - nightly CI >> >>> > jobs, stress tests, performance tests - what bad can happen? We >> >>> break the >> >>> > build? So what? We can always fix it by git revert. >> >>> > >> >>> > What do you think? Maybe you have better ideas? >> >>> > >> >>> > Thanks >> >>> > Seb >> >>> > >> >>> > [1] https://github.com/infinispan/infinispan/pull/3860 >> >>> >> >>> > [2] https://github.com/infinispan/infinispan/pull/4348 >> >>> >> >>> >> >>> >> >> >> >> >> > >> > -- >> > Tristan Tarrant >> > Infinispan Lead >> > JBoss, a division of Red Hat >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From vjuranek at redhat.com Wed Oct 5 06:26:08 2016 From: vjuranek at redhat.com (Vojtech Juranek) Date: Wed, 05 Oct 2016 12:26:08 +0200 Subject: [infinispan-dev] How to improve our PR queue In-Reply-To: <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> References: <57F4AE2A.1050205@redhat.com> <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> Message-ID: <22361307.ovFz8yV2Zg@localhost.localdomain> On Wednesday 05 October 2016 10:18:31 Tristan Tarrant wrote: > - PRs opened by external developers won't have the appropriate labels > for permission reasons. Add them. would be nice if the labels can be reset automatically once PR is updated. Labels are usually outdated and misleading after PR update and reviewer has to check the comments under PR (in better case, in worse the PR is ignored as it has "Changes required" label or similar) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 473 bytes Desc: This is a digitally signed message part. Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161005/cf53937e/attachment.bin From slaskawi at redhat.com Wed Oct 5 06:35:02 2016 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Wed, 5 Oct 2016 12:35:02 +0200 Subject: [infinispan-dev] How to improve our PR queue In-Reply-To: References: <57F4AE2A.1050205@redhat.com> <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> Message-ID: +1 - for the last time :D Originally I used +1/LGTM to let others look into the PR before integrating it. But I agree - it was impractical. Let's integrate stuff instead... On Wed, Oct 5, 2016 at 11:51 AM, Dan Berindei wrote: > I would also suggest avoiding +1/LGTM comments. It's enough to have > the author and the integrator responsible in case something goes > wrong, we don't need to "spread the blame" by having multiple +1 > comments. > > "LGTM, but" is fine if there's a specific area that you have doubts > about, and you're assigning the PR to someone else. Otherwise, if it > looks good, then we should integrate it without superfluous comments > :) > > Cheers > Dan > > > > On Wed, Oct 5, 2016 at 12:37 PM, Sebastian Laskawiec > wrote: > > I agree with both Tristan and Sanne - they key point here is to make > > reviewing PRs a habit. Doing it once a day (or even more often as Tristan > > suggested) is a good place to start in my opinion. > > > > The "rebase and merge" can speed the things up but it doesn't solve the > main > > problem. We simply need to be more courageous when integrating PRs and > avoid > > blocking project's progress. > > > > Thanks > > Sebastian > > > > > > > > On Wed, Oct 5, 2016 at 11:21 AM, Sanne Grinovero > > wrote: > >> > >> Nice suggestions. I would be careful in actually *not* creating > >> dedicated "pr merge sprints" as during such a sprint one would have > >> more frequent merges, and so making the keep-up-rebasing and conflict > >> resolving even more painful ;) > >> > >> I think the only solution is - like Tristan suggests - to make it a > >> discipline to regularly check for PRs to be merged. > >> > >> Slightly unrelated, but it might help: did you notice that Github > >> recently evolved their PR merge button? > >> For trivial changes, i.e. if you're confident to not need to re-run > >> the testsuite after a rebase, you can now just press the green button > >> to have it rebase & merge, without creating a merge point. > >> > >> I just enabled the feature on the Infinispan repo ;) > >> > >> Sanne > >> > >> > >> On 5 October 2016 at 09:18, Tristan Tarrant > wrote: > >> > [Moving to infinispan-dev] > >> > > >> > I agree with all of you. > >> > > >> > It is not respectful towards the developer who has opened the PR to > let > >> > it linger for longer than necessary, and also for the developer who > has > >> > opened it not to react on the comments (that happens too!) > >> > > >> > Let's set some ground rules for Pull Requests (this should go in the > >> > Contribution guide): > >> > > >> > Rules for everybody > >> > - Each and everyone of the core engineers looks at the PR queue at > least > >> > once per day. More frequently if possible, especially if engaging in > an > >> > active collaborative review. > >> > - We have more than one project: cachestores, console, integrations, > >> > quickstarts, clients, website... > >> > > >> > Rules for PR owners: > >> > - PRs *must* be kept "applicable" (no conflicts) > >> > - non-trivial PRs *must* have a corresponding Jira and the link to it > >> > *must* be present in the PR description. Viceversa, the Jira's > workflow > >> > must be advanced to "Pull Request Sent" and include the link to the PR > >> > - if the PR is for multiple branches and it cherry-picks cleanly to > all > >> > of them, no need to open multiple PRs. Use the "Backport" label to > >> > indicate this > >> > - if there is a need for separate PRs for different branches, clearly > >> > indicate the non-master branch it needs to be applied to in the PR > title > >> > using the [branch] notation > >> > - Ensure the correct labels are applied when opening a PR or when the > >> > status changes > >> > - The owner *must* react to comments / CI failures / requests for > rebase > >> > within 24 hours. Reaction means acknowledgement, not necessarily being > >> > able to fix the issues > >> > - If the owner cannot fix the issues related to comments / CI within a > >> > week, the PR should be closed and reopened when they can be addressed > >> > - While we still have some intermittent CI failures, we're in a > >> > situation where it is quite reliable. Exceptions obviously happen. > >> > > >> > Rules for PR reviewers > >> > - A PR should be acted upon (commented, merged) within 24 hours of its > >> > opening > >> > - A PR can be commented upon even if CI hasn't finished its job > >> > - PRs opened by external developers won't have the appropriate labels > >> > for permission reasons. Add them. > >> > - If the owner has addressed all comments / CI failures, the PR should > >> > be merged within a week > >> > - Some effects of a PR cannot be detected by CI. This for example > >> > includes verifying that docs / PDFs / etc render correctly, that > >> > distribution packages contain the appropriate files, etc. Do your best > >> > to evaluate these. > >> > > >> > > >> > With that being said, the solution for the current situation is to > >> > divide the queue among ourselves and work through it using the sprint > >> > approach (but we need to be careful with stability: "commit storms" > can > >> > be detrimental). > >> > > >> > Tristan > >> > > >> > On 05/10/16 09:39, Radim Vansa wrote: > >> >> I think that can be combined: sprint just explicitly prioritizes PRs > >> >> instead of normal development, so you won't be reviewing PRs only > when > >> >> "you have a free spot" but "until you can't review anything else". So > >> >> once the PR count hits a threshold, Tristan schedules sprint. > >> >> > >> >> R. > >> >> > >> >> On 10/05/2016 09:17 AM, Sebastian Laskawiec wrote: > >> >>> That's an interesting idea... but after a month or two, the PR queue > >> >>> will pile up again and we will need another sprint... > >> >>> > >> >>> IMO we should have a day-to-day strategy for doing this. > >> >>> > >> >>> On Wed, Oct 5, 2016 at 9:07 AM, Gustavo Fernandes < > gfernand at redhat.com > >> >>> > wrote: > >> >>> > >> >>> I propose doing with PRs the same we've been doing to areas that > >> >>> require some care, the so called 'sprints'. > >> >>> > >> >>> We are in more than 10, a couple of PRs integrated each and we > can > >> >>> get > >> >>> rid of most of the queue. > >> >>> > >> >>> Gustavo > >> >>> > >> >>> On Wed, Oct 5, 2016 at 7:36 AM, Sebastian Laskawiec > >> >>> > wrote: > >> >>> > Hey guys! > >> >>> > > >> >>> > I'm not sure what do you think about our PR queue but for me > it > >> >>> simply > >> >>> > sucks... > >> >>> > > >> >>> > Some of our PRs are sitting there since I remember (like this > >> >>> one [1] - Nov > >> >>> > 2015!!! I guess we should prepare an anniversary cake, > November > >> >>> will is > >> >>> > really soon :D) and some of them have more than 150 comments > [2] > >> >>> (maybe this > >> >>> > sounds weird since it's my PR, but if something is not good > >> >>> enough after 150 > >> >>> > comments, maybe we should throw it away and try to implement > it > >> >>> again - > >> >>> > differently?). > >> >>> > > >> >>> > After thinking about this for a little while, I would like to > >> >>> propose one > >> >>> > rule for reviewing PRs - when you have a free spot and you'd > >> >>> like to > >> >>> > integrate a couple of PRs - always start with the oldest and > go > >> >>> through all > >> >>> > "Ready for review" PRs. > >> >>> > > >> >>> > I know that integrating some new stuff will be tempting but > >> >>> please don't do > >> >>> > that. Even if the new PR modifies only one small line.. noooo. > >> >>> Go through > >> >>> > some old stuff instead. This way we should revisit old PRs > much > >> >>> more > >> >>> > frequently than new ones and this will force us to make some > >> >>> tough decisions > >> >>> > that we avoided for quite some time (e.g. whether or not we > >> >>> should integrate > >> >>> > [1] or [2] - but those are only examples). > >> >>> > > >> >>> > And finally, I think we should have more courage to integrate > >> >>> PRs especially > >> >>> > into master branch... After all, we have all machinery in > place > >> >>> - nightly CI > >> >>> > jobs, stress tests, performance tests - what bad can happen? > We > >> >>> break the > >> >>> > build? So what? We can always fix it by git revert. > >> >>> > > >> >>> > What do you think? Maybe you have better ideas? > >> >>> > > >> >>> > Thanks > >> >>> > Seb > >> >>> > > >> >>> > [1] https://github.com/infinispan/infinispan/pull/3860 > >> >>> > >> >>> > [2] https://github.com/infinispan/infinispan/pull/4348 > >> >>> > >> >>> > >> >>> > >> >> > >> >> > >> > > >> > -- > >> > Tristan Tarrant > >> > Infinispan Lead > >> > JBoss, a division of Red Hat > >> > _______________________________________________ > >> > infinispan-dev mailing list > >> > infinispan-dev at lists.jboss.org > >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161005/f430204f/attachment-0001.html From rvansa at redhat.com Wed Oct 5 07:41:37 2016 From: rvansa at redhat.com (Radim Vansa) Date: Wed, 5 Oct 2016 13:41:37 +0200 Subject: [infinispan-dev] How to improve our PR queue In-Reply-To: <22361307.ovFz8yV2Zg@localhost.localdomain> References: <57F4AE2A.1050205@redhat.com> <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> <22361307.ovFz8yV2Zg@localhost.localdomain> Message-ID: <57F4E6F1.8000202@redhat.com> On 10/05/2016 12:26 PM, Vojtech Juranek wrote: > On Wednesday 05 October 2016 10:18:31 Tristan Tarrant wrote: >> - PRs opened by external developers won't have the appropriate labels >> for permission reasons. Add them. > > would be nice if the labels can be reset automatically once PR is updated. > Labels are usually outdated and misleading after PR update and reviewer has to > check the comments under PR (in better case, in worse the PR is ignored as it > has "Changes required" label or similar) This is only a problem for PR authors who don't have write-permissions on the PR to update it. A workaround is to ask on IRC to update the labels; proper solution is a webhook (probably to a script on infinispan.org) that will change the labels accordingly. Radim > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From ttarrant at redhat.com Fri Oct 7 02:14:47 2016 From: ttarrant at redhat.com (Tristan Tarrant) Date: Fri, 7 Oct 2016 08:14:47 +0200 Subject: [infinispan-dev] How to improve our PR queue In-Reply-To: References: <57F4AE2A.1050205@redhat.com> <3a816b7b-502f-4d60-0b5d-6c5b0a31f5ac@redhat.com> Message-ID: On 05/10/16 11:51, Dan Berindei wrote: > I would also suggest avoiding +1/LGTM comments. It's enough to have > the author and the integrator responsible in case something goes > wrong, we don't need to "spread the blame" by having multiple +1 > comments. The only place I like using the thumbs up is when I'm the author and I'm agreeing with the reviewers comment. It has the useful side effect of keeping track of all the reviews thus limiting the chance I might miss something Tristan From rory.odonnell at oracle.com Fri Oct 7 06:44:47 2016 From: rory.odonnell at oracle.com (Rory O'Donnell) Date: Fri, 7 Oct 2016 11:44:47 +0100 Subject: [infinispan-dev] Early Access builds of JDK 8u122 b01 & JDK 9 EA b138 are available. Message-ID: Hi Galder, Early Access b01 for JDK 8u122 is available on java.net. Early Access b138 for JDK 9 is available on java.net, summary of changes are listed here . Early Access b138 (#5561) for JDK 9 with Project Jigsaw is available on java.net, summary of changes are listed here . There have been a number of fixes to bugs reported by Open Source projects since the last availability email : * 8038348 - b137 - Instance field load is replaced by wrong data Phi * 8041480 - b137 - ArrayIndexOutOfBoundsException when JTable contains certain string * 8145542 - b137 - The case failed automatically and thrown java.lang.ArrayIndexOutOfBoundsException exception. * 8151725 - b137 - [macosx] ArrayIndexOOB exception when displaying Devanagari text in JEditorPane There are two proposals requesting feedback via mailing lists : - More proposals for open JPMS issues [1] * One proposal to call out is #AwkwardStrongEncapsulation as this proposes to change setAccessible(true) so that it can't be used to break into non-public types/members in exported packages. This is a non-issue when using setAccessible to get access to non-public types/members of classes on the class path but will be an issue for code that uses this method to break into JDK-internals. We would appreciate as much help as possible testing these builds. If InaccessibleObjectException is thrown then examine the stack trace (run with -Dsun.reflect.debugModuleAccessChecks=true to uncover failed access attempts in code that shallows exceptions). If it looks like a library is attempting to access a non-public method or field of a platform class and make sure to *submit a bug in the issue tracker for the library.* - Proposal to Reorganize source classes in src.zip by modules [2] * This proposal might have a compatibility impact on IDEs or other tools that look in src.zip * Feedback via the jigsaw-dev mailing list Other points of interest: * JavaOne Presentations o Recorded Presentations * JDK 9 Schedule change proposal o for proposed schedule proposal see [3] Rgds,Rory [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-September/009365.html [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-October/009550.html [3] http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-September/004887.html -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161007/5fd84990/attachment.html From slaskawi at redhat.com Mon Oct 10 04:34:51 2016 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 10 Oct 2016 10:34:51 +0200 Subject: [infinispan-dev] Spring Boot Starters for Infinispan Message-ID: Hey Guys! Thanks to a lot of good stuff from both Tomek Zab?ocki [1] and Marco Yuen [2] we introduced Spring Boot Starters for Infinispan. The main repository for starters is located here [3] and there's also a preview version of the tutorial [4]. At the end of this week I plan to merge the last portion of PRs [4][5] and release it. In the meantime please have a look into it and play with it. Once everything is released, we can propose a Pull Request for Spring INITIALIZR [6] and make Infinispan even easier to use! Again, thanks a lot for great contributions guys! Sebastian [1] https://github.com/blocha [2] https://github.com/marcoy [3] https://github.com/infinispan/infinispan-spring-boot [4] https://github.com/infinispan/infinispan-simple-tutorials/pull/17 [5] https://github.com/infinispan/infinispan-spring-boot/pull/2 [6] http://start.spring.io/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161010/31c6e1d9/attachment.html From galder at redhat.com Mon Oct 10 11:49:13 2016 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Mon, 10 Oct 2016 17:49:13 +0200 Subject: [infinispan-dev] Func API in tx cache In-Reply-To: <57EA876D.6060300@redhat.com> References: <57EA876D.6060300@redhat.com> Message-ID: <3AD09175-A621-41B0-B4F4-75C697FF57E7@redhat.com> Hey Radim, Sorry for late reply. Indeed, I think you should start by getting RC done first. When I first experimented with this, I had bigger issues than the ones explained here. For example, how to pass down transaction context when multiple async operations are being executed within a transaction. In particular, say you have two async operations, chained, executed with T1 and T2 respectively, both part of the the same transaction. Suspending tx in T1 and resuming it when T2 starts is easy, the problem I had was how to get T1 to resume the transaction when T2 has completed async transactional operation. At the time I created a forum post in the transactions forum about it [1]. I was not able to able to try out the solution in [1], but it doesn't feel very idiomatic to me (e.g. needing to call CompletableFuture.get()). Did you consider this issue? Or are you avoiding it somehow? Cheers, [1] https://developer.jboss.org/thread/265595 -- Galder Zamarre?o Infinispan, Red Hat > On 27 Sep 2016, at 16:51, Radim Vansa wrote: > > Hi, > > seems I'll have to implement the functional stuff on tx caches [1][2] if > I want to get rid of DeltaAware et al. > > The general idea is quite simple - ReadOnly* commands should behave very > similar to non-tx mode, WriteOnly* commands will be just added as > modifications to the PrepareCommand and ReadWrite* commands will be both > added to modifications list, and sent to remote nodes where the result > won't be stored yet. > The results of operations should not be stored into transactional > context - the command will execute remotely (if the owners are remote) > unless the value was read by Get* beforehand. > > With repeatable-reads isolation, the situation gets more complicated. If > we use ReadOnly* that performs identity lookup (effectively the same as > Get*) and the entry was modified in during the transaction, we can > return two different results - so a read committed semantics. With write > skew check enabled, we could at least fail the transaction at the end > (the check would be performed reads as well if the transaction contains > functional reads), but we cannot rely on WSC always on with RR. > > Retrieving the whole entry and applying the functional command is not a > viable solution, IMO - that would completely defy the purpose of using > functional command. > > A possible solution would be to send the global transaction ID with > those read commands and keep a remote transactional context with read > entries for the duration of transaction on remote nodes, too. However, > if we do a Read* command to primary owner, it's possible that further > Get* command will hit backup. So, we could go to all owners with Read* > already during the transaction (slowing down functional reads > considerably), or read only from primary owner (which slows down Get*s > even if we don't use functional APIs - this makes it a no-go). I am not > 100% sure how a transaction transfer during ST will get into that. > > We could also do it the ostrich way - "Yes we've promissed RR but Func > will be only RC". I'll probably do that in the first draft anyway. > > Comments & opinions appreciated. > > Radim > > [1] https://issues.jboss.org/browse/ISPN-5806 > [2] https://issues.jboss.org/browse/ISPN-6573 > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From rvansa at redhat.com Tue Oct 11 13:16:34 2016 From: rvansa at redhat.com (Radim Vansa) Date: Tue, 11 Oct 2016 19:16:34 +0200 Subject: [infinispan-dev] Func API in tx cache In-Reply-To: <3AD09175-A621-41B0-B4F4-75C697FF57E7@redhat.com> References: <57EA876D.6060300@redhat.com> <3AD09175-A621-41B0-B4F4-75C697FF57E7@redhat.com> Message-ID: <57FD1E72.2080201@redhat.com> Hi Galder, I've looked to the Narayana thread but I also have some trouble understanding the R ret = CompletableFuture.supplyAsync(...) - that method returns CF, not directly R... so the answer that guys on the forum give you makes sense to me. Maybe you could revive that thread and clarify what you had in mind. You're describing general issue of async calls, not just functional API. Functional API is about moving some lambda to be executed in situ. Your question is no different as if you were calling putAsync and trying to chain it with another operations. TBH I haven't dealt with this problem at all, the tests I've written use the operations in sync way. If you try to do something transactionally-sensitive as a part of the lambda itself, you're doing it wrong - the lambda can be executed on different node which has no link to the original transaction. And the lambda should be purely functional (no side effects), otherwise you're poking the devil. Regarding RC vs. RR., in [1] I've decided to attach version on which the functional command operated with the return value if versions (and WSC) is used. Then, if the version changes during the transaction (say, you've did a functional read, and it changes before second read), WriteSkewException is thrown as we're not operating on the same value. However, this does not work after you modify the entry with a functional command; then your reads still work on the non-modified entry... I have to tweak this a bit yet, basically sending all the modifications to a given entry along with the read :-/ Note: RR without WSC is just broken, I should add a warning log somewhere... Radim [1] https://github.com/infinispan/infinispan/pull/4608 On 10/10/2016 05:49 PM, Galder Zamarre?o wrote: > Hey Radim, > > Sorry for late reply. Indeed, I think you should start by getting RC done first. > > When I first experimented with this, I had bigger issues than the ones explained here. For example, how to pass down transaction context when multiple async operations are being executed within a transaction. > > In particular, say you have two async operations, chained, executed with T1 and T2 respectively, both part of the the same transaction. Suspending tx in T1 and resuming it when T2 starts is easy, the problem I had was how to get T1 to resume the transaction when T2 has completed async transactional operation. At the time I created a forum post in the transactions forum about it [1]. I was not able to able to try out the solution in [1], but it doesn't feel very idiomatic to me (e.g. needing to call CompletableFuture.get()). > > Did you consider this issue? Or are you avoiding it somehow? > > Cheers, > > [1] https://developer.jboss.org/thread/265595 > -- > Galder Zamarre?o > Infinispan, Red Hat > >> On 27 Sep 2016, at 16:51, Radim Vansa wrote: >> >> Hi, >> >> seems I'll have to implement the functional stuff on tx caches [1][2] if >> I want to get rid of DeltaAware et al. >> >> The general idea is quite simple - ReadOnly* commands should behave very >> similar to non-tx mode, WriteOnly* commands will be just added as >> modifications to the PrepareCommand and ReadWrite* commands will be both >> added to modifications list, and sent to remote nodes where the result >> won't be stored yet. >> The results of operations should not be stored into transactional >> context - the command will execute remotely (if the owners are remote) >> unless the value was read by Get* beforehand. >> >> With repeatable-reads isolation, the situation gets more complicated. If >> we use ReadOnly* that performs identity lookup (effectively the same as >> Get*) and the entry was modified in during the transaction, we can >> return two different results - so a read committed semantics. With write >> skew check enabled, we could at least fail the transaction at the end >> (the check would be performed reads as well if the transaction contains >> functional reads), but we cannot rely on WSC always on with RR. >> >> Retrieving the whole entry and applying the functional command is not a >> viable solution, IMO - that would completely defy the purpose of using >> functional command. >> >> A possible solution would be to send the global transaction ID with >> those read commands and keep a remote transactional context with read >> entries for the duration of transaction on remote nodes, too. However, >> if we do a Read* command to primary owner, it's possible that further >> Get* command will hit backup. So, we could go to all owners with Read* >> already during the transaction (slowing down functional reads >> considerably), or read only from primary owner (which slows down Get*s >> even if we don't use functional APIs - this makes it a no-go). I am not >> 100% sure how a transaction transfer during ST will get into that. >> >> We could also do it the ostrich way - "Yes we've promissed RR but Func >> will be only RC". I'll probably do that in the first draft anyway. >> >> Comments & opinions appreciated. >> >> Radim >> >> [1] https://issues.jboss.org/browse/ISPN-5806 >> [2] https://issues.jboss.org/browse/ISPN-6573 >> >> -- >> Radim Vansa >> JBoss Performance Team >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From ttarrant at redhat.com Mon Oct 17 03:07:43 2016 From: ttarrant at redhat.com (Tristan Tarrant) Date: Mon, 17 Oct 2016 09:07:43 +0200 Subject: [infinispan-dev] Names, names, names... Message-ID: Hi all, something trivial and fun for a Monday morning. I've just issued a PR [1] to update the codename for Infinispan 9.0. And while we're at it, let's give a name to the new query language that Adrian (and Emmanuel) have designed. We already have a number of suggestions (which I summarize below) but please feel free to add your own. Please vote. IQL (Infinispan Query Language, already used by others). Ickle (Alternate pronunciation of above, also means "small") LQID (Language for Querying Infinispan Datagrids) QuIL ("Query Infinispan" Language) Tristan [1] https://github.com/infinispan/infinispan/pull/4617 -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From rvansa at redhat.com Mon Oct 17 03:35:38 2016 From: rvansa at redhat.com (Radim Vansa) Date: Mon, 17 Oct 2016 09:35:38 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: Message-ID: <58047F4A.9090700@redhat.com> On 10/17/2016 09:07 AM, Tristan Tarrant wrote: > Hi all, > > something trivial and fun for a Monday morning. > > I've just issued a PR [1] to update the codename for Infinispan 9.0. Checked out ISPN-1414 [1], status is Open (though it's most likely out of date). [1] https://issues.jboss.org/browse/ISPN-1414 > > And while we're at it, let's give a name to the new query language that > Adrian (and Emmanuel) have designed. We already have a number of > suggestions (which I summarize below) but please feel free to add your > own. Please vote. > > IQL (Infinispan Query Language, already used by others). > Ickle (Alternate pronunciation of above, also means "small") > LQID (Language for Querying Infinispan Datagrids) > QuIL ("Query Infinispan" Language) I like LQID with the pronounciation "liquid", but then we need to keep some fluent API at least for Java clients :) R. > > Tristan > > [1] https://github.com/infinispan/infinispan/pull/4617 -- Radim Vansa JBoss Performance Team From sanne at infinispan.org Mon Oct 17 08:16:02 2016 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 17 Oct 2016 14:16:02 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: <58047F4A.9090700@redhat.com> References: <58047F4A.9090700@redhat.com> Message-ID: On 17 October 2016 at 09:35, Radim Vansa wrote: > On 10/17/2016 09:07 AM, Tristan Tarrant wrote: >> Hi all, >> >> something trivial and fun for a Monday morning. >> >> I've just issued a PR [1] to update the codename for Infinispan 9.0. > > Checked out ISPN-1414 [1], status is Open (though it's most likely out > of date). > > [1] https://issues.jboss.org/browse/ISPN-1414 > >> >> And while we're at it, let's give a name to the new query language that >> Adrian (and Emmanuel) have designed. We already have a number of >> suggestions (which I summarize below) but please feel free to add your >> own. Please vote. >> >> IQL (Infinispan Query Language, already used by others). >> Ickle (Alternate pronunciation of above, also means "small") >> LQID (Language for Querying Infinispan Datagrids) >> QuIL ("Query Infinispan" Language) > > I like LQID with the pronounciation "liquid", but then we need to keep > some fluent API at least for Java clients :) I'd prefer a liquid API ;) About the names: I like both IQL and LQID Or may I propose another candidate: "INAQL" which stands for all of: - Inaql is Not Another Query Language - INfinispan Advanced Query Language - Infinispan Needs A Query Language > > R. > >> >> Tristan >> >> [1] https://github.com/infinispan/infinispan/pull/4617 > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From dan.berindei at gmail.com Mon Oct 17 09:19:20 2016 From: dan.berindei at gmail.com (Dan Berindei) Date: Mon, 17 Oct 2016 15:19:20 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: On Mon, Oct 17, 2016 at 2:16 PM, Sanne Grinovero wrote: > On 17 October 2016 at 09:35, Radim Vansa wrote: >> On 10/17/2016 09:07 AM, Tristan Tarrant wrote: >>> Hi all, >>> >>> something trivial and fun for a Monday morning. >>> >>> I've just issued a PR [1] to update the codename for Infinispan 9.0. >> >> Checked out ISPN-1414 [1], status is Open (though it's most likely out >> of date). >> >> [1] https://issues.jboss.org/browse/ISPN-1414 >> >>> >>> And while we're at it, let's give a name to the new query language that >>> Adrian (and Emmanuel) have designed. We already have a number of >>> suggestions (which I summarize below) but please feel free to add your >>> own. Please vote. >>> >>> IQL (Infinispan Query Language, already used by others). >>> Ickle (Alternate pronunciation of above, also means "small") >>> LQID (Language for Querying Infinispan Datagrids) >>> QuIL ("Query Infinispan" Language) >> >> I like LQID with the pronounciation "liquid", but then we need to keep >> some fluent API at least for Java clients :) > > I'd prefer a liquid API ;) > > About the names: I like both IQL and LQID > > Or may I propose another candidate: "INAQL" > which stands for all of: > - Inaql is Not Another Query Language > - INfinispan Advanced Query Language > - Infinispan Needs A Query Language > I like that last one, but I don't think the article should be part of the acronym, so I'd go with INQL :) Dan From sanne at infinispan.org Mon Oct 17 10:05:35 2016 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 17 Oct 2016 16:05:35 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: On 17 October 2016 at 15:19, Dan Berindei wrote: > On Mon, Oct 17, 2016 at 2:16 PM, Sanne Grinovero wrote: >> On 17 October 2016 at 09:35, Radim Vansa wrote: >>> On 10/17/2016 09:07 AM, Tristan Tarrant wrote: >>>> Hi all, >>>> >>>> something trivial and fun for a Monday morning. >>>> >>>> I've just issued a PR [1] to update the codename for Infinispan 9.0. >>> >>> Checked out ISPN-1414 [1], status is Open (though it's most likely out >>> of date). >>> >>> [1] https://issues.jboss.org/browse/ISPN-1414 >>> >>>> >>>> And while we're at it, let's give a name to the new query language that >>>> Adrian (and Emmanuel) have designed. We already have a number of >>>> suggestions (which I summarize below) but please feel free to add your >>>> own. Please vote. >>>> >>>> IQL (Infinispan Query Language, already used by others). >>>> Ickle (Alternate pronunciation of above, also means "small") >>>> LQID (Language for Querying Infinispan Datagrids) >>>> QuIL ("Query Infinispan" Language) >>> >>> I like LQID with the pronounciation "liquid", but then we need to keep >>> some fluent API at least for Java clients :) >> >> I'd prefer a liquid API ;) >> >> About the names: I like both IQL and LQID >> >> Or may I propose another candidate: "INAQL" >> which stands for all of: >> - Inaql is Not Another Query Language >> - INfinispan Advanced Query Language >> - Infinispan Needs A Query Language >> > > I like that last one, but I don't think the article should be part of > the acronym, so I'd go with INQL :) +1 even better indeed > > Dan > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From ugo.landini at gmail.com Mon Oct 17 11:30:51 2016 From: ugo.landini at gmail.com (ugol) Date: Mon, 17 Oct 2016 17:30:51 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: inql sounds weird in Italian :) On Mon, Oct 17, 2016 at 4:05 PM, Sanne Grinovero wrote: > On 17 October 2016 at 15:19, Dan Berindei wrote: >> On Mon, Oct 17, 2016 at 2:16 PM, Sanne Grinovero wrote: >>> On 17 October 2016 at 09:35, Radim Vansa wrote: >>>> On 10/17/2016 09:07 AM, Tristan Tarrant wrote: >>>>> Hi all, >>>>> >>>>> something trivial and fun for a Monday morning. >>>>> >>>>> I've just issued a PR [1] to update the codename for Infinispan 9.0. >>>> >>>> Checked out ISPN-1414 [1], status is Open (though it's most likely out >>>> of date). >>>> >>>> [1] https://issues.jboss.org/browse/ISPN-1414 >>>> >>>>> >>>>> And while we're at it, let's give a name to the new query language that >>>>> Adrian (and Emmanuel) have designed. We already have a number of >>>>> suggestions (which I summarize below) but please feel free to add your >>>>> own. Please vote. >>>>> >>>>> IQL (Infinispan Query Language, already used by others). >>>>> Ickle (Alternate pronunciation of above, also means "small") >>>>> LQID (Language for Querying Infinispan Datagrids) >>>>> QuIL ("Query Infinispan" Language) >>>> >>>> I like LQID with the pronounciation "liquid", but then we need to keep >>>> some fluent API at least for Java clients :) >>> >>> I'd prefer a liquid API ;) >>> >>> About the names: I like both IQL and LQID >>> >>> Or may I propose another candidate: "INAQL" >>> which stands for all of: >>> - Inaql is Not Another Query Language >>> - INfinispan Advanced Query Language >>> - Infinispan Needs A Query Language >>> >> >> I like that last one, but I don't think the article should be part of >> the acronym, so I'd go with INQL :) > > +1 even better indeed > >> >> Dan >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- uL Pragmatist http://blog.ugolandini.com http://www.flickr.com/photos/ugol/ From remerson at redhat.com Mon Oct 17 11:37:15 2016 From: remerson at redhat.com (Ryan Emerson) Date: Mon, 17 Oct 2016 11:37:15 -0400 (EDT) Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: <1406943215.7811464.1476718635050.JavaMail.zimbra@redhat.com> +1 for INQL ----- Original Message ----- From: "Dan Berindei" To: "infinispan -Dev List" Sent: Monday, 17 October, 2016 2:19:20 PM Subject: Re: [infinispan-dev] Names, names, names... On Mon, Oct 17, 2016 at 2:16 PM, Sanne Grinovero wrote: > On 17 October 2016 at 09:35, Radim Vansa wrote: >> On 10/17/2016 09:07 AM, Tristan Tarrant wrote: >>> Hi all, >>> >>> something trivial and fun for a Monday morning. >>> >>> I've just issued a PR [1] to update the codename for Infinispan 9.0. >> >> Checked out ISPN-1414 [1], status is Open (though it's most likely out >> of date). >> >> [1] https://issues.jboss.org/browse/ISPN-1414 >> >>> >>> And while we're at it, let's give a name to the new query language that >>> Adrian (and Emmanuel) have designed. We already have a number of >>> suggestions (which I summarize below) but please feel free to add your >>> own. Please vote. >>> >>> IQL (Infinispan Query Language, already used by others). >>> Ickle (Alternate pronunciation of above, also means "small") >>> LQID (Language for Querying Infinispan Datagrids) >>> QuIL ("Query Infinispan" Language) >> >> I like LQID with the pronounciation "liquid", but then we need to keep >> some fluent API at least for Java clients :) > > I'd prefer a liquid API ;) > > About the names: I like both IQL and LQID > > Or may I propose another candidate: "INAQL" > which stands for all of: > - Inaql is Not Another Query Language > - INfinispan Advanced Query Language > - Infinispan Needs A Query Language > I like that last one, but I don't think the article should be part of the acronym, so I'd go with INQL :) Dan _______________________________________________ infinispan-dev mailing list infinispan-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev From valerio.schiavoni at gmail.com Mon Oct 17 11:38:50 2016 From: valerio.schiavoni at gmail.com (Valerio Schiavoni) Date: Mon, 17 Oct 2016 17:38:50 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: On Mon, Oct 17, 2016 at 5:30 PM, ugol wrote: > inql sounds weird in Italian :) > > > >> I like that last one, but I don't think the article should be part of > >> the acronym, so I'd go with INQL :) > > > > +1 even better indeed > -1 for INQL, as Ugo said, the italian crowd won't be pleased ... ;-) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161017/a2c09fde/attachment-0001.html From sanne at infinispan.org Mon Oct 17 11:48:08 2016 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 17 Oct 2016 17:48:08 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: On 17 October 2016 at 17:38, Valerio Schiavoni wrote: > > On Mon, Oct 17, 2016 at 5:30 PM, ugol wrote: >> >> inql sounds weird in Italian :) >> >> >> >> I like that last one, but I don't think the article should be part of >> >> the acronym, so I'd go with INQL :) >> > >> > +1 even better indeed > > > -1 for INQL, as Ugo said, the italian crowd won't be pleased ... ;-) HaHa! Thanks Valerio and Ugo; I should have known better too since Italian is my first language but for some reason when in this context I'm thinking in English. Ok so I'd suggest to discard INQL and I revert to my previous suggestion "INAQL" however as mentioned I also like IQL and LQID. Tristan, sorry for the mess :) I'd say we're all giving (hopefully) useful input, but Adrian should get to decide how to call his baby. Thanks, Sanne From anistor at redhat.com Mon Oct 17 12:25:58 2016 From: anistor at redhat.com (Adrian Nistor) Date: Mon, 17 Oct 2016 19:25:58 +0300 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: Is IQL a reasonable name? I'd go for this one. On 10/17/2016 06:48 PM, Sanne Grinovero wrote: > On 17 October 2016 at 17:38, Valerio Schiavoni > wrote: >> On Mon, Oct 17, 2016 at 5:30 PM, ugol wrote: >>> inql sounds weird in Italian :) >>> >>> >>>>> I like that last one, but I don't think the article should be part of >>>>> the acronym, so I'd go with INQL :) >>>> +1 even better indeed >> >> -1 for INQL, as Ugo said, the italian crowd won't be pleased ... ;-) > HaHa! Thanks Valerio and Ugo; I should have known better too since > Italian is my first language but for some reason when in this context > I'm thinking in English. > > Ok so I'd suggest to discard INQL and I revert to my previous > suggestion "INAQL" however as mentioned I also like IQL and LQID. > Tristan, sorry for the mess :) I'd say we're all giving (hopefully) > useful input, but Adrian should get to decide how to call his baby. > > Thanks, > Sanne > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From daltodavide at gmail.com Mon Oct 17 17:51:16 2016 From: daltodavide at gmail.com (Davide D'Alto) Date: Mon, 17 Oct 2016 22:51:16 +0100 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: +1 for IQL On Mon, Oct 17, 2016 at 5:25 PM, Adrian Nistor wrote: > Is IQL a reasonable name? I'd go for this one. > > On 10/17/2016 06:48 PM, Sanne Grinovero wrote: >> On 17 October 2016 at 17:38, Valerio Schiavoni >> wrote: >>> On Mon, Oct 17, 2016 at 5:30 PM, ugol wrote: >>>> inql sounds weird in Italian :) >>>> >>>> >>>>>> I like that last one, but I don't think the article should be part of >>>>>> the acronym, so I'd go with INQL :) >>>>> +1 even better indeed >>> >>> -1 for INQL, as Ugo said, the italian crowd won't be pleased ... ;-) >> HaHa! Thanks Valerio and Ugo; I should have known better too since >> Italian is my first language but for some reason when in this context >> I'm thinking in English. >> >> Ok so I'd suggest to discard INQL and I revert to my previous >> suggestion "INAQL" however as mentioned I also like IQL and LQID. >> Tristan, sorry for the mess :) I'd say we're all giving (hopefully) >> useful input, but Adrian should get to decide how to call his baby. >> >> Thanks, >> Sanne >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From ttarrant at redhat.com Tue Oct 18 02:10:43 2016 From: ttarrant at redhat.com (Tristan Tarrant) Date: Tue, 18 Oct 2016 08:10:43 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: <40750417-adf2-0b20-f462-65b99380aade@infinispan.org> The reason I wanted to avoid it is that it has already been used to indicate a few query languages already (albeit obscure ones). But there is nothing inherently wrong with it. iQL: A Query Language for the Instance-Based Data Model IQL (Imhotep Query Language). Insight Query Language (IQL) Induct query language (IQL) Imperative Query Language (IQL) Tristan On 17/10/16 18:25, Adrian Nistor wrote: > Is IQL a reasonable name? I'd go for this one. > > On 10/17/2016 06:48 PM, Sanne Grinovero wrote: >> On 17 October 2016 at 17:38, Valerio Schiavoni >> wrote: >>> On Mon, Oct 17, 2016 at 5:30 PM, ugol wrote: >>>> inql sounds weird in Italian :) >>>> >>>> >>>>>> I like that last one, but I don't think the article should be part of >>>>>> the acronym, so I'd go with INQL :) >>>>> +1 even better indeed >>> >>> -1 for INQL, as Ugo said, the italian crowd won't be pleased ... ;-) >> HaHa! Thanks Valerio and Ugo; I should have known better too since >> Italian is my first language but for some reason when in this context >> I'm thinking in English. >> >> Ok so I'd suggest to discard INQL and I revert to my previous >> suggestion "INAQL" however as mentioned I also like IQL and LQID. >> Tristan, sorry for the mess :) I'd say we're all giving (hopefully) >> useful input, but Adrian should get to decide how to call his baby. >> >> Thanks, >> Sanne >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From rory.odonnell at oracle.com Sat Oct 22 09:56:51 2016 From: rory.odonnell at oracle.com (Rory O'Donnell) Date: Sat, 22 Oct 2016 21:56:51 +0800 Subject: [infinispan-dev] Early Access builds for JDK 8u122 b02 , JDK 9 & JDK 9 with Project Jigsaw b140 are available on java.net Message-ID: Hi Galder, Early Access b02 for JDK 8u122 is available , summary of changes are listed here. Early Access b140 (#5625) for JDK 9 with Project Jigsaw is available on java.net, summary of changes are listed here. Early Access b140 for JDK 9 is available on java.net, summary of changes are listed here . A couple of items to point out with regard to b140: 1. A fix for Cyclic interface initialization causes JVM crash is included in b140 2. We are requesting feedback on a change that went into JDK 9 b140 The java.io.FilePermission class was changed to remove pathname canonicalization from its creation, along with a system property to revert the behavior back to way it worked in the previous JDK release. We do this mainly for performance enhancement so that there is no need to consult the file system every time a FilePermisson is created. If you use a security manager and file permissions then you should read the details as described on the jdk9 mailing list [1] Feedback is requested via core-libs-dev at openjdk.java.net The new GA date of JDK 9 has been updated on the JDK 9 Project page [2], further details in Mark Reinhold's email [3]. Rgds,Rory [1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-October/005062.html [2] http://openjdk.java.net/projects/jdk9/ [3] http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-October/005092.html -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA, Dublin,Ireland -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161022/72d17330/attachment.html From sanne at infinispan.org Mon Oct 24 12:42:21 2016 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 24 Oct 2016 17:42:21 +0100 Subject: [infinispan-dev] Some memory usage data points of the Hibernate Search / Lucene query engine. Message-ID: Hi all, at our last meeting we had some chats about heap usage at runtime. I wasn't actually investigating this, but since I was running some tests now I thought it was interesting to share the following figures about the Hibernate Search / Lucene query engine. # Index size My test is running with an off-heap index of about 5 million entries, which translates to about 170MB of disk space. # Query type We're running a large query which matches every single one of the 5 million entries, and sorting the results on a Float property. # TLAB usage Running such a query takes 2.78 MB of TLAB; this data is never promoted so it's "cheap" to collect as there's no much interaction with other threads. # Beyond TLAB Allocating zero bytes ;) In terms of memory usage this looks quite good; kudos to the Lucene team of course as they do the heavy lifting, but I'm proud of the Hibernate Search team as well as we don't add significant overhead. We still have some work to do, as while memory looks good the sorting on Floats is still very heavy.. but we'll figure it out. Looking forward to see similar metrics from get/put tests on Infinispan ;-) N.B. this is an "off heap" index, but even if I use an "on heap" index, above figures are pretty much the same, and the performance doesn't get higher, but slower. Thanks, Sanne From emmanuel at hibernate.org Wed Oct 26 02:40:05 2016 From: emmanuel at hibernate.org (Emmanuel Bernard) Date: Wed, 26 Oct 2016 08:40:05 +0200 Subject: [infinispan-dev] Some memory usage data points of the Hibernate Search / Lucene query engine. In-Reply-To: References: Message-ID: <20161026064005.GA72969@hibernate.org> To put some context, that's a very unfavorable query for Lucene (sort by data other than score, retrieve all the data set). On Mon 16-10-24 17:42, Sanne Grinovero wrote: >Hi all, >at our last meeting we had some chats about heap usage at runtime. > >I wasn't actually investigating this, but since I was running some >tests now I thought it was interesting to share the following figures >about the Hibernate Search / Lucene query engine. > ># Index size >My test is running with an off-heap index of about 5 million entries, >which translates to about 170MB of disk space. > ># Query type >We're running a large query which matches every single one of the 5 >million entries, and sorting the results on a Float property. > ># TLAB usage >Running such a query takes 2.78 MB of TLAB; this data is never >promoted so it's "cheap" to collect as there's no much interaction >with other threads. > ># Beyond TLAB >Allocating zero bytes ;) > >In terms of memory usage this looks quite good; kudos to the Lucene >team of course as they do the heavy lifting, but I'm proud of the >Hibernate Search team as well as we don't add significant overhead. >We still have some work to do, as while memory looks good the sorting >on Floats is still very heavy.. but we'll figure it out. > >Looking forward to see similar metrics from get/put tests on Infinispan ;-) > >N.B. this is an "off heap" index, but even if I use an "on heap" >index, above figures are pretty much the same, and the performance >doesn't get higher, but slower. > >Thanks, >Sanne >_______________________________________________ >infinispan-dev mailing list >infinispan-dev at lists.jboss.org >https://lists.jboss.org/mailman/listinfo/infinispan-dev From dan.berindei at gmail.com Wed Oct 26 12:03:23 2016 From: dan.berindei at gmail.com (Dan Berindei) Date: Wed, 26 Oct 2016 19:03:23 +0300 Subject: [infinispan-dev] Func API in tx cache In-Reply-To: <57FD1E72.2080201@redhat.com> References: <57EA876D.6060300@redhat.com> <3AD09175-A621-41B0-B4F4-75C697FF57E7@redhat.com> <57FD1E72.2080201@redhat.com> Message-ID: In theory something like this should work, resuming the transaction after each asynchronous operation: https://paste.fedoraproject.org/461451/77496416/raw/ I'm not sure about the usability, or performance... The TransactionManager API doesn't allow asynchronous commit(), so one might as well bite the bullet and spawn a new thread to run their transaction in. A more pie-in-the-sky idea for the the functional API would be to support "batching" multiple evals over the same cache, not involving the transaction manager. That should allow us to catch WriteSkewExceptions and retry the batch ourselves, assuming the lambdas don't have side-effects. And if the transaction is Infinispan-only, we can also make the commit asynchronous. There's always the option of forcing the functional operations to be synchronous when they're joining a running TransactionManager transaction... Cheers Dan On Tue, Oct 11, 2016 at 8:16 PM, Radim Vansa wrote: > Hi Galder, > > I've looked to the Narayana thread but I also have some trouble > understanding the R ret = CompletableFuture.supplyAsync(...) - that > method returns CF, not directly R... so the answer that guys on the > forum give you makes sense to me. Maybe you could revive that thread and > clarify what you had in mind. > > You're describing general issue of async calls, not just functional API. > Functional API is about moving some lambda to be executed in situ. Your > question is no different as if you were calling putAsync and trying to > chain it with another operations. TBH I haven't dealt with this problem > at all, the tests I've written use the operations in sync way. > > If you try to do something transactionally-sensitive as a part of the > lambda itself, you're doing it wrong - the lambda can be executed on > different node which has no link to the original transaction. And the > lambda should be purely functional (no side effects), otherwise you're > poking the devil. > > Regarding RC vs. RR., in [1] I've decided to attach version on which the > functional command operated with the return value if versions (and WSC) > is used. Then, if the version changes during the transaction (say, > you've did a functional read, and it changes before second read), > WriteSkewException is thrown as we're not operating on the same value. > However, this does not work after you modify the entry with a functional > command; then your reads still work on the non-modified entry... I have > to tweak this a bit yet, basically sending all the modifications to a > given entry along with the read :-/ > Note: RR without WSC is just broken, I should add a warning log somewhere... > > Radim > > [1] https://github.com/infinispan/infinispan/pull/4608 > > On 10/10/2016 05:49 PM, Galder Zamarre?o wrote: >> Hey Radim, >> >> Sorry for late reply. Indeed, I think you should start by getting RC done first. >> >> When I first experimented with this, I had bigger issues than the ones explained here. For example, how to pass down transaction context when multiple async operations are being executed within a transaction. >> >> In particular, say you have two async operations, chained, executed with T1 and T2 respectively, both part of the the same transaction. Suspending tx in T1 and resuming it when T2 starts is easy, the problem I had was how to get T1 to resume the transaction when T2 has completed async transactional operation. At the time I created a forum post in the transactions forum about it [1]. I was not able to able to try out the solution in [1], but it doesn't feel very idiomatic to me (e.g. needing to call CompletableFuture.get()). >> >> Did you consider this issue? Or are you avoiding it somehow? >> >> Cheers, >> >> [1] https://developer.jboss.org/thread/265595 >> -- >> Galder Zamarre?o >> Infinispan, Red Hat >> >>> On 27 Sep 2016, at 16:51, Radim Vansa wrote: >>> >>> Hi, >>> >>> seems I'll have to implement the functional stuff on tx caches [1][2] if >>> I want to get rid of DeltaAware et al. >>> >>> The general idea is quite simple - ReadOnly* commands should behave very >>> similar to non-tx mode, WriteOnly* commands will be just added as >>> modifications to the PrepareCommand and ReadWrite* commands will be both >>> added to modifications list, and sent to remote nodes where the result >>> won't be stored yet. >>> The results of operations should not be stored into transactional >>> context - the command will execute remotely (if the owners are remote) >>> unless the value was read by Get* beforehand. >>> >>> With repeatable-reads isolation, the situation gets more complicated. If >>> we use ReadOnly* that performs identity lookup (effectively the same as >>> Get*) and the entry was modified in during the transaction, we can >>> return two different results - so a read committed semantics. With write >>> skew check enabled, we could at least fail the transaction at the end >>> (the check would be performed reads as well if the transaction contains >>> functional reads), but we cannot rely on WSC always on with RR. >>> >>> Retrieving the whole entry and applying the functional command is not a >>> viable solution, IMO - that would completely defy the purpose of using >>> functional command. >>> >>> A possible solution would be to send the global transaction ID with >>> those read commands and keep a remote transactional context with read >>> entries for the duration of transaction on remote nodes, too. However, >>> if we do a Read* command to primary owner, it's possible that further >>> Get* command will hit backup. So, we could go to all owners with Read* >>> already during the transaction (slowing down functional reads >>> considerably), or read only from primary owner (which slows down Get*s >>> even if we don't use functional APIs - this makes it a no-go). I am not >>> 100% sure how a transaction transfer during ST will get into that. >>> >>> We could also do it the ostrich way - "Yes we've promissed RR but Func >>> will be only RC". I'll probably do that in the first draft anyway. >>> >>> Comments & opinions appreciated. >>> >>> Radim >>> >>> [1] https://issues.jboss.org/browse/ISPN-5806 >>> [2] https://issues.jboss.org/browse/ISPN-6573 >>> >>> -- >>> Radim Vansa >>> JBoss Performance Team >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From mudokonman at gmail.com Wed Oct 26 18:06:43 2016 From: mudokonman at gmail.com (William Burns) Date: Wed, 26 Oct 2016 22:06:43 +0000 Subject: [infinispan-dev] Data Container configuration changes Message-ID: I have been working on adding in off heap support for a given cache. I wanted to check in and let you all know what I was thinking for the configuration and changes that would come about with it. TLDR; New config under data container to enable off heap, StoreAsBinary removed, Equivalence removed First I was planning on adding new sub elements of data container. These would be instance, binary and off-heap. Only of the three could be picked as they are mutually exclusive. Instance is as we operate now where we store the instance of the object passed to us. Binary is essentially what we have now that is called storeAsBinary with both keys and values converted. Lastly off-heap would store the entry as a byte[] store completely in native memory. Example: The reason it is a subelement instead of a property is because off-heap will most likely require some additional configuration to tell how many entries to store in the a bucket (think non resizing HashMap). With these changes storeAsBinary becomes redundant, so I was planning on removing this configuration completely. I would rather remove since this is 9.0 and not deprecate. As far as I know nobody really used it before. Also another side effect is I was removing all of the Equivalence classes. I am not sure if I can plainly remove them since they have lived in commons for quite a while, but it would be best if I could, although I am fine deprecating. In its place the instance setting for data-container will always wrap byte[] to satisfy equals and hashCode methods. Any feedback would be appreciated. Thanks, - Will -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161026/fa5bae03/attachment.html From rvansa at redhat.com Thu Oct 27 04:36:14 2016 From: rvansa at redhat.com (Radim Vansa) Date: Thu, 27 Oct 2016 10:36:14 +0200 Subject: [infinispan-dev] Async API (was: Func API in tx cache) In-Reply-To: References: <57EA876D.6060300@redhat.com> <3AD09175-A621-41B0-B4F4-75C697FF57E7@redhat.com> <57FD1E72.2080201@redhat.com> Message-ID: First of all, please let's separate "functional API" and "async API". Functional API already provides only the async API, but this discussion should not be specific to functional calls. More inline. On 10/26/2016 06:03 PM, Dan Berindei wrote: > In theory something like this should work, resuming the transaction > after each asynchronous operation: > > https://paste.fedoraproject.org/461451/77496416/raw/ > > I'm not sure about the usability, or performance... The > TransactionManager API doesn't allow asynchronous commit(), so one > might as well bite the bullet and spawn a new thread to run their > transaction in. I don't think that the commitTransactionAsync() is correct, because it assumes that continueTransactionAsync() was invoked beforehand. If you run just cache.commitTransactionAsync(cache.putAsync(...)) the thread calling the handler may not be associated with given tx. So you would have to use: default TxCompletableFuture continueTransactionAsync(CompletableFuture asyncOperation) { ... } default CompletableFuture commitTransactionAsync(TxCompletableFuture asyncOperation) { ... } to enforce invoking continue. I am starting to worry our LocalTransaction, TxInvocationContext and other classes that currently assume single-threaded access. Once we start executing multiple async operations using the same context, we'll see tons of race conditions. It's not hard to handle them within execution of single command (e.g. when doing getAll, we already have to sync the responses if using invokeRemotelyAsync), but with multiple commands it's a different cup of coffee. I would consider single big lock on the context/transaction that will guarantee that commands for the same transaction won't run in parallel (so put(keyA, ...) will get to a point when it's waiting for an entry lock/doing RPC, and only then a put(keyB, ...) can continue. As the RPC responses come, these will be executed serially). While this is non-trivial to implement, it will simplify the reasoning - I am worried that just changing all collections to concurrent ones won't do the trick. I've checked JTA spec about thread-safety and it permits multiple threads working on the same transaction but it's a bit vague how to achieve that: "Multiple threads may concurrently be associated with the same global transaction." "Note that some transaction manager implementations allow a suspended transaction to be resumed by a different thread. This feature is not required by JTA." So it's probably a question to Narayana guys if you can suspend a transaction in one thread and resume it in two threads. JTA is not too much async-friendly, as XAResource docs says: The XAResource interface, in order to be better integrated with the Java environment, differs from the standard X/Open XA interface in the following ways: * Asynchronous operations are not supported. Java supports multi-threaded processing and most databases do not support asynchronous operations. * The DTP concept of ?Thread of Control? maps to all Java threads that are given access to the XAResource and Connection objects. For example, it is legal (although in practice rarely used) for two different Java threads to perform the start and end operations on the same XAResource object. * ... > > A more pie-in-the-sky idea for the the functional API would be to > support "batching" multiple evals over the same cache, not involving > the transaction manager. That should allow us to catch > WriteSkewExceptions and retry the batch ourselves, assuming the > lambdas don't have side-effects. And if the transaction is > Infinispan-only, we can also make the commit asynchronous. I am not sure if I follow... you're assuming that all the async ops are fired without any logic based on intermediate results. WriteSkewException is thrown when you've already read something, and then decided based on the value. R. > > There's always the option of forcing the functional operations to be > synchronous when they're joining a running TransactionManager > transaction... > > Cheers > Dan > > > On Tue, Oct 11, 2016 at 8:16 PM, Radim Vansa wrote: >> Hi Galder, >> >> I've looked to the Narayana thread but I also have some trouble >> understanding the R ret = CompletableFuture.supplyAsync(...) - that >> method returns CF, not directly R... so the answer that guys on the >> forum give you makes sense to me. Maybe you could revive that thread and >> clarify what you had in mind. >> >> You're describing general issue of async calls, not just functional API. >> Functional API is about moving some lambda to be executed in situ. Your >> question is no different as if you were calling putAsync and trying to >> chain it with another operations. TBH I haven't dealt with this problem >> at all, the tests I've written use the operations in sync way. >> >> If you try to do something transactionally-sensitive as a part of the >> lambda itself, you're doing it wrong - the lambda can be executed on >> different node which has no link to the original transaction. And the >> lambda should be purely functional (no side effects), otherwise you're >> poking the devil. >> >> Regarding RC vs. RR., in [1] I've decided to attach version on which the >> functional command operated with the return value if versions (and WSC) >> is used. Then, if the version changes during the transaction (say, >> you've did a functional read, and it changes before second read), >> WriteSkewException is thrown as we're not operating on the same value. >> However, this does not work after you modify the entry with a functional >> command; then your reads still work on the non-modified entry... I have >> to tweak this a bit yet, basically sending all the modifications to a >> given entry along with the read :-/ >> Note: RR without WSC is just broken, I should add a warning log somewhere... >> >> Radim >> >> [1] https://github.com/infinispan/infinispan/pull/4608 >> >> On 10/10/2016 05:49 PM, Galder Zamarre?o wrote: >>> Hey Radim, >>> >>> Sorry for late reply. Indeed, I think you should start by getting RC done first. >>> >>> When I first experimented with this, I had bigger issues than the ones explained here. For example, how to pass down transaction context when multiple async operations are being executed within a transaction. >>> >>> In particular, say you have two async operations, chained, executed with T1 and T2 respectively, both part of the the same transaction. Suspending tx in T1 and resuming it when T2 starts is easy, the problem I had was how to get T1 to resume the transaction when T2 has completed async transactional operation. At the time I created a forum post in the transactions forum about it [1]. I was not able to able to try out the solution in [1], but it doesn't feel very idiomatic to me (e.g. needing to call CompletableFuture.get()). >>> >>> Did you consider this issue? Or are you avoiding it somehow? >>> >>> Cheers, >>> >>> [1] https://developer.jboss.org/thread/265595 >>> -- >>> Galder Zamarre?o >>> Infinispan, Red Hat >>> >>>> On 27 Sep 2016, at 16:51, Radim Vansa wrote: >>>> >>>> Hi, >>>> >>>> seems I'll have to implement the functional stuff on tx caches [1][2] if >>>> I want to get rid of DeltaAware et al. >>>> >>>> The general idea is quite simple - ReadOnly* commands should behave very >>>> similar to non-tx mode, WriteOnly* commands will be just added as >>>> modifications to the PrepareCommand and ReadWrite* commands will be both >>>> added to modifications list, and sent to remote nodes where the result >>>> won't be stored yet. >>>> The results of operations should not be stored into transactional >>>> context - the command will execute remotely (if the owners are remote) >>>> unless the value was read by Get* beforehand. >>>> >>>> With repeatable-reads isolation, the situation gets more complicated. If >>>> we use ReadOnly* that performs identity lookup (effectively the same as >>>> Get*) and the entry was modified in during the transaction, we can >>>> return two different results - so a read committed semantics. With write >>>> skew check enabled, we could at least fail the transaction at the end >>>> (the check would be performed reads as well if the transaction contains >>>> functional reads), but we cannot rely on WSC always on with RR. >>>> >>>> Retrieving the whole entry and applying the functional command is not a >>>> viable solution, IMO - that would completely defy the purpose of using >>>> functional command. >>>> >>>> A possible solution would be to send the global transaction ID with >>>> those read commands and keep a remote transactional context with read >>>> entries for the duration of transaction on remote nodes, too. However, >>>> if we do a Read* command to primary owner, it's possible that further >>>> Get* command will hit backup. So, we could go to all owners with Read* >>>> already during the transaction (slowing down functional reads >>>> considerably), or read only from primary owner (which slows down Get*s >>>> even if we don't use functional APIs - this makes it a no-go). I am not >>>> 100% sure how a transaction transfer during ST will get into that. >>>> >>>> We could also do it the ostrich way - "Yes we've promissed RR but Func >>>> will be only RC". I'll probably do that in the first draft anyway. >>>> >>>> Comments & opinions appreciated. >>>> >>>> Radim >>>> >>>> [1] https://issues.jboss.org/browse/ISPN-5806 >>>> [2] https://issues.jboss.org/browse/ISPN-6573 >>>> >>>> -- >>>> Radim Vansa >>>> JBoss Performance Team >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> Radim Vansa >> JBoss Performance Team >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From galder at redhat.com Thu Oct 27 04:38:11 2016 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Thu, 27 Oct 2016 10:38:11 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: <58047F4A.9090700@redhat.com> Message-ID: <5BD55C89-D357-43C6-B9C8-08EC84B08D09@redhat.com> Sounds weird in Spanish too ;) -- Galder Zamarre?o Infinispan, Red Hat > On 17 Oct 2016, at 17:38, Valerio Schiavoni wrote: > > > On Mon, Oct 17, 2016 at 5:30 PM, ugol wrote: > inql sounds weird in Italian :) > > > >> I like that last one, but I don't think the article should be part of > >> the acronym, so I'd go with INQL :) > > > > +1 even better indeed > > -1 for INQL, as Ugo said, the italian crowd won't be pleased ... ;-) > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From jonathan.halliday at redhat.com Thu Oct 27 04:49:51 2016 From: jonathan.halliday at redhat.com (Jonathan Halliday) Date: Thu, 27 Oct 2016 09:49:51 +0100 Subject: [infinispan-dev] Async API In-Reply-To: References: <57EA876D.6060300@redhat.com> <3AD09175-A621-41B0-B4F4-75C697FF57E7@redhat.com> <57FD1E72.2080201@redhat.com> Message-ID: <4f282ea4-8e8b-8fe9-f8d9-268aba101d12@redhat.com> On 27/10/16 09:36, Radim Vansa wrote: > So it's probably a question to Narayana guys if you can suspend a > transaction in one thread and resume it in two threads. yes you can. Where it gets hairy is ensuring all the threads are done before you commit. By default we warn rather than block, though there is a (non-standard) plugin api to register your own handler for it. Jonathan. -- Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From pedro at infinispan.org Thu Oct 27 04:56:05 2016 From: pedro at infinispan.org (Pedro Ruivo) Date: Thu, 27 Oct 2016 09:56:05 +0100 Subject: [infinispan-dev] Data Container configuration changes In-Reply-To: References: Message-ID: <122bcbd2-bb36-815e-c69d-b19f3716362d@infinispan.org> On 26-10-2016 23:06, William Burns wrote: > I have been working on adding in off heap support for a given cache. I > wanted to check in and let you all know what I was thinking for the > configuration and changes that would come about with it. > > TLDR; > New config under data container to enable off heap, StoreAsBinary > removed, Equivalence removed > > First I was planning on adding new sub elements of data container. > These would be instance, binary and off-heap. Only of the three could > be picked as they are mutually exclusive. Instance is as we operate now > where we store the instance of the object passed to us. Binary is > essentially what we have now that is called storeAsBinary with both keys > and values converted. Lastly off-heap would store the entry as a byte[] > store completely in native memory. I prefer 'object' instead of 'instance'. Are you also planning to remove the expiration and/or eviction configuration elements and set them in the data-container sub elements? > > Example: > > > > > > The reason it is a subelement instead of a property is because off-heap > will most likely require some additional configuration to tell how many > entries to store in the a bucket (think non resizing HashMap). > > With these changes storeAsBinary becomes redundant, so I was planning on > removing this configuration completely. I would rather remove since > this is 9.0 and not deprecate. As far as I know nobody really used it > before. > > Also another side effect is I was removing all of the Equivalence > classes. I am not sure if I can plainly remove them since they have > lived in commons for quite a while, but it would be best if I could, > although I am fine deprecating. In its place the instance setting for > data-container will always wrap byte[] to satisfy equals and hashCode > methods. > > Any feedback would be appreciated. > > Thanks, > - Will > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > From mudokonman at gmail.com Thu Oct 27 09:20:36 2016 From: mudokonman at gmail.com (William Burns) Date: Thu, 27 Oct 2016 13:20:36 +0000 Subject: [infinispan-dev] Data Container configuration changes In-Reply-To: <122bcbd2-bb36-815e-c69d-b19f3716362d@infinispan.org> References: <122bcbd2-bb36-815e-c69d-b19f3716362d@infinispan.org> Message-ID: On Thu, Oct 27, 2016 at 4:56 AM Pedro Ruivo wrote: > > > On 26-10-2016 23:06, William Burns wrote: > > I have been working on adding in off heap support for a given cache. I > > wanted to check in and let you all know what I was thinking for the > > configuration and changes that would come about with it. > > > > TLDR; > > New config under data container to enable off heap, StoreAsBinary > > removed, Equivalence removed > > > > First I was planning on adding new sub elements of data container. > > These would be instance, binary and off-heap. Only of the three could > > be picked as they are mutually exclusive. Instance is as we operate now > > where we store the instance of the object passed to us. Binary is > > essentially what we have now that is called storeAsBinary with both keys > > and values converted. Lastly off-heap would store the entry as a byte[] > > store completely in native memory. > > I prefer 'object' instead of 'instance'. > Sounds fine with me. > > Are you also planning to remove the expiration and/or eviction > configuration elements and set them in the data-container sub elements? > I wasn't planning on touching those. But now that you mention it, eviction only makes sense in the data container, where as expiration is container and cache store. And taking this further storeAsBinary is both as well, only off-heap is container only. I wonder if instead we should have a separate storage element at the same level as data-container. I can see it either way. > > > > > > Example: > > > > > > > > > > > > The reason it is a subelement instead of a property is because off-heap > > will most likely require some additional configuration to tell how many > > entries to store in the a bucket (think non resizing HashMap). > > > > With these changes storeAsBinary becomes redundant, so I was planning on > > removing this configuration completely. I would rather remove since > > this is 9.0 and not deprecate. As far as I know nobody really used it > > before. > > > > Also another side effect is I was removing all of the Equivalence > > classes. I am not sure if I can plainly remove them since they have > > lived in commons for quite a while, but it would be best if I could, > > although I am fine deprecating. In its place the instance setting for > > data-container will always wrap byte[] to satisfy equals and hashCode > > methods. > > > > Any feedback would be appreciated. > > > > Thanks, > > - Will > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161027/5eb3c73a/attachment.html From ttarrant at redhat.com Thu Oct 27 09:37:53 2016 From: ttarrant at redhat.com (Tristan Tarrant) Date: Thu, 27 Oct 2016 15:37:53 +0200 Subject: [infinispan-dev] Data Container configuration changes In-Reply-To: References: Message-ID: <58188460-9945-cb53-31da-775587051a5f@infinispan.org> On 27/10/16 00:06, William Burns wrote: > The reason it is a subelement instead of a property is because off-heap > will most likely require some additional configuration to tell how many > entries to store in the a bucket (think non resizing HashMap). Aside from that, it is best to use elements when possible over attributes, as XSD allows more control over constraints and exclusivity. > With these changes storeAsBinary becomes redundant, so I was planning on > removing this configuration completely. I would rather remove since > this is 9.0 and not deprecate. As far as I know nobody really used it > before. While this can be easily removed from the parser, since it supports versioning, I'm not sure I'd want to see it go away from the API. We haven't been very good at doing this in the past and I'd rather go down the conservative route. > Also another side effect is I was removing all of the Equivalence > classes. I am not sure if I can plainly remove them since they have > lived in commons for quite a while, but it would be best if I could, > although I am fine deprecating. In its place the instance setting for > data-container will always wrap byte[] to satisfy equals and hashCode > methods. Equivalence is only really used in the context of compatibility mode and that has uses (hybrid servers). How would that continue working (until we get transcoding) ? Tristan -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From pedro at infinispan.org Thu Oct 27 09:46:40 2016 From: pedro at infinispan.org (Pedro Ruivo) Date: Thu, 27 Oct 2016 14:46:40 +0100 Subject: [infinispan-dev] Data Container configuration changes In-Reply-To: References: <122bcbd2-bb36-815e-c69d-b19f3716362d@infinispan.org> Message-ID: On 27-10-2016 14:20, William Burns wrote: > > > On Thu, Oct 27, 2016 at 4:56 AM Pedro Ruivo > wrote: > > > > On 26-10-2016 23:06, William Burns wrote: > > I have been working on adding in off heap support for a given > cache. I > > wanted to check in and let you all know what I was thinking for the > > configuration and changes that would come about with it. > > > > TLDR; > > New config under data container to enable off heap, StoreAsBinary > > removed, Equivalence removed > > > > First I was planning on adding new sub elements of data container. > > These would be instance, binary and off-heap. Only of the three could > > be picked as they are mutually exclusive. Instance is as we > operate now > > where we store the instance of the object passed to us. Binary is > > essentially what we have now that is called storeAsBinary with > both keys > > and values converted. Lastly off-heap would store the entry as a > byte[] > > store completely in native memory. > > I prefer 'object' instead of 'instance'. > > > Sounds fine with me. > > > > Are you also planning to remove the expiration and/or eviction > configuration elements and set them in the data-container sub elements? > > > I wasn't planning on touching those. But now that you mention it, > eviction only makes sense in the data container, where as expiration is > container and cache store. And taking this further storeAsBinary is > both as well, only off-heap is container only. I wonder if instead we > should have a separate storage element at the same level as > data-container. I can see it either way. Let me know if this makes sense: //no changes here //one of the following //no changes here wdyt? > > > > > > > > Example: > > > > > > > > > > > > The reason it is a subelement instead of a property is because > off-heap > > will most likely require some additional configuration to tell how > many > > entries to store in the a bucket (think non resizing HashMap). > > > > With these changes storeAsBinary becomes redundant, so I was > planning on > > removing this configuration completely. I would rather remove since > > this is 9.0 and not deprecate. As far as I know nobody really used it > > before. > > > > Also another side effect is I was removing all of the Equivalence > > classes. I am not sure if I can plainly remove them since they have > > lived in commons for quite a while, but it would be best if I could, > > although I am fine deprecating. In its place the instance setting for > > data-container will always wrap byte[] to satisfy equals and hashCode > > methods. > > > > Any feedback would be appreciated. > > > > Thanks, > > - Will > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > From mudokonman at gmail.com Thu Oct 27 10:01:54 2016 From: mudokonman at gmail.com (William Burns) Date: Thu, 27 Oct 2016 14:01:54 +0000 Subject: [infinispan-dev] Data Container configuration changes In-Reply-To: <58188460-9945-cb53-31da-775587051a5f@infinispan.org> References: <58188460-9945-cb53-31da-775587051a5f@infinispan.org> Message-ID: On Thu, Oct 27, 2016 at 9:40 AM Tristan Tarrant wrote: > On 27/10/16 00:06, William Burns wrote: > > The reason it is a subelement instead of a property is because off-heap > > will most likely require some additional configuration to tell how many > > entries to store in the a bucket (think non resizing HashMap). > > Aside from that, it is best to use elements when possible over > attributes, as XSD allows more control over constraints and exclusivity. > > > With these changes storeAsBinary becomes redundant, so I was planning on > > removing this configuration completely. I would rather remove since > > this is 9.0 and not deprecate. As far as I know nobody really used it > > before. > > While this can be easily removed from the parser, since it supports > versioning, I'm not sure I'd want to see it go away from the API. We > haven't been very good at doing this in the past and I'd rather go down > the conservative route. > Not sure I understand, so you are saying to remove it from the parser/xsd but keep it in the programmatic configuration? And just to clarify the new binary element would replace storeAsBinary. > > > Also another side effect is I was removing all of the Equivalence > > classes. I am not sure if I can plainly remove them since they have > > lived in commons for quite a while, but it would be best if I could, > > although I am fine deprecating. In its place the instance setting for > > data-container will always wrap byte[] to satisfy equals and hashCode > > methods. > > Equivalence is only really used in the context of compatibility mode and > that has uses (hybrid servers). How would that continue working (until > we get transcoding) ? > Equivalence is only used when you are storing byte[] objects, which Infinispan would support directly with no configuration with the new changes (we would wrap the byte[] to ensure equals/hashCode). Compatibility would work the same way it does right now, I just had to add in some handling for the wrapper in some parts of code. > > Tristan > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161027/291b5b13/attachment.html From ttarrant at redhat.com Thu Oct 27 10:13:18 2016 From: ttarrant at redhat.com (Tristan Tarrant) Date: Thu, 27 Oct 2016 16:13:18 +0200 Subject: [infinispan-dev] Data Container configuration changes In-Reply-To: References: <58188460-9945-cb53-31da-775587051a5f@infinispan.org> Message-ID: <3193b08b-748a-b5f2-81b8-d72dc0039a6a@infinispan.org> On 27/10/16 16:01, William Burns wrote: > Not sure I understand, so you are saying to remove it from the > parser/xsd but keep it in the programmatic configuration? And just to > clarify the new binary element would replace storeAsBinary. Yes. The parser, when confronted with a pre 9.0 schema, would print out a warning, but it would use the new binary element internally. > Equivalence is only used when you are storing byte[] objects, which > Infinispan would support directly with no configuration with the new > changes (we would wrap the byte[] to ensure equals/hashCode). > Compatibility would work the same way it does right now, I just had to > add in some handling for the wrapper in some parts of code. Great. Go ahead. Tristan -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From rvansa at redhat.com Thu Oct 27 11:00:36 2016 From: rvansa at redhat.com (Radim Vansa) Date: Thu, 27 Oct 2016 17:00:36 +0200 Subject: [infinispan-dev] Async API In-Reply-To: <4f282ea4-8e8b-8fe9-f8d9-268aba101d12@redhat.com> References: <57EA876D.6060300@redhat.com> <3AD09175-A621-41B0-B4F4-75C697FF57E7@redhat.com> <57FD1E72.2080201@redhat.com> <4f282ea4-8e8b-8fe9-f8d9-268aba101d12@redhat.com> Message-ID: <61beebd4-85ba-a5bc-7e3b-80a2fcb05157@redhat.com> On 10/27/2016 10:49 AM, Jonathan Halliday wrote: > On 27/10/16 09:36, Radim Vansa wrote: > >> So it's probably a question to Narayana guys if you can suspend a >> transaction in one thread and resume it in two threads. > yes you can. Where it gets hairy is ensuring all the threads are done > before you commit. By default we warn rather than block, though there is > a (non-standard) plugin api to register your own handler for it. Warn/block what exactly? Any thread could commit the tx, and it commits that for all threads - I understand that it can be complex on the user side, but how can TM know that there's a thread that's still working on the transaction (do you expect that all other threads than the committing one will suspend the transaction when these are done?). Btw., is there any API to just fork the tx? (rather than suspending and resuming) Can you just resume a non-suspended transaction acquired by tm.getTransaction() in another thread? Thanks! Radim > > Jonathan. > > -- Radim Vansa JBoss Performance Team From mudokonman at gmail.com Thu Oct 27 11:26:43 2016 From: mudokonman at gmail.com (William Burns) Date: Thu, 27 Oct 2016 15:26:43 +0000 Subject: [infinispan-dev] Data Container configuration changes In-Reply-To: References: <122bcbd2-bb36-815e-c69d-b19f3716362d@infinispan.org> Message-ID: On Thu, Oct 27, 2016 at 9:56 AM Pedro Ruivo wrote: > > > On 27-10-2016 14:20, William Burns wrote: > > > > > > On Thu, Oct 27, 2016 at 4:56 AM Pedro Ruivo > > wrote: > > > > > > > > On 26-10-2016 23:06, William Burns wrote: > > > I have been working on adding in off heap support for a given > > cache. I > > > wanted to check in and let you all know what I was thinking for the > > > configuration and changes that would come about with it. > > > > > > TLDR; > > > New config under data container to enable off heap, StoreAsBinary > > > removed, Equivalence removed > > > > > > First I was planning on adding new sub elements of data container. > > > These would be instance, binary and off-heap. Only of the three > could > > > be picked as they are mutually exclusive. Instance is as we > > operate now > > > where we store the instance of the object passed to us. Binary is > > > essentially what we have now that is called storeAsBinary with > > both keys > > > and values converted. Lastly off-heap would store the entry as a > > byte[] > > > store completely in native memory. > > > > I prefer 'object' instead of 'instance'. > > > > > > Sounds fine with me. > > > > > > > > Are you also planning to remove the expiration and/or eviction > > configuration elements and set them in the data-container sub > elements? > > > > > > I wasn't planning on touching those. But now that you mention it, > > eviction only makes sense in the data container, where as expiration is > > container and cache store. And taking this further storeAsBinary is > > both as well, only off-heap is container only. I wonder if instead we > > should have a separate storage element at the same level as > > data-container. I can see it either way. > > Let me know if this makes sense: > > //no changes here > > > //one of the following > > > > > > //no changes here > > wdyt? > Seems fine to me. And to be honest I forgot to mention this but EvictionStrategy and EvictionPolicy are completely redundant now. Policy has been for a while as we always used the same thread and Strategy is only Caffeine and for off heap I was thinking of a simple LRU. This means that the data container element would be removed in favor of "memory"? The reason being is that equivalence will be gone and afaik we never really supported a custom data container (eviction wouldn't work with it and neither would off heap). In that case why not just keep using data container element? > > > > > > > > > > > > > > Example: > > > > > > > > > > > > > > > > > > The reason it is a subelement instead of a property is because > > off-heap > > > will most likely require some additional configuration to tell how > > many > > > entries to store in the a bucket (think non resizing HashMap). > > > > > > With these changes storeAsBinary becomes redundant, so I was > > planning on > > > removing this configuration completely. I would rather remove > since > > > this is 9.0 and not deprecate. As far as I know nobody really > used it > > > before. > > > > > > Also another side effect is I was removing all of the Equivalence > > > classes. I am not sure if I can plainly remove them since they > have > > > lived in commons for quite a while, but it would be best if I > could, > > > although I am fine deprecating. In its place the instance setting > for > > > data-container will always wrap byte[] to satisfy equals and > hashCode > > > methods. > > > > > > Any feedback would be appreciated. > > > > > > Thanks, > > > - Will > > > > > > > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev at lists.jboss.org infinispan-dev at lists.jboss.org> > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org infinispan-dev at lists.jboss.org> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161027/d0c73bd2/attachment-0001.html From jonathan.halliday at redhat.com Thu Oct 27 11:32:31 2016 From: jonathan.halliday at redhat.com (Jonathan Halliday) Date: Thu, 27 Oct 2016 16:32:31 +0100 Subject: [infinispan-dev] Async API In-Reply-To: <61beebd4-85ba-a5bc-7e3b-80a2fcb05157@redhat.com> References: <57EA876D.6060300@redhat.com> <3AD09175-A621-41B0-B4F4-75C697FF57E7@redhat.com> <57FD1E72.2080201@redhat.com> <4f282ea4-8e8b-8fe9-f8d9-268aba101d12@redhat.com> <61beebd4-85ba-a5bc-7e3b-80a2fcb05157@redhat.com> Message-ID: On 27/10/16 16:00, Radim Vansa wrote: > On 10/27/2016 10:49 AM, Jonathan Halliday wrote: >> On 27/10/16 09:36, Radim Vansa wrote: >> >>> So it's probably a question to Narayana guys if you can suspend a >>> transaction in one thread and resume it in two threads. >> yes you can. Where it gets hairy is ensuring all the threads are done >> before you commit. By default we warn rather than block, though there is >> a (non-standard) plugin api to register your own handler for it. > > Warn/block what exactly? Any thread could commit the tx, and it commits > that for all threads - I understand that it can be complex on the user > side, but how can TM know that there's a thread that's still working on > the transaction (do you expect that all other threads than the > committing one will suspend the transaction when these are done?). If it is suspended then by definition it is not currently working on the tx, though it may not be done. We don't really care about those so much as we care about the ones that are NOT suspended, since nasty things happen to them if the tx context changes under their feet. e.g. JDBC connections that switch from jta-tx-enlisted to autonomous-autocommit without warning, which means the SQL running on them ain't going to do what the thread expects. We track which threads are active (i.e. associated, not suspended) in a tx and warn if you commit whilst it's greater than just the calling thread. But to prevent the aforementioned issue, warning is insufficient - you need to block the commit by waiting on the other thread(s) doing a tx suspend, or fail immediately with an exception. A variation on this applies even in the synchronous api, since the tx timeout/rollback is on a background thread, so a long running transaction can be killed and leave the single business logic thread in an unexpected tx context. libraries such as hibernate do their best to narrow the timing window, but it's still there. > Btw., is there any API to just fork the tx? (rather than suspending and > resuming) Can you just resume a non-suspended transaction acquired by > tm.getTransaction() in another thread? yes. Jonathan. > Thanks! > > Radim > >> >> Jonathan. >> >> > > -- Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From emmanuel at hibernate.org Fri Oct 28 11:49:41 2016 From: emmanuel at hibernate.org (Emmanuel Bernard) Date: Fri, 28 Oct 2016 17:49:41 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: References: Message-ID: <20161028154941.GK2991@hibernate.org> I like Ickle and LQID personally. And Adrian is way too reasonable on a name thread ;) On Mon 16-10-17 9:07, Tristan Tarrant wrote: >Hi all, > >something trivial and fun for a Monday morning. > >I've just issued a PR [1] to update the codename for Infinispan 9.0. > >And while we're at it, let's give a name to the new query language that >Adrian (and Emmanuel) have designed. We already have a number of >suggestions (which I summarize below) but please feel free to add your >own. Please vote. > >IQL (Infinispan Query Language, already used by others). >Ickle (Alternate pronunciation of above, also means "small") >LQID (Language for Querying Infinispan Datagrids) >QuIL ("Query Infinispan" Language) > >Tristan > >[1] https://github.com/infinispan/infinispan/pull/4617 >-- >Tristan Tarrant >Infinispan Lead >JBoss, a division of Red Hat >_______________________________________________ >infinispan-dev mailing list >infinispan-dev at lists.jboss.org >https://lists.jboss.org/mailman/listinfo/infinispan-dev From thomas.qvarnstrom at gmail.com Fri Oct 28 12:18:30 2016 From: thomas.qvarnstrom at gmail.com (=?utf-8?Q?Thomas_Qvarnstr=C3=B6m_Privat?=) Date: Fri, 28 Oct 2016 18:18:30 +0200 Subject: [infinispan-dev] Names, names, names... In-Reply-To: <20161028154941.GK2991@hibernate.org> References: <20161028154941.GK2991@hibernate.org> Message-ID: How about YAQL, Yet Another Query Language? > On 28 Oct 2016, at 17:49, Emmanuel Bernard wrote: > > I like Ickle and LQID personally. And Adrian is way too reasonable on a > name thread ;) > > On Mon 16-10-17 9:07, Tristan Tarrant wrote: >> Hi all, >> >> something trivial and fun for a Monday morning. >> >> I've just issued a PR [1] to update the codename for Infinispan 9.0. >> >> And while we're at it, let's give a name to the new query language that >> Adrian (and Emmanuel) have designed. We already have a number of >> suggestions (which I summarize below) but please feel free to add your >> own. Please vote. >> >> IQL (Infinispan Query Language, already used by others). >> Ickle (Alternate pronunciation of above, also means "small") >> LQID (Language for Querying Infinispan Datagrids) >> QuIL ("Query Infinispan" Language) >> >> Tristan >> >> [1] https://github.com/infinispan/infinispan/pull/4617 >> -- >> Tristan Tarrant >> Infinispan Lead >> JBoss, a division of Red Hat >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From rvansa at redhat.com Mon Oct 31 04:50:51 2016 From: rvansa at redhat.com (Radim Vansa) Date: Mon, 31 Oct 2016 09:50:51 +0100 Subject: [infinispan-dev] Data Container configuration changes In-Reply-To: References: <122bcbd2-bb36-815e-c69d-b19f3716362d@infinispan.org> Message-ID: On 10/27/2016 05:26 PM, William Burns wrote: > > > On Thu, Oct 27, 2016 at 9:56 AM Pedro Ruivo > wrote: > > > > On 27-10-2016 14:20, William Burns wrote: > > > > > > On Thu, Oct 27, 2016 at 4:56 AM Pedro Ruivo > > > >> wrote: > > > > > > > > On 26-10-2016 23:06, William Burns wrote: > > > I have been working on adding in off heap support for a given > > cache. I > > > wanted to check in and let you all know what I was > thinking for the > > > configuration and changes that would come about with it. > > > > > > TLDR; > > > New config under data container to enable off heap, > StoreAsBinary > > > removed, Equivalence removed > > > > > > First I was planning on adding new sub elements of data > container. > > > These would be instance, binary and off-heap. Only of the > three could > > > be picked as they are mutually exclusive. Instance is as we > > operate now > > > where we store the instance of the object passed to us. > Binary is > > > essentially what we have now that is called storeAsBinary with > > both keys > > > and values converted. Lastly off-heap would store the > entry as a > > byte[] > > > store completely in native memory. > > > > I prefer 'object' instead of 'instance'. > > > > > > Sounds fine with me. > > > > > > > > Are you also planning to remove the expiration and/or eviction > > configuration elements and set them in the data-container > sub elements? > > > > > > I wasn't planning on touching those. But now that you mention it, > > eviction only makes sense in the data container, where as > expiration is > > container and cache store. And taking this further storeAsBinary is > > both as well, only off-heap is container only. I wonder if > instead we > > should have a separate storage element at the same level as > > data-container. I can see it either way. > > Let me know if this makes sense: > > //no changes here > > > //one of the following > > > > > > //no changes here > > wdyt? > While I prefer "on-heap" instead of "object" or "instance", I don't think that "binary" should be its own element. Are there any attributes specific to that (do you plan to have keys="false" values="true"? I guess not). "on-heap" and "off-heap" is a binary ( :-) ) option, "on-heap-binary" is just a flavour of "on-heap". For comparison, HC uses OBJECT|BINARY|NATIVE where "NATIVE" means off-heap. I like "format= OBJECT|BINARY" as a child of on-heap, either as attribute or element. I haven't found similar settings in Coherence - seems that they store data in a serialized form only when they persist to disk/flash or offload to non-managed memory. Regarding Equivalence: can't we wrap objects in a similar way we do with byte[] if Equivalance (different from AnyInstance) is defined? I can definitely see use case when the hashCode() is not very well defined and user can't change the class - he has to wrap the objects themselves. R. > > Seems fine to me. And to be honest I forgot to mention this but > EvictionStrategy and EvictionPolicy are completely redundant now. > Policy has been for a while as we always used the same thread and > Strategy is only Caffeine and for off heap I was thinking of a simple LRU. > > This means that the data container element would be removed in favor > of "memory"? The reason being is that equivalence will be gone and > afaik we never really supported a custom data container (eviction > wouldn't work with it and neither would off heap). In that case why > not just keep using data container element? > > > > > > > > > > > > > > > Example: > > > > > > > > > > > > > > > > > > The reason it is a subelement instead of a property is because > > off-heap > > > will most likely require some additional configuration to > tell how > > many > > > entries to store in the a bucket (think non resizing HashMap). > > > > > > With these changes storeAsBinary becomes redundant, so I was > > planning on > > > removing this configuration completely. I would rather > remove since > > > this is 9.0 and not deprecate. As far as I know nobody > really used it > > > before. > > > > > > Also another side effect is I was removing all of the > Equivalence > > > classes. I am not sure if I can plainly remove them since > they have > > > lived in commons for quite a while, but it would be best > if I could, > > > although I am fine deprecating. In its place the instance > setting for > > > data-container will always wrap byte[] to satisfy equals > and hashCode > > > methods. > > > > > > Any feedback would be appreciated. > > > > > > Thanks, > > > - Will > > > > > > > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev at lists.jboss.org > > > > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From galder at redhat.com Mon Oct 31 11:43:12 2016 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Mon, 31 Oct 2016 16:43:12 +0100 Subject: [infinispan-dev] Weekly meeting minutes Message-ID: <3C9C9F5E-48DF-4452-8178-5AB7A63C0C9B@redhat.com> Meetings of our weekly developer meeting can be found here: http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2016/infinispan.2016-10-31-15.07.html Cheers, -- Galder Zamarre?o Infinispan, Red Hat From sanne at infinispan.org Mon Oct 31 14:24:22 2016 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 31 Oct 2016 18:24:22 +0000 Subject: [infinispan-dev] Weekly meeting minutes In-Reply-To: <3C9C9F5E-48DF-4452-8178-5AB7A63C0C9B@redhat.com> References: <3C9C9F5E-48DF-4452-8178-5AB7A63C0C9B@redhat.com> Message-ID: thanks for the meeting minutes, I see a question came up about expecting Hibernate Search to provide Externalizers for clustered queries. I'll start some thoughts about that in a separate thread; the short version: that won't happen but we can help. On 31 October 2016 at 15:43, Galder Zamarre?o wrote: > Meetings of our weekly developer meeting can be found here: > http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2016/infinispan.2016-10-31-15.07.html > > Cheers, > -- > Galder Zamarre?o > Infinispan, Red Hat > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From gustavo at infinispan.org Mon Oct 31 14:35:16 2016 From: gustavo at infinispan.org (Gustavo Fernandes) Date: Mon, 31 Oct 2016 18:35:16 +0000 Subject: [infinispan-dev] Weekly meeting minutes In-Reply-To: References: <3C9C9F5E-48DF-4452-8178-5AB7A63C0C9B@redhat.com> Message-ID: On Mon, Oct 31, 2016 at 6:24 PM, Sanne Grinovero wrote: > thanks for the meeting minutes, I see a question came up about > expecting Hibernate Search to provide Externalizers for clustered > queries. > AFAICT, it's Infinispan who will provide externalizers for Hibernate Search classes [1]. [1] https://issues.jboss.org/browse/ISPN-7156 > > I'll start some thoughts about that in a separate thread; the short > version: that won't happen but we can help. > > On 31 October 2016 at 15:43, Galder Zamarre?o wrote: > > Meetings of our weekly developer meeting can be found here: > > http://transcripts.jboss.org/meeting/irc.freenode.org/ > infinispan/2016/infinispan.2016-10-31-15.07.html > > > > Cheers, > > -- > > Galder Zamarre?o > > Infinispan, Red Hat > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161031/074a48d0/attachment-0001.html From sanne at infinispan.org Mon Oct 31 14:52:27 2016 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 31 Oct 2016 18:52:27 +0000 Subject: [infinispan-dev] Missing Externalizers for the Lucene Query classes Message-ID: Following up on today's meeting minutes. Galder asked if Hibernate Search was going to provide externalizers for the Lucene Query classes; let me clarify that I don't think that those belong in the Hibernate Search code base, not least we hope to avoid ever needing to implement them. A soft reason to not have them in Hibernate Search is that this project never needs to serialise any Query; this is a requirement of Infinispan Query only, needed to implement Infinispan specific extensions to the query engine. Although, these are very nice extensions so I'd not like to see them dropped: I'd hope that Infinispan could work around the lack of proper externalizers for the moment, as it has always been able to do so far. A stronger reason is that this would introduce circular dependencies between the two projects, and a big overhead of release coordination: we had this in the past, very all very glad this is in the past! When we'll have IQL, this will both define a good "on the wire" representation which would solve the serialization problem, and IQL will also limit the amount of Query types which we will need to support, as at that point we will be able to limit the support for Clustered Queries (which is the feature needing to serialize the queries) to those which IQL can express, and thus serialize. At that point we'll be able to deprecate the the Clustered Query API which accepts a user instance of the Lucene Query, and only run clustered queries for queries expressed over IQL. Not least we'll be able to automatically determine if the query is best run as a clustered query or as a local query, removing this complexity from the user's responsibilities. In conclusion, we'll still be using the "Clustered Query" functionality, but not exposing it, and by doing so we won't need any externalizer. But for now please keep the tests running with the existing externalizer strategies: we just need to keep it functional, but there's no need to optimise performance of these externalizers as we'll get rid of them. Thanks, Sanne