Hi David, Please find attach graphs belonging to two runs that compare: infinispan-4.0.0(repl-sync) - home grown marshalling layer infinispan-4.0.0(repl-sync-jbmar) - infinispan + jbmar 1.1.2 infinispan-4.0.0(repl-sync-jbmar)-rXYZ - infinispan + jbmar with revision Not sure what's the conclusion here tbh. The results of 1.1.2 almost look opposite in each test. I've also attached some information from previously run profiling sessions with a couple of local machines we have in Neuchatel. I profiled the faster of the two machines. Actually, looking at this profiled data, these tests are for synchronous replicated caches but I see no traces of actually reading the stream, only writing to it, hmmmm. I'm adding Externalizers to class table and implementing marshaller/unmarshaller pooling as my next tasks. Regards, David M. Lloyd wrote: > OK, I tried out a few things. You might want to try introducing these > one at a time (i.e. update up to rev 173, then 174, then 175 and see > how each one does). In particular, I think 175 has just as much > chance of slowing things down as speeding them up - either you're > getting tons of collisions in the hash table or the profiler is > skewing the results there (maybe try filtering out > org.jboss.marshalling.util.IdentityIntMap and java.lang.System to see > if that gives a different picture). > > I feel pretty good about 173 and 174 though I think the profiler will > skew 173 unless you have that UTFUtils filter installed. If 175 slows > things down (outside of the profiler), let me know and I'll revert > it. None of my tests showed much difference but I don't have any good > benchmarks that really exercise that code right now. > > There's a couple things left to try yet, like looking at replacing > ConcurrentReferenceHashMap (assuming that isn't the profiler again). > > ------------------------------------------------------------------------ > r175 | david.lloyd(a)jboss.com | 2009-05-08 00:17:46 -0500 (Fri, 08 May > 2009) | 1 line > > Try a trick to decrease the liklihood of collisions > ------------------------------------------------------------------------ > r174 | david.lloyd(a)jboss.com | 2009-05-08 00:04:39 -0500 (Fri, 08 May > 2009) | 1 line > > Replacement caching is not economical; the cost is one extra hash > table get for non-replaced objects, two hash table gets (total) for > replaced objects. Removing the cache gets rid of the cost for > non-replaced objects, while replaced objects now have to be replaced > again before the single hash table hit. > ------------------------------------------------------------------------ > r173 | david.lloyd(a)jboss.com | 2009-05-07 23:44:52 -0500 (Thu, 07 May > 2009) | 1 line > > JBMAR-52 - Avoid extra copy of char array (1.5 of 2) > ------------------------------------------------------------------------ > > - DML

David, A few questions about RiverMarshaller: 1. Why not make classCache and externalizers in RM live longer than the start/finish period? For example, if you had a create/destroy lifecycles that expanded for example Cache.start()/stop(), then classCache and externalizers will be shared by all writes and I think this could enhance performance. Wouldn't do this for instancesCache since it would blow out. 2. Looking at the profiling data, I'm slightly concerned about writeReplace() code are in the for(;;) loop in RM. I mean, writeReplace only makes sense for Serializable classes but this section of code is being executed a lot, much more than the number of Serializable classes are in the tests. I just a ran quick test and for a String, this section of the code is executed but why do so when you know for sure that a String that is final does not have and cannot have writeReplace()? I think section should go right to the bottom to the obj instanceof Serializable section or somewhere where is less in the critical path. Regards, Galder Zamarreno wrote: > Profiler data with JBMAR r174 > > Galder Zamarreno wrote: >> Hi David, >> >> Please find attach graphs belonging to two runs that compare: >> >> infinispan-4.0.0(repl-sync) - home grown marshalling layer >> infinispan-4.0.0(repl-sync-jbmar) - infinispan + jbmar 1.1.2 >> infinispan-4.0.0(repl-sync-jbmar)-rXYZ - infinispan + jbmar with >> revision >> >> Not sure what's the conclusion here tbh. The results of 1.1.2 almost >> look opposite in each test. >> >> I've also attached some information from previously run profiling >> sessions with a couple of local machines we have in Neuchatel. I >> profiled the faster of the two machines. >> >> Actually, looking at this profiled data, these tests are for >> synchronous replicated caches but I see no traces of actually reading >> the stream, only writing to it, hmmmm. >> >> I'm adding Externalizers to class table and implementing >> marshaller/unmarshaller pooling as my next tasks. >> >> Regards, >> >> David M. Lloyd wrote: >>> OK, I tried out a few things. You might want to try introducing >>> these one at a time (i.e. update up to rev 173, then 174, then 175 >>> and see how each one does). In particular, I think 175 has just as >>> much chance of slowing things down as speeding them up - either >>> you're getting tons of collisions in the hash table or the profiler >>> is skewing the results there (maybe try filtering out >>> org.jboss.marshalling.util.IdentityIntMap and java.lang.System to >>> see if that gives a different picture). >>> >>> I feel pretty good about 173 and 174 though I think the profiler >>> will skew 173 unless you have that UTFUtils filter installed. If >>> 175 slows things down (outside of the profiler), let me know and >>> I'll revert it. None of my tests showed much difference but I don't >>> have any good benchmarks that really exercise that code right now. >>> >>> There's a couple things left to try yet, like looking at replacing >>> ConcurrentReferenceHashMap (assuming that isn't the profiler again). >>> >>> ------------------------------------------------------------------------ >>> >>> r175 | david.lloyd(a)jboss.com | 2009-05-08 00:17:46 -0500 (Fri, 08 >>> May 2009) | 1 line >>> >>> Try a trick to decrease the liklihood of collisions >>> ------------------------------------------------------------------------ >>> >>> r174 | david.lloyd(a)jboss.com | 2009-05-08 00:04:39 -0500 (Fri, 08 >>> May 2009) | 1 line >>> >>> Replacement caching is not economical; the cost is one extra hash >>> table get for non-replaced objects, two hash table gets (total) for >>> replaced objects. Removing the cache gets rid of the cost for >>> non-replaced objects, while replaced objects now have to be replaced >>> again before the single hash table hit. >>> ------------------------------------------------------------------------ >>> >>> r173 | david.lloyd(a)jboss.com | 2009-05-07 23:44:52 -0500 (Thu, 07 >>> May 2009) | 1 line >>> >>> JBMAR-52 - Avoid extra copy of char array (1.5 of 2) >>> ------------------------------------------------------------------------ >>> >>> >>> - DML >> >

Galder, would you mind switching to trunk and trying that version (unofficial 1.2.0.CR1 - yes I *promise* I'll fix the versioning and get a source blob built asap!)? - DML On 05/08/2009 12:09 PM, Galder Zamarreno wrote: > Profiler data with JBMAR r174 > > Galder Zamarreno wrote: >> Hi David, >> >> Please find attach graphs belonging to two runs that compare: >> >> infinispan-4.0.0(repl-sync) - home grown marshalling layer >> infinispan-4.0.0(repl-sync-jbmar) - infinispan + jbmar 1.1.2 >> infinispan-4.0.0(repl-sync-jbmar)-rXYZ - infinispan + jbmar with >> revision >> >> Not sure what's the conclusion here tbh. The results of 1.1.2 almost >> look opposite in each test. >> >> I've also attached some information from previously run profiling >> sessions with a couple of local machines we have in Neuchatel. I >> profiled the faster of the two machines. >> >> Actually, looking at this profiled data, these tests are for >> synchronous replicated caches but I see no traces of actually reading >> the stream, only writing to it, hmmmm. >> >> I'm adding Externalizers to class table and implementing >> marshaller/unmarshaller pooling as my next tasks. >> >> Regards, >> >> David M. Lloyd wrote: >>> OK, I tried out a few things. You might want to try introducing >>> these one at a time (i.e. update up to rev 173, then 174, then 175 >>> and see how each one does). In particular, I think 175 has just as >>> much chance of slowing things down as speeding them up - either >>> you're getting tons of collisions in the hash table or the profiler >>> is skewing the results there (maybe try filtering out >>> org.jboss.marshalling.util.IdentityIntMap and java.lang.System to >>> see if that gives a different picture). >>> >>> I feel pretty good about 173 and 174 though I think the profiler >>> will skew 173 unless you have that UTFUtils filter installed. If >>> 175 slows things down (outside of the profiler), let me know and >>> I'll revert it. None of my tests showed much difference but I don't >>> have any good benchmarks that really exercise that code right now. >>> >>> There's a couple things left to try yet, like looking at replacing >>> ConcurrentReferenceHashMap (assuming that isn't the profiler again). >>> >>> ------------------------------------------------------------------------ >>> >>> r175 | david.lloyd(a)jboss.com | 2009-05-08 00:17:46 -0500 (Fri, 08 >>> May 2009) | 1 line >>> >>> Try a trick to decrease the liklihood of collisions >>> ------------------------------------------------------------------------ >>> >>> r174 | david.lloyd(a)jboss.com | 2009-05-08 00:04:39 -0500 (Fri, 08 >>> May 2009) | 1 line >>> >>> Replacement caching is not economical; the cost is one extra hash >>> table get for non-replaced objects, two hash table gets (total) for >>> replaced objects. Removing the cache gets rid of the cost for >>> non-replaced objects, while replaced objects now have to be replaced >>> again before the single hash table hit. >>> ------------------------------------------------------------------------ >>> >>> r173 | david.lloyd(a)jboss.com | 2009-05-07 23:44:52 -0500 (Thu, 07 >>> May 2009) | 1 line >>> >>> JBMAR-52 - Avoid extra copy of char array (1.5 of 2) >>> ------------------------------------------------------------------------ >>> >>> >>> - DML >> >

OK, then I think there must be something outside of JBMAR that is impacting this. I've gone through and literally removed a number of the top hotspots from the profiler reports which should have made a big difference if the reports were correct. Is there any way I can get ahold of your whole test case?

- DML On 05/11/2009 11:06 AM, Galder Zamarreno wrote: > Hey, > > Run a couple of round of tests with JBMAR trunk (r200) and please find > attached the results. No real improvement I'm afraid. I'll look into > the profile data from r200 tomorrow. > > I'm currently working on the pooling. > > Regards, > > David M. Lloyd wrote: >> Galder, would you mind switching to trunk and trying that version >> (unofficial 1.2.0.CR1 - yes I *promise* I'll fix the versioning and >> get a source blob built asap!)? >> >> - DML >> >> On 05/08/2009 12:09 PM, Galder Zamarreno wrote: >>> Profiler data with JBMAR r174 >>> >>> Galder Zamarreno wrote: >>>> Hi David, >>>> >>>> Please find attach graphs belonging to two runs that compare: >>>> >>>> infinispan-4.0.0(repl-sync) - home grown marshalling layer >>>> infinispan-4.0.0(repl-sync-jbmar) - infinispan + jbmar 1.1.2 >>>> infinispan-4.0.0(repl-sync-jbmar)-rXYZ - infinispan + jbmar with >>>> revision >>>> >>>> Not sure what's the conclusion here tbh. The results of 1.1.2 >>>> almost look opposite in each test. >>>> >>>> I've also attached some information from previously run profiling >>>> sessions with a couple of local machines we have in Neuchatel. I >>>> profiled the faster of the two machines. >>>> >>>> Actually, looking at this profiled data, these tests are for >>>> synchronous replicated caches but I see no traces of actually >>>> reading the stream, only writing to it, hmmmm. >>>> >>>> I'm adding Externalizers to class table and implementing >>>> marshaller/unmarshaller pooling as my next tasks. >>>> >>>> Regards, >>>> >>>> David M. Lloyd wrote: >>>>> OK, I tried out a few things. You might want to try introducing >>>>> these one at a time (i.e. update up to rev 173, then 174, then 175 >>>>> and see how each one does). In particular, I think 175 has just >>>>> as much chance of slowing things down as speeding them up - either >>>>> you're getting tons of collisions in the hash table or the >>>>> profiler is skewing the results there (maybe try filtering out >>>>> org.jboss.marshalling.util.IdentityIntMap and java.lang.System to >>>>> see if that gives a different picture). >>>>> >>>>> I feel pretty good about 173 and 174 though I think the profiler >>>>> will skew 173 unless you have that UTFUtils filter installed. If >>>>> 175 slows things down (outside of the profiler), let me know and >>>>> I'll revert it. None of my tests showed much difference but I >>>>> don't have any good benchmarks that really exercise that code >>>>> right now. >>>>> >>>>> There's a couple things left to try yet, like looking at replacing >>>>> ConcurrentReferenceHashMap (assuming that isn't the profiler again). >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> r175 | david.lloyd(a)jboss.com | 2009-05-08 00:17:46 -0500 (Fri, 08 >>>>> May 2009) | 1 line >>>>> >>>>> Try a trick to decrease the liklihood of collisions >>>>> ------------------------------------------------------------------------ >>>>> >>>>> r174 | david.lloyd(a)jboss.com | 2009-05-08 00:04:39 -0500 (Fri, 08 >>>>> May 2009) | 1 line >>>>> >>>>> Replacement caching is not economical; the cost is one extra hash >>>>> table get for non-replaced objects, two hash table gets (total) >>>>> for replaced objects. Removing the cache gets rid of the cost for >>>>> non-replaced objects, while replaced objects now have to be >>>>> replaced again before the single hash table hit. >>>>> ------------------------------------------------------------------------ >>>>> >>>>> r173 | david.lloyd(a)jboss.com | 2009-05-07 23:44:52 -0500 (Thu, 07 >>>>> May 2009) | 1 line >>>>> >>>>> JBMAR-52 - Avoid extra copy of char array (1.5 of 2) >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> - DML >>>> >>> > > > ------------------------------------------------------------------------ > > > ------------------------------------------------------------------------ > > > ------------------------------------------------------------------------ > > > ------------------------------------------------------------------------ >

Galder Zamarreno wrote: > > > David M. Lloyd wrote: >> OK, then I think there must be something outside of JBMAR that is >> impacting this. I've gone through and literally removed a number of >> the top hotspots from the profiler reports which should have made a >> big difference if the reports were correct. >> >> Is there any way I can get ahold of your whole test case? > > I'm a real ^%%$£%: > > 1.- In the cluster tests I had run, the nodes never joined and hence > never formed a cluster, so each node run independently! > Todo: improve benchmark framework so that if a cluster test is being > run with multiple nodes, the cluster view does indeed have that number > of nodes!! > > 2.- More importantly, all along I've been ignoring > cache-products/xyz/config.sh and hence when updating JBMAR, I was > simply copying the folder, without updating config.sh. Result? All the > JBMAR tests have been running with the very 1st JBMAR version tested. Todo: cache-products/xyz/config.sh should not have the directory hard-coded, it should retrieve the directory name of pwd or something similar. > > Once again, I'm a real ^%%$£%^%*&%*£!! > > I'm working to fix these two issues asap and will come back with > information asap. > >> >> - DML >> >> On 05/11/2009 11:06 AM, Galder Zamarreno wrote: >>> Hey, >>> >>> Run a couple of round of tests with JBMAR trunk (r200) and please >>> find attached the results. No real improvement I'm afraid. I'll look >>> into the profile data from r200 tomorrow. >>> >>> I'm currently working on the pooling. >>> >>> Regards, >>> >>> David M. Lloyd wrote: >>>> Galder, would you mind switching to trunk and trying that version >>>> (unofficial 1.2.0.CR1 - yes I *promise* I'll fix the versioning and >>>> get a source blob built asap!)? >>>> >>>> - DML >>>> >>>> On 05/08/2009 12:09 PM, Galder Zamarreno wrote: >>>>> Profiler data with JBMAR r174 >>>>> >>>>> Galder Zamarreno wrote: >>>>>> Hi David, >>>>>> >>>>>> Please find attach graphs belonging to two runs that compare: >>>>>> >>>>>> infinispan-4.0.0(repl-sync) - home grown marshalling layer >>>>>> infinispan-4.0.0(repl-sync-jbmar) - infinispan + jbmar 1.1.2 >>>>>> infinispan-4.0.0(repl-sync-jbmar)-rXYZ - infinispan + jbmar with >>>>>> revision >>>>>> >>>>>> Not sure what's the conclusion here tbh. The results of 1.1.2 >>>>>> almost look opposite in each test. >>>>>> >>>>>> I've also attached some information from previously run profiling >>>>>> sessions with a couple of local machines we have in Neuchatel. I >>>>>> profiled the faster of the two machines. >>>>>> >>>>>> Actually, looking at this profiled data, these tests are for >>>>>> synchronous replicated caches but I see no traces of actually >>>>>> reading the stream, only writing to it, hmmmm. >>>>>> >>>>>> I'm adding Externalizers to class table and implementing >>>>>> marshaller/unmarshaller pooling as my next tasks. >>>>>> >>>>>> Regards, >>>>>> >>>>>> David M. Lloyd wrote: >>>>>>> OK, I tried out a few things. You might want to try introducing >>>>>>> these one at a time (i.e. update up to rev 173, then 174, then >>>>>>> 175 and see how each one does). In particular, I think 175 has >>>>>>> just as much chance of slowing things down as speeding them up - >>>>>>> either you're getting tons of collisions in the hash table or >>>>>>> the profiler is skewing the results there (maybe try filtering >>>>>>> out org.jboss.marshalling.util.IdentityIntMap and >>>>>>> java.lang.System to see if that gives a different picture). >>>>>>> >>>>>>> I feel pretty good about 173 and 174 though I think the profiler >>>>>>> will skew 173 unless you have that UTFUtils filter installed. >>>>>>> If 175 slows things down (outside of the profiler), let me know >>>>>>> and I'll revert it. None of my tests showed much difference but >>>>>>> I don't have any good benchmarks that really exercise that code >>>>>>> right now. >>>>>>> >>>>>>> There's a couple things left to try yet, like looking at >>>>>>> replacing ConcurrentReferenceHashMap (assuming that isn't the >>>>>>> profiler again). >>>>>>> >>>>>>> ------------------------------------------------------------------------ >>>>>>> >>>>>>> r175 | david.lloyd(a)jboss.com | 2009-05-08 00:17:46 -0500 (Fri, >>>>>>> 08 May 2009) | 1 line >>>>>>> >>>>>>> Try a trick to decrease the liklihood of collisions >>>>>>> ------------------------------------------------------------------------ >>>>>>> >>>>>>> r174 | david.lloyd(a)jboss.com | 2009-05-08 00:04:39 -0500 (Fri, >>>>>>> 08 May 2009) | 1 line >>>>>>> >>>>>>> Replacement caching is not economical; the cost is one extra >>>>>>> hash table get for non-replaced objects, two hash table gets >>>>>>> (total) for replaced objects. Removing the cache gets rid of >>>>>>> the cost for non-replaced objects, while replaced objects now >>>>>>> have to be replaced again before the single hash table hit. >>>>>>> ------------------------------------------------------------------------ >>>>>>> >>>>>>> r173 | david.lloyd(a)jboss.com | 2009-05-07 23:44:52 -0500 (Thu, >>>>>>> 07 May 2009) | 1 line >>>>>>> >>>>>>> JBMAR-52 - Avoid extra copy of char array (1.5 of 2) >>>>>>> ------------------------------------------------------------------------ >>>>>>> >>>>>>> >>>>>>> - DML >>>>>> >>>>> >>> >>> >>> ------------------------------------------------------------------------ >>> >>> >>> >>> ------------------------------------------------------------------------ >>> >>> >>> >>> ------------------------------------------------------------------------ >>> >>> >>> >>> ------------------------------------------------------------------------ >>> >>> >

I've managed to get tests run making sure that the right JBMAR versions and the clusters are formed correctly. JBMAR looks faster is most cases of async communications already. It's worth noting that with sync, the differences are minimal, network and waiting for responses is probably a bigger latency than what u can gain with marshalling. I'll run another batch of async tests to see whether it's clearer which JBMAR version I should go for. Note that in async mode and 8 nodes, I started to see some view changes under load and also some NAKACK errors. I'll move now to implement pooling and improvement explained in https://jira.jboss.org/jira/browse/ISPN-59. Regards, Galder Zamarreno wrote: > > > Galder Zamarreno wrote: >> >> >> David M. Lloyd wrote: >>> OK, then I think there must be something outside of JBMAR that is >>> impacting this. I've gone through and literally removed a number of >>> the top hotspots from the profiler reports which should have made a >>> big difference if the reports were correct. >>> >>> Is there any way I can get ahold of your whole test case? >> >> I'm a real ^%%$£%: >> >> 1.- In the cluster tests I had run, the nodes never joined and hence >> never formed a cluster, so each node run independently! >> Todo: improve benchmark framework so that if a cluster test is being >> run with multiple nodes, the cluster view does indeed have that >> number of nodes!! >> >> 2.- More importantly, all along I've been ignoring >> cache-products/xyz/config.sh and hence when updating JBMAR, I was >> simply copying the folder, without updating config.sh. Result? All >> the JBMAR tests have been running with the very 1st JBMAR version >> tested. > > Todo: cache-products/xyz/config.sh should not have the directory > hard-coded, it should retrieve the directory name of pwd or something > similar. > >> >> Once again, I'm a real ^%%$£%^%*&%*£!! >> >> I'm working to fix these two issues asap and will come back with >> information asap. >> >>> >>> - DML >>> >>> On 05/11/2009 11:06 AM, Galder Zamarreno wrote: >>>> Hey, >>>> >>>> Run a couple of round of tests with JBMAR trunk (r200) and please >>>> find attached the results. No real improvement I'm afraid. I'll >>>> look into the profile data from r200 tomorrow. >>>> >>>> I'm currently working on the pooling. >>>> >>>> Regards, >>>> >>>> David M. Lloyd wrote: >>>>> Galder, would you mind switching to trunk and trying that version >>>>> (unofficial 1.2.0.CR1 - yes I *promise* I'll fix the versioning >>>>> and get a source blob built asap!)? >>>>> >>>>> - DML >>>>> >>>>> On 05/08/2009 12:09 PM, Galder Zamarreno wrote: >>>>>> Profiler data with JBMAR r174 >>>>>> >>>>>> Galder Zamarreno wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> Please find attach graphs belonging to two runs that compare: >>>>>>> >>>>>>> infinispan-4.0.0(repl-sync) - home grown marshalling layer >>>>>>> infinispan-4.0.0(repl-sync-jbmar) - infinispan + jbmar 1.1.2 >>>>>>> infinispan-4.0.0(repl-sync-jbmar)-rXYZ - infinispan + jbmar with >>>>>>> revision >>>>>>> >>>>>>> Not sure what's the conclusion here tbh. The results of 1.1.2 >>>>>>> almost look opposite in each test. >>>>>>> >>>>>>> I've also attached some information from previously run >>>>>>> profiling sessions with a couple of local machines we have in >>>>>>> Neuchatel. I profiled the faster of the two machines. >>>>>>> >>>>>>> Actually, looking at this profiled data, these tests are for >>>>>>> synchronous replicated caches but I see no traces of actually >>>>>>> reading the stream, only writing to it, hmmmm. >>>>>>> >>>>>>> I'm adding Externalizers to class table and implementing >>>>>>> marshaller/unmarshaller pooling as my next tasks. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> David M. Lloyd wrote: >>>>>>>> OK, I tried out a few things. You might want to try >>>>>>>> introducing these one at a time (i.e. update up to rev 173, >>>>>>>> then 174, then 175 and see how each one does). In particular, >>>>>>>> I think 175 has just as much chance of slowing things down as >>>>>>>> speeding them up - either you're getting tons of collisions in >>>>>>>> the hash table or the profiler is skewing the results there >>>>>>>> (maybe try filtering out >>>>>>>> org.jboss.marshalling.util.IdentityIntMap and java.lang.System >>>>>>>> to see if that gives a different picture). >>>>>>>> >>>>>>>> I feel pretty good about 173 and 174 though I think the >>>>>>>> profiler will skew 173 unless you have that UTFUtils filter >>>>>>>> installed. If 175 slows things down (outside of the profiler), >>>>>>>> let me know and I'll revert it. None of my tests showed much >>>>>>>> difference but I don't have any good benchmarks that really >>>>>>>> exercise that code right now. >>>>>>>> >>>>>>>> There's a couple things left to try yet, like looking at >>>>>>>> replacing ConcurrentReferenceHashMap (assuming that isn't the >>>>>>>> profiler again). >>>>>>>> >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> >>>>>>>> r175 | david.lloyd(a)jboss.com | 2009-05-08 00:17:46 -0500 (Fri, >>>>>>>> 08 May 2009) | 1 line >>>>>>>> >>>>>>>> Try a trick to decrease the liklihood of collisions >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> >>>>>>>> r174 | david.lloyd(a)jboss.com | 2009-05-08 00:04:39 -0500 (Fri, >>>>>>>> 08 May 2009) | 1 line >>>>>>>> >>>>>>>> Replacement caching is not economical; the cost is one extra >>>>>>>> hash table get for non-replaced objects, two hash table gets >>>>>>>> (total) for replaced objects. Removing the cache gets rid of >>>>>>>> the cost for non-replaced objects, while replaced objects now >>>>>>>> have to be replaced again before the single hash table hit. >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> >>>>>>>> r173 | david.lloyd(a)jboss.com | 2009-05-07 23:44:52 -0500 (Thu, >>>>>>>> 07 May 2009) | 1 line >>>>>>>> >>>>>>>> JBMAR-52 - Avoid extra copy of char array (1.5 of 2) >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> >>>>>>>> >>>>>>>> - DML >>>>>>> >>>>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >> > ------------------------------------------------------------------------ ------------------------------------------------------------------------ ------------------------------------------------------------------------ ------------------------------------------------------------------------

Are we also testing that the marshalled data is no bigger in size than the current marshalling ? Maybe a unit test, e.g. to verify that a Boolean gets marshalled to 1 or 2 bytes and not 22 like with ObjectOutputStream.writeObject() ! Galder Zamarreno wrote: > I've managed to get tests run making sure that the right JBMAR > versions and the clusters are formed correctly. > > JBMAR looks faster is most cases of async communications already. It's > worth noting that with sync, the differences are minimal, network and > waiting for responses is probably a bigger latency than what u can > gain with marshalling. > > I'll run another batch of async tests to see whether it's clearer > which JBMAR version I should go for. > > Note that in async mode and 8 nodes, I started to see some view > changes under load and also some NAKACK errors. > > I'll move now to implement pooling and improvement explained in > https://jira.jboss.org/jira/browse/ISPN-59. > > Regards, > > Galder Zamarreno wrote: >> >> >> Galder Zamarreno wrote: >>> >>> >>> David M. Lloyd wrote: >>>> OK, then I think there must be something outside of JBMAR that is >>>> impacting this. I've gone through and literally removed a number >>>> of the top hotspots from the profiler reports which should have >>>> made a big difference if the reports were correct. >>>> >>>> Is there any way I can get ahold of your whole test case? >>> >>> I'm a real ^%%$£%: >>> >>> 1.- In the cluster tests I had run, the nodes never joined and hence >>> never formed a cluster, so each node run independently! >>> Todo: improve benchmark framework so that if a cluster test is being >>> run with multiple nodes, the cluster view does indeed have that >>> number of nodes!! >>> >>> 2.- More importantly, all along I've been ignoring >>> cache-products/xyz/config.sh and hence when updating JBMAR, I was >>> simply copying the folder, without updating config.sh. Result? All >>> the JBMAR tests have been running with the very 1st JBMAR version >>> tested. >> >> Todo: cache-products/xyz/config.sh should not have the directory >> hard-coded, it should retrieve the directory name of pwd or something >> similar. >> >>> >>> Once again, I'm a real ^%%$£%^%*&%*£!! >>> >>> I'm working to fix these two issues asap and will come back with >>> information asap. >>> >>>> >>>> - DML >>>> >>>> On 05/11/2009 11:06 AM, Galder Zamarreno wrote: >>>>> Hey, >>>>> >>>>> Run a couple of round of tests with JBMAR trunk (r200) and please >>>>> find attached the results. No real improvement I'm afraid. I'll >>>>> look into the profile data from r200 tomorrow. >>>>> >>>>> I'm currently working on the pooling. >>>>> >>>>> Regards, >>>>> >>>>> David M. Lloyd wrote: >>>>>> Galder, would you mind switching to trunk and trying that version >>>>>> (unofficial 1.2.0.CR1 - yes I *promise* I'll fix the versioning >>>>>> and get a source blob built asap!)? >>>>>> >>>>>> - DML >>>>>> >>>>>> On 05/08/2009 12:09 PM, Galder Zamarreno wrote: >>>>>>> Profiler data with JBMAR r174 >>>>>>> >>>>>>> Galder Zamarreno wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> Please find attach graphs belonging to two runs that compare: >>>>>>>> >>>>>>>> infinispan-4.0.0(repl-sync) - home grown marshalling layer >>>>>>>> infinispan-4.0.0(repl-sync-jbmar) - infinispan + jbmar 1.1.2 >>>>>>>> infinispan-4.0.0(repl-sync-jbmar)-rXYZ - infinispan + jbmar >>>>>>>> with revision >>>>>>>> >>>>>>>> Not sure what's the conclusion here tbh. The results of 1.1.2 >>>>>>>> almost look opposite in each test. >>>>>>>> >>>>>>>> I've also attached some information from previously run >>>>>>>> profiling sessions with a couple of local machines we have in >>>>>>>> Neuchatel. I profiled the faster of the two machines. >>>>>>>> >>>>>>>> Actually, looking at this profiled data, these tests are for >>>>>>>> synchronous replicated caches but I see no traces of actually >>>>>>>> reading the stream, only writing to it, hmmmm. >>>>>>>> >>>>>>>> I'm adding Externalizers to class table and implementing >>>>>>>> marshaller/unmarshaller pooling as my next tasks. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> David M. Lloyd wrote: >>>>>>>>> OK, I tried out a few things. You might want to try >>>>>>>>> introducing these one at a time (i.e. update up to rev 173, >>>>>>>>> then 174, then 175 and see how each one does). In particular, >>>>>>>>> I think 175 has just as much chance of slowing things down as >>>>>>>>> speeding them up - either you're getting tons of collisions in >>>>>>>>> the hash table or the profiler is skewing the results there >>>>>>>>> (maybe try filtering out >>>>>>>>> org.jboss.marshalling.util.IdentityIntMap and java.lang.System >>>>>>>>> to see if that gives a different picture). >>>>>>>>> >>>>>>>>> I feel pretty good about 173 and 174 though I think the >>>>>>>>> profiler will skew 173 unless you have that UTFUtils filter >>>>>>>>> installed. If 175 slows things down (outside of the >>>>>>>>> profiler), let me know and I'll revert it. None of my tests >>>>>>>>> showed much difference but I don't have any good benchmarks >>>>>>>>> that really exercise that code right now. >>>>>>>>> >>>>>>>>> There's a couple things left to try yet, like looking at >>>>>>>>> replacing ConcurrentReferenceHashMap (assuming that isn't the >>>>>>>>> profiler again). >>>>>>>>> >>>>>>>>> ------------------------------------------------------------------------ >>>>>>>>> >>>>>>>>> r175 | david.lloyd(a)jboss.com | 2009-05-08 00:17:46 -0500 (Fri, >>>>>>>>> 08 May 2009) | 1 line >>>>>>>>> >>>>>>>>> Try a trick to decrease the liklihood of collisions >>>>>>>>> ------------------------------------------------------------------------ >>>>>>>>> >>>>>>>>> r174 | david.lloyd(a)jboss.com | 2009-05-08 00:04:39 -0500 (Fri, >>>>>>>>> 08 May 2009) | 1 line >>>>>>>>> >>>>>>>>> Replacement caching is not economical; the cost is one extra >>>>>>>>> hash table get for non-replaced objects, two hash table gets >>>>>>>>> (total) for replaced objects. Removing the cache gets rid of >>>>>>>>> the cost for non-replaced objects, while replaced objects now >>>>>>>>> have to be replaced again before the single hash table hit. >>>>>>>>> ------------------------------------------------------------------------ >>>>>>>>> >>>>>>>>> r173 | david.lloyd(a)jboss.com | 2009-05-07 23:44:52 -0500 (Thu, >>>>>>>>> 07 May 2009) | 1 line >>>>>>>>> >>>>>>>>> JBMAR-52 - Avoid extra copy of char array (1.5 of 2) >>>>>>>>> ------------------------------------------------------------------------ >>>>>>>>> >>>>>>>>> >>>>>>>>> - DML >>>>>>>> >>>>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>> >> > > > ------------------------------------------------------------------------ > > > ------------------------------------------------------------------------ > > > ------------------------------------------------------------------------ > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------ > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 13 May 2009, at 21:28, David M. Lloyd wrote: > - This object is predefined = 1 byte > - This is the custom ID of the predefined object = 1 byte > - This is the custom object data = N bytes Can't the first 2 be combined into a single byte? That's what I do in my hand-wired marshaller. I assume that all objects are predefined, and use a special, reserved 'custom ID' to denote that the object is *not* predefined. Basically, I serialize a boolean into 2 bytes.

Yes, Galder has such a test. :-) And yes, JBoss Marshalling writes booleans as a single byte

(or, in the case of boolean[], a single bit).

The test won't likely pass fully until Galder reimplements the custom marshalling parts as ObjectTable instead of Externalizers. With the current Externalizer-based design, it might write: - This is an externalized object = 1 byte - The class of the externalized object is in the class table = 1 byte - The class table entry for the externalized class = 1 byte - The externalizer object is predefined = 1 byte - This is the custom ID of the externalizer object = 1 byte - Begin custom object data = 1 byte - Custom object data length = 1-4 bytes depending on N - This is the custom object data = N bytes - End custom data = 1 byte Giving 8+N to 11+N bytes or more, depending on the mix of objects and primitive data in the custom data block, due to the security features in Externalizers which are not needed in ISPN. With an object table it would write: - This object is predefined = 1 byte - This is the custom ID of the predefined object = 1 byte - This is the custom object data = N bytes