[infinispan-dev] Re: JProfiler snapshots for Infinispan+JBMAR
David M. Lloyd
david.lloyd at redhat.com
Wed May 13 16:28:57 EDT 2009
Yes, Galder has such a test. :-) And yes, JBoss Marshalling writes
booleans as a single byte (or, in the case of boolean[], a single bit).
The test won't likely pass fully until Galder reimplements the custom
marshalling parts as ObjectTable instead of Externalizers. With the
current Externalizer-based design, it might write:
- This is an externalized object = 1 byte
- The class of the externalized object is in the class table = 1 byte
- The class table entry for the externalized class = 1 byte
- The externalizer object is predefined = 1 byte
- This is the custom ID of the externalizer object = 1 byte
- Begin custom object data = 1 byte
- Custom object data length = 1-4 bytes depending on N
- This is the custom object data = N bytes
- End custom data = 1 byte
Giving 8+N to 11+N bytes or more, depending on the mix of objects and
primitive data in the custom data block, due to the security features in
Externalizers which are not needed in ISPN. With an object table it would
write:
- This object is predefined = 1 byte
- This is the custom ID of the predefined object = 1 byte
- This is the custom object data = N bytes
Giving a constant 2+N bytes for custom objects with no additional overhead
for regular serialized or externalized user objects. This is what we're
talking about in that IRC transcript in ISPN-59.
- DML
On 05/13/2009 02:56 PM, Bela Ban wrote:
> Are we also testing that the marshalled data is no bigger in size than
> the current marshalling ?
>
> Maybe a unit test, e.g. to verify that a Boolean gets marshalled to 1 or
> 2 bytes and not 22 like with ObjectOutputStream.writeObject() !
>
> Galder Zamarreno wrote:
>> I've managed to get tests run making sure that the right JBMAR
>> versions and the clusters are formed correctly.
>>
>> JBMAR looks faster is most cases of async communications already. It's
>> worth noting that with sync, the differences are minimal, network and
>> waiting for responses is probably a bigger latency than what u can
>> gain with marshalling.
>>
>> I'll run another batch of async tests to see whether it's clearer
>> which JBMAR version I should go for.
>>
>> Note that in async mode and 8 nodes, I started to see some view
>> changes under load and also some NAKACK errors.
>>
>> I'll move now to implement pooling and improvement explained in
>> https://jira.jboss.org/jira/browse/ISPN-59.
>>
>> Regards,
>>
>> Galder Zamarreno wrote:
>>>
>>>
>>> Galder Zamarreno wrote:
>>>>
>>>>
>>>> David M. Lloyd wrote:
>>>>> OK, then I think there must be something outside of JBMAR that is
>>>>> impacting this. I've gone through and literally removed a number
>>>>> of the top hotspots from the profiler reports which should have
>>>>> made a big difference if the reports were correct.
>>>>>
>>>>> Is there any way I can get ahold of your whole test case?
>>>>
>>>> I'm a real ^%%$£%:
>>>>
>>>> 1.- In the cluster tests I had run, the nodes never joined and hence
>>>> never formed a cluster, so each node run independently!
>>>> Todo: improve benchmark framework so that if a cluster test is being
>>>> run with multiple nodes, the cluster view does indeed have that
>>>> number of nodes!!
>>>>
>>>> 2.- More importantly, all along I've been ignoring
>>>> cache-products/xyz/config.sh and hence when updating JBMAR, I was
>>>> simply copying the folder, without updating config.sh. Result? All
>>>> the JBMAR tests have been running with the very 1st JBMAR version
>>>> tested.
>>>
>>> Todo: cache-products/xyz/config.sh should not have the directory
>>> hard-coded, it should retrieve the directory name of pwd or something
>>> similar.
>>>
>>>>
>>>> Once again, I'm a real ^%%$£%^%*&%*£!!
>>>>
>>>> I'm working to fix these two issues asap and will come back with
>>>> information asap.
>>>>
>>>>>
>>>>> - DML
>>>>>
>>>>> On 05/11/2009 11:06 AM, Galder Zamarreno wrote:
>>>>>> Hey,
>>>>>>
>>>>>> Run a couple of round of tests with JBMAR trunk (r200) and please
>>>>>> find attached the results. No real improvement I'm afraid. I'll
>>>>>> look into the profile data from r200 tomorrow.
>>>>>>
>>>>>> I'm currently working on the pooling.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> David M. Lloyd wrote:
>>>>>>> Galder, would you mind switching to trunk and trying that version
>>>>>>> (unofficial 1.2.0.CR1 - yes I *promise* I'll fix the versioning
>>>>>>> and get a source blob built asap!)?
>>>>>>>
>>>>>>> - DML
>>>>>>>
>>>>>>> On 05/08/2009 12:09 PM, Galder Zamarreno wrote:
>>>>>>>> Profiler data with JBMAR r174
>>>>>>>>
>>>>>>>> Galder Zamarreno wrote:
>>>>>>>>> Hi David,
>>>>>>>>>
>>>>>>>>> Please find attach graphs belonging to two runs that compare:
>>>>>>>>>
>>>>>>>>> infinispan-4.0.0(repl-sync) - home grown marshalling layer
>>>>>>>>> infinispan-4.0.0(repl-sync-jbmar) - infinispan + jbmar 1.1.2
>>>>>>>>> infinispan-4.0.0(repl-sync-jbmar)-rXYZ - infinispan + jbmar
>>>>>>>>> with revision
>>>>>>>>>
>>>>>>>>> Not sure what's the conclusion here tbh. The results of 1.1.2
>>>>>>>>> almost look opposite in each test.
>>>>>>>>>
>>>>>>>>> I've also attached some information from previously run
>>>>>>>>> profiling sessions with a couple of local machines we have in
>>>>>>>>> Neuchatel. I profiled the faster of the two machines.
>>>>>>>>>
>>>>>>>>> Actually, looking at this profiled data, these tests are for
>>>>>>>>> synchronous replicated caches but I see no traces of actually
>>>>>>>>> reading the stream, only writing to it, hmmmm.
>>>>>>>>>
>>>>>>>>> I'm adding Externalizers to class table and implementing
>>>>>>>>> marshaller/unmarshaller pooling as my next tasks.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> David M. Lloyd wrote:
>>>>>>>>>> OK, I tried out a few things. You might want to try
>>>>>>>>>> introducing these one at a time (i.e. update up to rev 173,
>>>>>>>>>> then 174, then 175 and see how each one does). In particular,
>>>>>>>>>> I think 175 has just as much chance of slowing things down as
>>>>>>>>>> speeding them up - either you're getting tons of collisions in
>>>>>>>>>> the hash table or the profiler is skewing the results there
>>>>>>>>>> (maybe try filtering out
>>>>>>>>>> org.jboss.marshalling.util.IdentityIntMap and java.lang.System
>>>>>>>>>> to see if that gives a different picture).
>>>>>>>>>>
>>>>>>>>>> I feel pretty good about 173 and 174 though I think the
>>>>>>>>>> profiler will skew 173 unless you have that UTFUtils filter
>>>>>>>>>> installed. If 175 slows things down (outside of the
>>>>>>>>>> profiler), let me know and I'll revert it. None of my tests
>>>>>>>>>> showed much difference but I don't have any good benchmarks
>>>>>>>>>> that really exercise that code right now.
>>>>>>>>>>
>>>>>>>>>> There's a couple things left to try yet, like looking at
>>>>>>>>>> replacing ConcurrentReferenceHashMap (assuming that isn't the
>>>>>>>>>> profiler again).
>>>>>>>>>>
>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> r175 | david.lloyd at jboss.com | 2009-05-08 00:17:46 -0500 (Fri,
>>>>>>>>>> 08 May 2009) | 1 line
>>>>>>>>>>
>>>>>>>>>> Try a trick to decrease the liklihood of collisions
>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> r174 | david.lloyd at jboss.com | 2009-05-08 00:04:39 -0500 (Fri,
>>>>>>>>>> 08 May 2009) | 1 line
>>>>>>>>>>
>>>>>>>>>> Replacement caching is not economical; the cost is one extra
>>>>>>>>>> hash table get for non-replaced objects, two hash table gets
>>>>>>>>>> (total) for replaced objects. Removing the cache gets rid of
>>>>>>>>>> the cost for non-replaced objects, while replaced objects now
>>>>>>>>>> have to be replaced again before the single hash table hit.
>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> r173 | david.lloyd at jboss.com | 2009-05-07 23:44:52 -0500 (Thu,
>>>>>>>>>> 07 May 2009) | 1 line
>>>>>>>>>>
>>>>>>>>>> JBMAR-52 - Avoid extra copy of char array (1.5 of 2)
>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - DML
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>
>>>
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>> ------------------------------------------------------------------------
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
More information about the infinispan-dev
mailing list