[infinispan-dev] Bye bye wrappers, ComparingConcurrentHashMapv8 is here (ISPN-2281)

[Galder Zamarreño]

On Mar 19, 2013, at 8:56 PM, Mircea Markus <mmarkus at redhat.com> wrote:

> 
> On 18 Mar 2013, at 12:21, Galder Zamarreño wrote:
> 
>> Hi all,
>> 
>> A heads up on what is going on with https://issues.jboss.org/browse/ISPN-2281
>> 
>> While discussing this, Tristan and I came to the conclusion that we could avoid the need to create some wrappers required to fulfill requirements in this JIRA, and as a side effect, reduce the memory consumption of Infinispan servers, if we could have internal data containers based on concurrent hash maps that took a custom function for equals/hashCode…etc. By doing that, you could effectively have **byte[] keys and values for maps**.
>> 
>> By doing that, you avoid creating wrappers (yippee!) for keys (bye bye ByteArrayKey), and combined with a better way to pass metadata into Infinispan Caches (i.e. version) that is stored within the internal cache entries, you avoid wrapper values too! (bye bye CacheValue).
> Do you plan to use Versioned*CacheValue for storing values? if so you'd still create a version object to be aggregated in Versioned*CacheValue.

^ Yes, precisely! This was very easy to do actually. Once I had the API to feed in version (and other metadata down the line…), it was very easy to change the core to use internal cache entry classes to store. Hence, no extra wrapper value other than what Infinispan uses internally.

>> 
>> Doing the latter was relatively simple (I have this stashed), but having a CHM that could take a byte[] as key wasn't that easy, since we can't change JDK CHM.
> why's that?  copyright?

^ Well, I cannot modify the JDK source code directly. I'd have to fork the JDK CHM, but I prefer to take the CHMv8 version since it gets rid of segments.

>> 
>> This is why, I've created a new CHM, based on the CHMv8, called ComparingConcurrentHashMapv8 (thx Tristan for the name!). The work for this can be seen in: https://github.com/galderz/infinispan/commit/351e29d327d163ca8e941edf873f6d46b43cfae1
> this relies on sun.misc.Unsafe, so won't work with a non-oracle JVM.

^ Hmmm, possibly. So we might need a JDK CHM version for it. I've already transformed BoundedCHM to it, so woud be easy to achieve.

> This class would be aggregated in the DataContainer in the case we use infinispan in server mode, right?

^ Right now, I've added two configuration options to dataContainer element, called comparingKey and comparingValue, both of which are class names to implementations of Comparing. If either is defined, a Comparing*CHM would be used. So, Hot Rod servers would need to have ComparingByteArray defined in comparingKey and comparingValue in their configuration. 

I think this is something we can control in both the default Infinispan and JDG servers.

Thanks for the excellent feedback! :)

>> 
>> I'm sending it here so that I can get feedback early on. I've added some tests as well that verify that ComparingConcurrentHashMapv8, with byte[] keys and values, works as expected, and checks that the expectations are opposite with JDK CHM. It also tests new function-based methods.
>> 
>> To make it easier to track changes as original CHMv8 evolves, I've marked all changes with a marker comment that should make it easy to apply same changes in new CHMv8 versions. Plus, with the tests I've added, it can easily be seen if it works as expected or not.
>> 
>> Two important TODOs, which will be most likely separated into separate JIRAs:
>> 
>> 1. Note that TreeBin has not been modified to use custom equals/hashCode functions. That is cos I need to implement a way to compare byte arrays, i.e. provide equivalent logic for Comparable.compare().
>> 
>> 2. Compare memory consumption of a CHMv8 with wrapper classes for byte arrays versus ComparingConcurrentHashMapv8<byte[], byte[]>. I'll do that once it's closer to CR stages. Right now fulfilling requirements in ISPN-2281 is more priority.
>> 
>> Finally, I need to do the same thing with out BoundedConcurrentHashMap, iow, provide a way to do comparison based on custom equals/hashCode. That's gonna be my next task, before I get to transform Infinispan Servers to take a type directly, and avoid relying on ByteArrayKey or CacheValue wrappers. 
>> 
>> IOW, you'll be able to say: create an Infinsipan Server that has String as key and value of type X, where X is the actual data type, no metadata!! The metadata (version, encoding, whatever is requried to fulfill the compatibility reqs in ISPN-2281) will be passed as part of the put/replace…etc (I will email this around when in place). 
>> 
>> Cheers,
>> --
>> Galder Zamarreño
>> galder at redhat.com
>> twitter.com/galderz
>> 
>> Project Lead, Escalante
>> http://escalante.io
>> 
>> Engineer, Infinispan
>> http://infinispan.org
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> Cheers,
> -- 
> Mircea Markus
> Infinispan lead (www.infinispan.org)
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarreño
galder at redhat.com
twitter.com/galderz

Project Lead, Escalante
http://escalante.io

Engineer, Infinispan
http://infinispan.org