[infinispan-dev] Object instance reuse for reference data during Marshalling

[Sanne Grinovero]

Hi all,

we discussed this need a while back [1] and there's great news: Galder
created a demo showing how this might work on github:
 - https://github.com/galderz/shared-object-spaces

In case you want to better understand what we're after I'd suggest to
have a look: the code is trivial to read and the example is spot-on.

This single test is the goal and the setup should be self-explanatory:
 - https://github.com/galderz/shared-object-spaces/blob/master/src/main/java/sos/SharedObjectSpaces.java#L38

The magic is of course in how the Externalizers are allowed to keep a
registry of instances, essentially to not be fully stateless anymore.

I will now explore building this capability in Hibernate OGM, as some
performance tests had shown that without this feature the amount of
data was "balooning" unexpectedly - especially when some object is
frequently read/written and there's no much variation. In short, we
need to reap the benefits of a properly normalized data structure.

Some questions for further evolution:

# Injection framework

I don't love this need to produce serialized metadata at build time
and have to include it in our jars. I have just been playing with the
demo to see if I can avoid the injection but I failed and can see that
it's necessary for now.

Essentially I'd prefer to have the CountryFactory *instance* to be
created and managed by OGM before bootstrapping Infinispan, and then
inject the *instance* into the componentregistry. I guess I'm just
asking if we'll eventually be able to inject actual instances in the
component registry rather than injecting either FQN or Class
instances?

Today I need to:

gcr.registerComponent(countryFactory, CountryFactory.class);

I'd love to do:

gcr.registerComponent(countryFactory, countryFactoryInstance);

This would allow me to look up the CountryFactory instance from the
componentRegistry in various other integration points, like during the
LifecycleCallbacks methods where I'd register the custom
externalizers.
[Incidentally I believe I asked the same capability for other
components, like the Transport to be injected by instance as in some
situations I'd want to configure JGroups programmatically].

# Cache scoping

These components are global now as the Externalizer registry is
global. I understand the need to have global externalizers for
Infinispan itself, but when it comes to user data encoding - which is
cache specific - could we separate this?
I'm guessing this would help with multi-tenancy designs and with
sharing of CacheManagers across different deployments (classloaders)
too.
Or - like we also asked some while back - to allow independent
CacheManagers to share some core components like the Transport. It's
another long time debate which I'd really hope to see improved.

# Size management in the registry implementations

This example code uses a simple hashmap. Clearly a real solution would
need a bounded container, possibly with some eviction or allow
explicit eviction control to the application.
I will need a way to either inject a local, bounded Cache instance
within this component, or possibly programmatically just create a
simple cache out of the scope of the typical CacheManager ?

I'm also wondering if this Cache should use weak references.

The primary use case in OGM is for trivially small and bounded data
(metadata identifiers) so this is not a problem for my specific use
case, I'm merely mentioning it as others might need this.

# Requirements?

OGM is still using Infinispan 8. Do you think I could try this out
already or should we finish the upgrade to Infinispan 9 first? The
demo seems to work with Infinispan 8.2 as well: just wondering if I'm
missing something.

Thanks!
Sanne

[1] - https://issues.jboss.org/browse/ISPN-2133 ,
http://lists.jboss.org/pipermail/infinispan-dev/2012-June/010925.html
, http://lists.jboss.org/pipermail/infinispan-dev/2016-April/016602.html