[infinispan-dev] Potential for context object for serial/deserial in marshalling rework

William Burns mudokonman at gmail.com
Thu Apr 21 09:18:59 EDT 2016


On Thu, Apr 21, 2016 at 6:56 AM Sanne Grinovero <sanne at infinispan.org>
wrote:

> Hey Galder,
> these "scopes" you mention sound cool, but I'm afraid you'd end up
> designing a user friendly API more than allowing the level of
> performance optimisations we need.
>
> If we pass a "context" map, or eventually make it a map of maps when
> you'll figure you have nested structures such as [object type, object
> id], then:
>  - each deserialization event will need to perform multiple
> [concurrent]Map lookups
>

Unfortunately my implementation is quite similar although it is using a
HashMap and IdentityMap.  The main benefit of the externalizer I had was
that it only affected instances that extend it.


>  - I wouldn't be able to plug in an ad-hoc data structure: e.g. Will's
> "instance" strategy seems efficient :)
>  - we'll need to implement size-capping strategies, i.e. these would
> need to support eviction strategies, not least configuration options
> for such eviction.
>  - similar cost traversing multiple [concurrent]Map to write to the
> "context cache"
>
> I would suggest instead to give more flexibility to custom
> Externalizer implementors by making them stateful: if you could allow
> them to have something akin to the ComponentRegistry injected within
> the constructor, then a power user could pre-register custom services,
> look them up only once during construction, and then benefit from the
> performance of final field reads, from ad-hoc optimised containers.
> The implementor would also be fully responsible of making a reasonable
> choice to cap size.
>

To be honest I have been wanting this for a while.  Distributed Streams has
to do its own method of injecting the ComponentRegistry because we need
some cache components when using indexless querying.  Instead we have to do
[5] & [6] which is a bit cumbersome and is not usable by end users
whatsoever.

[5]
https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/stream/impl/intops/IntermediateOperation.java#L27
[6]
https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/stream/impl/LocalStreamManagerImpl.java#L251


>
> The complexity is then to make sure the components are booted in the
> right order, and that the Externalizer instance is bound to the right
> scope.
> I realize that the Externalizer chain needs to be booted early, but
> the services it would look up are unlikely to need anything else so it
> shouldn't be hard to figure a reasonable order for bootsrapping these
> components.
>
> For example, one complexity would be that an Externalizer instance
> which is bound to cache-specific services can not be part of an
> externalizer chain used for a different Cache, but I believe this is
> in place already because of the classloader requirements.
>
> WDYT, is this feasible? I realize it makes it harder to use, but then
> again I expect such tricks to be applied only by internal component
> and expert-level extensions (i.e. Hibernate OGM, Lucene, etc..).
>
> Thanks,
> Sanne
>
> On 21 April 2016 at 10:35, Galder Zamarreño <galder at redhat.com> wrote:
> > For example, I'd like to get some clarification on the scope of the
> context object:
> >
> > @Will, will the solution you provided in [2] deal with multiple JGroups
> marshalling request callbacks? Or is it only designed for a single
> marshalling request callback coming from JGroups tgat contains a
> CacheStatusResponse map? Since it's a thread local is not easy to see
> what's the lifespan of these instances stored in the thread local...
> >
> > The simplest thing would be to have a new context created for each
> JGroups marshalling request.
> >
> > However, more optimizations could be achieved from:
> >
> > 1. For non-transaction operations, a context per operation. So if a
> cache.put() results in two operations being serialized (get to return
> previous value, and put itself), then the context could expand those two
> serializations.
> >
> > 2. For transaction operations, a context per transaction.
> >
> > @Sanne, would this be enough for you? In your [4] dev post you seem to
> want to go beyond marshalling into how we'd keep references of objects in
> memory, e.g. if a String is repeated in many places, have a way to
> centralise that storage in memory itself.
> >
> > Cheers,
> >
> > [4]
> http://lists.jboss.org/pipermail/infinispan-dev/2012-June/010925.html
> > --
> > Galder Zamarreño
> > Infinispan, Red Hat
> >
> >> On 21 Apr 2016, at 10:07, Galder Zamarreño <galder at redhat.com> wrote:
> >>
> >> Hey guys,
> >>
> >> Just a quick heads up about [1].
> >>
> >> As I was looking at the marshalling code in core, I spotted the work
> done for [2] and by extension [3].
> >>
> >> I can certainly see the practicality of Will's solution in [2] which
> fitted quite well with the current marshalling architecture, but as we
> rethink the entire marshalling layer in [1], I'm wondering if a
> context-object where we can track repeated fields like Strings,
> Addresses... would be more suitable. For starters, we'd get rid of thread
> locals and could be more easily exposed in other places.
> >>
> >> Any ideas or updates you have on the topic please let me know.
> >>
> >> Cheers,
> >>
> >> [1] https://issues.jboss.org/browse/ISPN-6498
> >> [2] https://issues.jboss.org/browse/ISPN-4979
> >> [3] https://issues.jboss.org/browse/ISPN-2133
> >> --
> >> Galder Zamarreño
> >> Infinispan, Red Hat
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160421/d7dd79ce/attachment.html 


More information about the infinispan-dev mailing list