Hey guys, Just a quick heads up about [1]. As I was looking at the marshalling code in core, I spotted the work done for [2] and by extension [3]. I can certainly see the practicality of Will's solution in [2] which fitted quite well with the current marshalling architecture, but as we rethink the entire marshalling layer in [1], I'm wondering if a context-object where we can track repeated fields like Strings, Addresses... would be more suitable. For starters, we'd get rid of thread locals and could be more easily exposed in other places. Any ideas or updates you have on the topic please let me know. Cheers, [1] https://issues.jboss.org/browse/ISPN-6498 [2] https://issues.jboss.org/browse/ISPN-4979 [3] https://issues.jboss.org/browse/ISPN-2133 -- Galder Zamarreño Infinispan, Red Hat _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 21 Apr 2016, at 10:07, Galder Zamarreño <galder(a)redhat.com&gt; wrote: Hey guys, Just a quick heads up about [1]. As I was looking at the marshalling code in core, I spotted the work done for [2] and by extension [3]. I can certainly see the practicality of Will's solution in [2] which fitted quite well with the current marshalling architecture, but as we rethink the entire marshalling layer in [1], I'm wondering if a context-object where we can track repeated fields like Strings, Addresses... would be more suitable. For starters, we'd get rid of thread locals and could be more easily exposed in other places. Any ideas or updates you have on the topic please let me know. Cheers, [1] https://issues.jboss.org/browse/ISPN-6498 [2] https://issues.jboss.org/browse/ISPN-4979 [3] https://issues.jboss.org/browse/ISPN-2133 -- Galder Zamarreño Infinispan, Red Hat _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Hey Galder, these "scopes" you mention sound cool, but I'm afraid you'd end up designing a user friendly API more than allowing the level of performance optimisations we need. If we pass a "context" map, or eventually make it a map of maps when you'll figure you have nested structures such as [object type, object id], then: - each deserialization event will need to perform multiple [concurrent]Map lookups

- I wouldn't be able to plug in an ad-hoc data structure: e.g. Will's "instance" strategy seems efficient :) - we'll need to implement size-capping strategies, i.e. these would need to support eviction strategies, not least configuration options for such eviction. - similar cost traversing multiple [concurrent]Map to write to the "context cache" I would suggest instead to give more flexibility to custom Externalizer implementors by making them stateful: if you could allow them to have something akin to the ComponentRegistry injected within the constructor, then a power user could pre-register custom services, look them up only once during construction, and then benefit from the performance of final field reads, from ad-hoc optimised containers. The implementor would also be fully responsible of making a reasonable choice to cap size.

The complexity is then to make sure the components are booted in the right order, and that the Externalizer instance is bound to the right scope. I realize that the Externalizer chain needs to be booted early, but the services it would look up are unlikely to need anything else so it shouldn't be hard to figure a reasonable order for bootsrapping these components. For example, one complexity would be that an Externalizer instance which is bound to cache-specific services can not be part of an externalizer chain used for a different Cache, but I believe this is in place already because of the classloader requirements. WDYT, is this feasible? I realize it makes it harder to use, but then again I expect such tricks to be applied only by internal component and expert-level extensions (i.e. Hibernate OGM, Lucene, etc..). Thanks, Sanne On 21 April 2016 at 10:35, Galder Zamarreño <galder(a)redhat.com&gt; wrote: > For example, I'd like to get some clarification on the scope of the context object: > > @Will, will the solution you provided in [2] deal with multiple JGroups marshalling request callbacks? Or is it only designed for a single marshalling request callback coming from JGroups tgat contains a CacheStatusResponse map? Since it's a thread local is not easy to see what's the lifespan of these instances stored in the thread local... > > The simplest thing would be to have a new context created for each JGroups marshalling request. > > However, more optimizations could be achieved from: > > 1. For non-transaction operations, a context per operation. So if a cache.put() results in two operations being serialized (get to return previous value, and put itself), then the context could expand those two serializations. > > 2. For transaction operations, a context per transaction. > > @Sanne, would this be enough for you? In your [4] dev post you seem to want to go beyond marshalling into how we'd keep references of objects in memory, e.g. if a String is repeated in many places, have a way to centralise that storage in memory itself. > > Cheers, > > [4] http://lists.jboss.org/pipermail/infinispan-dev/2012-June/010925.html > -- > Galder Zamarreño > Infinispan, Red Hat > >> On 21 Apr 2016, at 10:07, Galder Zamarreño <galder(a)redhat.com&gt; wrote: >> >> Hey guys, >> >> Just a quick heads up about [1]. >> >> As I was looking at the marshalling code in core, I spotted the work done for [2] and by extension [3]. >> >> I can certainly see the practicality of Will's solution in [2] which fitted quite well with the current marshalling architecture, but as we rethink the entire marshalling layer in [1], I'm wondering if a context-object where we can track repeated fields like Strings, Addresses... would be more suitable. For starters, we'd get rid of thread locals and could be more easily exposed in other places. >> >> Any ideas or updates you have on the topic please let me know. >> >> Cheers, >> >> [1] https://issues.jboss.org/browse/ISPN-6498 >> [2] https://issues.jboss.org/browse/ISPN-4979 >> [3] https://issues.jboss.org/browse/ISPN-2133 >> -- >> Galder Zamarreño >> Infinispan, Red Hat >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev