<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Thu, Apr 21, 2016 at 6:56 AM Sanne Grinovero <<a href="mailto:sanne@infinispan.org">sanne@infinispan.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hey Galder,<br>
these "scopes" you mention sound cool, but I'm afraid you'd end up<br>
designing a user friendly API more than allowing the level of<br>
performance optimisations we need.<br>
<br>
If we pass a "context" map, or eventually make it a map of maps when<br>
you'll figure you have nested structures such as [object type, object<br>
id], then:<br>
- each deserialization event will need to perform multiple<br>
[concurrent]Map lookups<br></blockquote><div> </div><div>Unfortunately my implementation is quite similar although it is using a HashMap and IdentityMap. The main benefit of the externalizer I had was that it only affected instances that extend it.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
- I wouldn't be able to plug in an ad-hoc data structure: e.g. Will's<br>
"instance" strategy seems efficient :)<br>
- we'll need to implement size-capping strategies, i.e. these would<br>
need to support eviction strategies, not least configuration options<br>
for such eviction.<br>
- similar cost traversing multiple [concurrent]Map to write to the<br>
"context cache"<br>
<br>
I would suggest instead to give more flexibility to custom<br>
Externalizer implementors by making them stateful: if you could allow<br>
them to have something akin to the ComponentRegistry injected within<br>
the constructor, then a power user could pre-register custom services,<br>
look them up only once during construction, and then benefit from the<br>
performance of final field reads, from ad-hoc optimised containers.<br>
The implementor would also be fully responsible of making a reasonable<br>
choice to cap size.<br></blockquote><div><br></div><div>To be honest I have been wanting this for a while. Distributed Streams has to do its own method of injecting the ComponentRegistry because we need some cache components when using indexless querying. Instead we have to do [5] & [6] which is a bit cumbersome and is not usable by end users whatsoever.</div><div><br></div><div>[5] <a href="https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/stream/impl/intops/IntermediateOperation.java#L27">https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/stream/impl/intops/IntermediateOperation.java#L27</a></div><div>[6] <a href="https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/stream/impl/LocalStreamManagerImpl.java#L251">https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/stream/impl/LocalStreamManagerImpl.java#L251</a></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
The complexity is then to make sure the components are booted in the<br>
right order, and that the Externalizer instance is bound to the right<br>
scope.<br>
I realize that the Externalizer chain needs to be booted early, but<br>
the services it would look up are unlikely to need anything else so it<br>
shouldn't be hard to figure a reasonable order for bootsrapping these<br>
components.<br>
<br>
For example, one complexity would be that an Externalizer instance<br>
which is bound to cache-specific services can not be part of an<br>
externalizer chain used for a different Cache, but I believe this is<br>
in place already because of the classloader requirements.<br>
<br>
WDYT, is this feasible? I realize it makes it harder to use, but then<br>
again I expect such tricks to be applied only by internal component<br>
and expert-level extensions (i.e. Hibernate OGM, Lucene, etc..).<br>
<br>
Thanks,<br>
Sanne<br>
<br>
On 21 April 2016 at 10:35, Galder Zamarreño <<a href="mailto:galder@redhat.com" target="_blank">galder@redhat.com</a>> wrote:<br>
> For example, I'd like to get some clarification on the scope of the context object:<br>
><br>
> @Will, will the solution you provided in [2] deal with multiple JGroups marshalling request callbacks? Or is it only designed for a single marshalling request callback coming from JGroups tgat contains a CacheStatusResponse map? Since it's a thread local is not easy to see what's the lifespan of these instances stored in the thread local...<br>
><br>
> The simplest thing would be to have a new context created for each JGroups marshalling request.<br>
><br>
> However, more optimizations could be achieved from:<br>
><br>
> 1. For non-transaction operations, a context per operation. So if a cache.put() results in two operations being serialized (get to return previous value, and put itself), then the context could expand those two serializations.<br>
><br>
> 2. For transaction operations, a context per transaction.<br>
><br>
> @Sanne, would this be enough for you? In your [4] dev post you seem to want to go beyond marshalling into how we'd keep references of objects in memory, e.g. if a String is repeated in many places, have a way to centralise that storage in memory itself.<br>
><br>
> Cheers,<br>
><br>
> [4] <a href="http://lists.jboss.org/pipermail/infinispan-dev/2012-June/010925.html" rel="noreferrer" target="_blank">http://lists.jboss.org/pipermail/infinispan-dev/2012-June/010925.html</a><br>
> --<br>
> Galder Zamarreño<br>
> Infinispan, Red Hat<br>
><br>
>> On 21 Apr 2016, at 10:07, Galder Zamarreño <<a href="mailto:galder@redhat.com" target="_blank">galder@redhat.com</a>> wrote:<br>
>><br>
>> Hey guys,<br>
>><br>
>> Just a quick heads up about [1].<br>
>><br>
>> As I was looking at the marshalling code in core, I spotted the work done for [2] and by extension [3].<br>
>><br>
>> I can certainly see the practicality of Will's solution in [2] which fitted quite well with the current marshalling architecture, but as we rethink the entire marshalling layer in [1], I'm wondering if a context-object where we can track repeated fields like Strings, Addresses... would be more suitable. For starters, we'd get rid of thread locals and could be more easily exposed in other places.<br>
>><br>
>> Any ideas or updates you have on the topic please let me know.<br>
>><br>
>> Cheers,<br>
>><br>
>> [1] <a href="https://issues.jboss.org/browse/ISPN-6498" rel="noreferrer" target="_blank">https://issues.jboss.org/browse/ISPN-6498</a><br>
>> [2] <a href="https://issues.jboss.org/browse/ISPN-4979" rel="noreferrer" target="_blank">https://issues.jboss.org/browse/ISPN-4979</a><br>
>> [3] <a href="https://issues.jboss.org/browse/ISPN-2133" rel="noreferrer" target="_blank">https://issues.jboss.org/browse/ISPN-2133</a><br>
>> --<br>
>> Galder Zamarreño<br>
>> Infinispan, Red Hat<br>
>><br>
>><br>
>> _______________________________________________<br>
>> infinispan-dev mailing list<br>
>> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
><br>
><br>
> _______________________________________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
<br>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a></blockquote></div></div>