[infinispan-dev] Data Container configuration changes
William Burns
mudokonman at gmail.com
Fri Nov 11 11:16:57 EST 2016
Since I have had some time to think it over I like Pedro's layout, unless
someone has a strong objection.
Responses inline.
On Mon, Oct 31, 2016 at 4:51 AM Radim Vansa <rvansa at redhat.com> wrote:
> On 10/27/2016 05:26 PM, William Burns wrote:
> >
> >
> > On Thu, Oct 27, 2016 at 9:56 AM Pedro Ruivo <pedro at infinispan.org
> > <mailto:pedro at infinispan.org>> wrote:
> >
> >
> >
> > On 27-10-2016 14:20, William Burns wrote:
> > >
> > >
> > > On Thu, Oct 27, 2016 at 4:56 AM Pedro Ruivo
> > <pedro at infinispan.org <mailto:pedro at infinispan.org>
> > > <mailto:pedro at infinispan.org <mailto:pedro at infinispan.org>>>
> wrote:
> > >
> > >
> > >
> > > On 26-10-2016 23:06, William Burns wrote:
> > > > I have been working on adding in off heap support for a given
> > > cache. I
> > > > wanted to check in and let you all know what I was
> > thinking for the
> > > > configuration and changes that would come about with it.
> > > >
> > > > TLDR;
> > > > New config under data container to enable off heap,
> > StoreAsBinary
> > > > removed, Equivalence removed
> > > >
> > > > First I was planning on adding new sub elements of data
> > container.
> > > > These would be instance, binary and off-heap. Only of the
> > three could
> > > > be picked as they are mutually exclusive. Instance is as we
> > > operate now
> > > > where we store the instance of the object passed to us.
> > Binary is
> > > > essentially what we have now that is called storeAsBinary
> with
> > > both keys
> > > > and values converted. Lastly off-heap would store the
> > entry as a
> > > byte[]
> > > > store completely in native memory.
> > >
> > > I prefer 'object' instead of 'instance'.
> > >
> > >
> > > Sounds fine with me.
> > >
> > >
> > >
> > > Are you also planning to remove the expiration and/or eviction
> > > configuration elements and set them in the data-container
> > sub elements?
> > >
> > >
> > > I wasn't planning on touching those. But now that you mention it,
> > > eviction only makes sense in the data container, where as
> > expiration is
> > > container and cache store. And taking this further storeAsBinary
> is
> > > both as well, only off-heap is container only. I wonder if
> > instead we
> > > should have a separate storage element at the same level as
> > > data-container. I can see it either way.
> >
> > Let me know if this makes sense:
> >
> > <expiration> //no changes here
> > <memory evictionStrategy=... evictionPolicy=...>
>
There wouldn't be any elements here. The strategy would always be TinyLFU
as this is all Caffeine supports. The thread policy though is interesting,
I would say for now don't let it be configured and it will run in the same
thead. However Caffeine does have async eviction which we could utilize in
the future. But I think we can look into that later and not part of this.
> >
> > //one of the following
> > <on-heap max-entries=.../>
> > <on-heap-binary max-size=.../>
> > <off-heap ...max-size? and off-heap config.../>
>
I think I like the naming of the elements as OBJECT, BINARY and
OFF-HEAP/NATIVE that Radim mentioned.
I am thinking for now OBJECT would only support max-entries and the other 2
could support both max-entries or max-size.
> >
> > </memory>
> > <persistence> //no changes here
> >
> > wdyt?
> >
>
> While I prefer "on-heap" instead of "object" or "instance", I don't
> think that "binary" should be its own element. Are there any attributes
> specific to that (do you plan to have keys="false" values="true"? I
> guess not). "on-heap" and "off-heap" is a binary ( :-) ) option,
> "on-heap-binary" is just a flavour of "on-heap".
>
Right now the plan is to remove keys and values and always do both. But we
may have to change that later, I don't know. Having separate elements
gives us more flexibility which I like.
>
> For comparison, HC uses
> <in-memory-format>OBJECT|BINARY|NATIVE</in-memory-format> where "NATIVE"
> means off-heap. I like "format= OBJECT|BINARY" as a child of on-heap,
> either as attribute or element. I haven't found similar settings in
> Coherence - seems that they store data in a serialized form only when
> they persist to disk/flash or offload to non-managed memory.
>
> Regarding Equivalence: can't we wrap objects in a similar way we do with
> byte[] if Equivalance (different from AnyInstance) is defined? I can
> definitely see use case when the hashCode() is not very well defined and
> user can't change the class - he has to wrap the objects themselves.
>
Hrmm we could add that back in if it is really needed. As far as I know
though no one has really used this. I would rather make it more
minimalistic to begin with and add back on as we find we need features.
>
> R.
>
> >
> > Seems fine to me. And to be honest I forgot to mention this but
> > EvictionStrategy and EvictionPolicy are completely redundant now.
> > Policy has been for a while as we always used the same thread and
> > Strategy is only Caffeine and for off heap I was thinking of a simple
> LRU.
> >
> > This means that the data container element would be removed in favor
> > of "memory"? The reason being is that equivalence will be gone and
> > afaik we never really supported a custom data container (eviction
> > wouldn't work with it and neither would off heap). In that case why
> > not just keep using data container element?
> >
> > >
> > >
> > >
> > >
> > > >
> > > > Example:
> > > >
> > > > <data-container>
> > > > <off-heap/>
> > > > </data-container>
> > > >
> > > > The reason it is a subelement instead of a property is
> because
> > > off-heap
> > > > will most likely require some additional configuration to
> > tell how
> > > many
> > > > entries to store in the a bucket (think non resizing
> HashMap).
> > > >
> > > > With these changes storeAsBinary becomes redundant, so I was
> > > planning on
> > > > removing this configuration completely. I would rather
> > remove since
> > > > this is 9.0 and not deprecate. As far as I know nobody
> > really used it
> > > > before.
> > > >
> > > > Also another side effect is I was removing all of the
> > Equivalence
> > > > classes. I am not sure if I can plainly remove them since
> > they have
> > > > lived in commons for quite a while, but it would be best
> > if I could,
> > > > although I am fine deprecating. In its place the instance
> > setting for
> > > > data-container will always wrap byte[] to satisfy equals
> > and hashCode
> > > > methods.
> > > >
> > > > Any feedback would be appreciated.
> > > >
> > > > Thanks,
> > > > - Will
> > > >
> > > >
> > > > _______________________________________________
> > > > infinispan-dev mailing list
> > > > infinispan-dev at lists.jboss.org
> > <mailto:infinispan-dev at lists.jboss.org>
> > <mailto:infinispan-dev at lists.jboss.org
> > <mailto:infinispan-dev at lists.jboss.org>>
> > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > > >
> > > _______________________________________________
> > > infinispan-dev mailing list
> > > infinispan-dev at lists.jboss.org
> > <mailto:infinispan-dev at lists.jboss.org>
> > <mailto:infinispan-dev at lists.jboss.org
> > <mailto:infinispan-dev at lists.jboss.org>>
> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >
> > >
> > >
> > > _______________________________________________
> > > infinispan-dev mailing list
> > > infinispan-dev at lists.jboss.org
> > <mailto:infinispan-dev at lists.jboss.org>
> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org <mailto:
> infinispan-dev at lists.jboss.org>
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161111/0de94bbe/attachment-0001.html
More information about the infinispan-dev
mailing list