[infinispan-dev] Data Container configuration changes
Radim Vansa
rvansa at redhat.com
Mon Oct 31 04:50:51 EDT 2016
On 10/27/2016 05:26 PM, William Burns wrote:
>
>
> On Thu, Oct 27, 2016 at 9:56 AM Pedro Ruivo <pedro at infinispan.org
> <mailto:pedro at infinispan.org>> wrote:
>
>
>
> On 27-10-2016 14:20, William Burns wrote:
> >
> >
> > On Thu, Oct 27, 2016 at 4:56 AM Pedro Ruivo
> <pedro at infinispan.org <mailto:pedro at infinispan.org>
> > <mailto:pedro at infinispan.org <mailto:pedro at infinispan.org>>> wrote:
> >
> >
> >
> > On 26-10-2016 23:06, William Burns wrote:
> > > I have been working on adding in off heap support for a given
> > cache. I
> > > wanted to check in and let you all know what I was
> thinking for the
> > > configuration and changes that would come about with it.
> > >
> > > TLDR;
> > > New config under data container to enable off heap,
> StoreAsBinary
> > > removed, Equivalence removed
> > >
> > > First I was planning on adding new sub elements of data
> container.
> > > These would be instance, binary and off-heap. Only of the
> three could
> > > be picked as they are mutually exclusive. Instance is as we
> > operate now
> > > where we store the instance of the object passed to us.
> Binary is
> > > essentially what we have now that is called storeAsBinary with
> > both keys
> > > and values converted. Lastly off-heap would store the
> entry as a
> > byte[]
> > > store completely in native memory.
> >
> > I prefer 'object' instead of 'instance'.
> >
> >
> > Sounds fine with me.
> >
> >
> >
> > Are you also planning to remove the expiration and/or eviction
> > configuration elements and set them in the data-container
> sub elements?
> >
> >
> > I wasn't planning on touching those. But now that you mention it,
> > eviction only makes sense in the data container, where as
> expiration is
> > container and cache store. And taking this further storeAsBinary is
> > both as well, only off-heap is container only. I wonder if
> instead we
> > should have a separate storage element at the same level as
> > data-container. I can see it either way.
>
> Let me know if this makes sense:
>
> <expiration> //no changes here
> <memory evictionStrategy=... evictionPolicy=...>
>
> //one of the following
> <on-heap max-entries=.../>
> <on-heap-binary max-size=.../>
> <off-heap ...max-size? and off-heap config.../>
>
> </memory>
> <persistence> //no changes here
>
> wdyt?
>
While I prefer "on-heap" instead of "object" or "instance", I don't
think that "binary" should be its own element. Are there any attributes
specific to that (do you plan to have keys="false" values="true"? I
guess not). "on-heap" and "off-heap" is a binary ( :-) ) option,
"on-heap-binary" is just a flavour of "on-heap".
For comparison, HC uses
<in-memory-format>OBJECT|BINARY|NATIVE</in-memory-format> where "NATIVE"
means off-heap. I like "format= OBJECT|BINARY" as a child of on-heap,
either as attribute or element. I haven't found similar settings in
Coherence - seems that they store data in a serialized form only when
they persist to disk/flash or offload to non-managed memory.
Regarding Equivalence: can't we wrap objects in a similar way we do with
byte[] if Equivalance (different from AnyInstance) is defined? I can
definitely see use case when the hashCode() is not very well defined and
user can't change the class - he has to wrap the objects themselves.
R.
>
> Seems fine to me. And to be honest I forgot to mention this but
> EvictionStrategy and EvictionPolicy are completely redundant now.
> Policy has been for a while as we always used the same thread and
> Strategy is only Caffeine and for off heap I was thinking of a simple LRU.
>
> This means that the data container element would be removed in favor
> of "memory"? The reason being is that equivalence will be gone and
> afaik we never really supported a custom data container (eviction
> wouldn't work with it and neither would off heap). In that case why
> not just keep using data container element?
>
> >
> >
> >
> >
> > >
> > > Example:
> > >
> > > <data-container>
> > > <off-heap/>
> > > </data-container>
> > >
> > > The reason it is a subelement instead of a property is because
> > off-heap
> > > will most likely require some additional configuration to
> tell how
> > many
> > > entries to store in the a bucket (think non resizing HashMap).
> > >
> > > With these changes storeAsBinary becomes redundant, so I was
> > planning on
> > > removing this configuration completely. I would rather
> remove since
> > > this is 9.0 and not deprecate. As far as I know nobody
> really used it
> > > before.
> > >
> > > Also another side effect is I was removing all of the
> Equivalence
> > > classes. I am not sure if I can plainly remove them since
> they have
> > > lived in commons for quite a while, but it would be best
> if I could,
> > > although I am fine deprecating. In its place the instance
> setting for
> > > data-container will always wrap byte[] to satisfy equals
> and hashCode
> > > methods.
> > >
> > > Any feedback would be appreciated.
> > >
> > > Thanks,
> > > - Will
> > >
> > >
> > > _______________________________________________
> > > infinispan-dev mailing list
> > > infinispan-dev at lists.jboss.org
> <mailto:infinispan-dev at lists.jboss.org>
> <mailto:infinispan-dev at lists.jboss.org
> <mailto:infinispan-dev at lists.jboss.org>>
> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> <mailto:infinispan-dev at lists.jboss.org>
> <mailto:infinispan-dev at lists.jboss.org
> <mailto:infinispan-dev at lists.jboss.org>>
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> <mailto:infinispan-dev at lists.jboss.org>
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team
More information about the infinispan-dev
mailing list