<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 15, 2014 at 5:49 PM, Sanne Grinovero <span dir="ltr"><<a href="mailto:sanne@infinispan.org" target="_blank">sanne@infinispan.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class=""><div class="h5">On 15 July 2014 14:15, Dan Berindei <<a href="mailto:dan.berindei@gmail.com">dan.berindei@gmail.com</a>> wrote:<br>
><br>
><br>
><br>
> On Tue, Jul 15, 2014 at 12:51 AM, Sanne Grinovero <<a href="mailto:sanne@infinispan.org">sanne@infinispan.org</a>><br>
> wrote:<br>
>><br>
>> Hi all,<br>
>> I was toying with a custom CacheStore experiment, and am having some<br>
>> friction with some of the new SPIs.<br>
>><br>
>> So interface org.infinispan.marshall.core.MarshalledEntryFactory<K, V><br>
>> is an helper to use in the CacheStorei implementation, which exposes<br>
>> three methods:<br>
>><br>
>> MarshalledEntry<K,V> newMarshalledEntry(ByteBuffer key, ByteBuffer<br>
>> valueBytes, ByteBuffer metadataBytes);<br>
>> MarshalledEntry<K,V> newMarshalledEntry(Object key, ByteBuffer<br>
>> valueBytes, ByteBuffer metadataBytes);<br>
>> MarshalledEntry<K,V> newMarshalledEntry(Object key, Object value,<br>
>> InternalMetadata im);<br>
>><br>
>> In my CacheStore - and I suspect most efficiency minded<br>
>> implementations - I don't care about the value Object but I express a<br>
>> specific physical layout for the metadata, so to run for example an<br>
>> efficient "purge expired" task.<br>
>> So, the key is given, the value Object needs to be serialized, but the<br>
>> InternalMetadata I can map to specific fields.<br>
>><br>
>> Problem is at read time: I don't have a marshalled version of the<br>
>> Metadata but I need to unmarshall the value.. there is no helper to<br>
>> cover for this case.<br>
>><br>
>> Wouldn't this interface be more practical if it had:<br>
>><br>
>> Object unMarshallKey(ByteBuffer);<br>
>> Object unMarshallValue(ByteBuffer);<br>
>> InternalMetadata unMarshallMetadata(ByteBuffer);<br>
>> MarshalledEntry newMarshalledEntry(Object key, Object value,<br>
>> InternalMetadata im);<br>
><br>
><br>
> I guess the idea was that MarshalledEntry unmarshalls the key, value, and<br>
> metadata lazily. Even a purge listener may only be interested in the key,<br>
> and in that case case we can avoid unmarshalling the value.<br>
><br>
> I think you can do what you want by stuffing the bytes for everything in the<br>
> MarshalledEntry, and unmarshalling the data via<br>
> MarshalledEntry.getMetadata().<br>
<br>
<br>
</div></div>That's what I did but it's far from optimal, so I'm proposing the improvement.<br></blockquote><div><br></div><div>Could you expand a bit on why this is suboptimal? The way I see it, you have to support any custom Metadata implementation anyway, so you have to either read and deserialize the entire metadata, or store a duplicate of the expiration timestamp somewhere else.</div>
<div><br></div><div>In LevelDB we store the timestamps in a separate DB, doing 2 LevelDB writes for every store write (with expiration), and I would be very interested in an alternative solution that only wrote to 1 DB.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="">
<br>
>> Also, I'd get rid of generics. They are not helping at all, I can<br>
>> hardly couple my custom CacheStore implementation to the end user's<br>
>> domain model, right?<br>
><br>
><br>
> I can see the user writing a custom JDBC store that stores the object's<br>
> properties into separate columns and thus supporting a single type. But it's<br>
> a pretty specialized case, and the user can very well do the casts himself.<br>
<br>
</div>Exactly<br>
<div class=""><br>
><br>
> Maybe Paul and Will have more stuff to add here, they been discussing about<br>
> generics in the cache store SPIs around<br>
> <a href="https://github.com/infinispan/infinispan/pull/2705" target="_blank">https://github.com/infinispan/infinispan/pull/2705</a><br>
><br>
>><br>
>> I was also quite surprised that other existing CacheStore<br>
>> implementations don't have this limitation; peeking in the<br>
>> JDBCCacheStore to see how this is supposed to work, it seems that<br>
>> essentially it duplicates the data by serializazing the<br>
>> InternalMetadata in the BLOB but also stored an Expiry column to query<br>
>> via SQL. I was interested to see how the Purge method could be<br>
>> implemented efficiently, and found a "TODO notify listeners" ;-)<br>
><br>
><br>
> I believe the reason why we don't support purge listeners in the JDBC store<br>
> is that we don't want to fetch the entries from the database at all. We<br>
> can't ask CacheNotifier whether there are any listeners registered ATM, we<br>
> need that to avoid the overhead when there are no listeners.<br>
<br>
</div>I understand the reason, still it's wrong isn't it ;-)<br>
Having to load each entry for the "maybe there's a listener" case is<br>
definitely silly, we should inform the CacheStore instance if<br>
notifications are needed or not, and if they need just the key or the<br>
whole entry, no need for metadata.<br>
And for CacheStore instances which don't respect this we should at<br>
least document the limitation, or open a JIRA to get it done. Would<br>
you agree if I opened issues for each weirdness I'm finding in the<br>
existing CacheStore implementations? I've hit some more in the<br>
meantime.<br>
<span class=""><font color="#888888"><br></font></span></blockquote><div><br></div><div>Sure. I'm not too enthusiastic about changing the persistence SPI, but we should at least have a discussion on the pros and cons of the choices we are making there.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span class=""><font color="#888888"></font></span><div class="">
<div class="h5"><br>
><br>
>><br>
>> All other JDBC based stores serialize buckets in groups, REST store<br>
>> doesn't do purging, LevelDB also does duplication for the metadata,<br>
>> Cassandra is outdated and doesn't do events on expiry.<br></div></div></blockquote><div> <br></div></div></div></div>