On the other hand if the group feature adds a way to:
- get all the keys,
- get a subset of the keys based on a filter
Infinispan will be able to start supporting use cases restricted to Cassandra in the past
(esp around time series).
I am assuming groups offer offer a way to add / change and remove a key from / to a group
without having to load all the group or even the group keys.
On 22 Jan 2014, at 14:33, Emmanuel Bernard <emmanuel(a)hibernate.org> wrote:
BTW query support on groups (by entry of each group) is an
interesting non covered use case today.
On 22 Jan 2014, at 14:26, Emmanuel Bernard <emmanuel(a)hibernate.org> wrote:
> Conceptually I like the grouping API better than AtomicMap as I don’t have to rely on
a specific Infinispan type.
>
> We do use FineGrainedAtomicMap both for the entity and the association persistence
(not AtomicMap). It is particularly critical for how we store the association navigation
information. I don’t want one update to literally prevent the whole association from being
updated. This is the same semantic a RDBMS has and that’s why Manik and I designed the
FGAM requirements.
>
> So my question is what are the differences between the grouping API and the FGAM in
particular for:
>
> - the amount of data sent back and forth (seems like grouping is sending the data
naturally per key as “delta compared to the group"
> - the locking level when a new entry is added to the FGAM / Grouping API
> - the locking level when a new entry is removed to the FGAM / Grouping API
> - the locking level when a new entry is updated to the FGAM / Grouping API
> - the overall network verbosity
> - does grouping offer the same repeatable read protection that AtomicMap offers
within a transaction?
>
> I think retrying as a transaction workaround is quite fragile. We can offer it as a
solution but supporting or encouraging it is another story. Unless each OGM nodes do
behave like a transaction but that would be wrong. I am also concerned about reading data
form a group that are inconsistent.
>
> Emmanuel
>
> On 21 Jan 2014, at 16:07, Mircea Markus <mmarkus(a)redhat.com> wrote:
>
>> Hi Emmanuel,
>>
>> Just had a good chat with Davide on this and one solution to overcome the
shortcoming you mentioned in the above email would be to enhance the hotrod client to
support grouping:
>>
>> RemoteClient.put(G g, K k, V v); //first param is the group
>> RemoteClinet.getGroup(G g) : Map<K,V>;
>>
>> It requires an enhancement on our local grouping API: EmbeddedCache.getGroup(G).
This is something useful for us in a broader context, as it is the step needed to be able
to deprecated AtomicMaps and get suggest them being replaced with Grouping.
>>
>> This approach still has some limitations compared to the current embedded
integration:
>> - performance caused by the lack of transactions: this means increased TCP
chattiness between the Hot Rod client and the server.
>> - you'd have to handle atomicity, potentially by retrying an operation
>>
>> What do you think?
>>
>>
>> On Dec 3, 2013, at 3:10 AM, Mircea Markus <mmarkus(a)redhat.com> wrote:
>>
>>>
>>> On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard <emmanuel(a)hibernate.org>
wrote:
>>>
>>>> It's an interesting approach that would work fine-ish for entities
>>>> assuming the Hot Rod client is multi threaded and assuming the client
>>>> uses Future to parallelize the calls.
>>>
>>> The Java Hotrod client is both multithreaded and exposes an Async API.
>>>
>>>>
>>>> But it won't work for associations as we have them designed today.
>>>> Each association - or more precisely the query results to go from an
>>>> entity A1 to the list of entities B associated to it - is represented by
>>>> an AtomicMap.
>>>> Each entry in this map does correspond to an entry in the association.
>>>>
>>>> While we can "guess" the column names and build from the
metadata the
>>>> list of composed keys for entities, we cannot do the same for
>>>> associations as the key is literally the (composite) id of the
>>>> association and we cannot guess that most of the time (we can in very
>>>> pathological cases).
>>>> We could imagine that we list the association row keys in a special
>>>> entry to work around that but this approach is just as problematic and
>>>> is conceptually the same.
>>>> The only solution would be to lock the whole association for each
>>>> operation and I guess impose some versioning / optimistic lock.
>>>>
>>>> That is not a pattern that scales sufficiently from my experience.
>>>
>>> I think so too :-)
>>>
>>>> That's the problem with interconnected data :)
>>>>
>>>> Emmanuel
>>>>
>>>> On Mon 2013-11-18 23:05, Mircea Markus wrote:
>>>>> Neither the grouping API nor the AtomicMap work over hotrod.
>>>>> Between the grouping API and AtomicMap, I think the one that would
make more sense migrating is the grouping API.
>>>>> One way or the other, I think the hotrod protocol would require an
enhancement - mind raising a JIRA for that?
>>>>> For now I guess you can sacrifice performance and always sending the
entire object across on every update instead of only the deltas?
>>>>>
>>>>> On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard
<emmanuel(a)hibernate.org> wrote:
>>>>>
>>>>>> Someone mentioned the grouping API as some sort of alternative
to
>>>>>> AtomicMap. Maybe we should use that?
>>>>>> Note that if we don't have a fine-grained approach we will
need to
>>>>>> make sure we *copy* the complex data structure upon reads to
mimic
>>>>>> proper transaction isolation.
>>>>>>
>>>>>> On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
>>>>>>> On 12 November 2013 14:54, Emmanuel Bernard
<emmanuel(a)hibernate.org> wrote:
>>>>>>>> On the transaction side, we can start without them.
>>>>>>>
>>>>>>> +1 on omitting transactions for now.
>>>>>>>
>>>>>>> And on the missing AtomicMaps, I hope the Infinispan will
want to implement it?
>>>>>>> Would be good to eventually converge on similar featuresets
on remote
>>>>>>> vs embedded APIs.
>>>>>>>
>>>>>>> I know the embedded version relies on batching/transactions,
but I
>>>>>>> guess we could obtain a similar effect with some ad-hoc
commands in
>>>>>>> Hot Rod?
>>>>>>>
>>>>>>> Sanne
>>>>>>>
>>>>>>>>
>>>>>>>> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
>>>>>>>>> Hi,
>>>>>>>>> I'm working on the integration between HotRod and
OGM.
>>>>>>>>>
>>>>>>>>> We already have a dialect for Inifinispan and I'm
trying to follow the same
>>>>>>>>> logic.
>>>>>>>>> At the moment I'm having two problems:
>>>>>>>>>
>>>>>>>>> 1) In the Infinispan dialect we are using the
AtomicMap and the
>>>>>>>>> AtomicMapLookup but this classes don't work with
the RemoteCache. Is there
>>>>>>>>> an equivalent for HotRod?
>>>>>>>>>
>>>>>>>>> 2) As far as I know HotRod does not support
transactions. I've found a link
>>>>>>>>> to a branch on Mircea repository:
>>>>>>>>>
https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
>>>>>>>>> Is this something I could/should use?
>>>>>>>>>
>>>>>>>>> Any help is appreciated.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Davide
>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> infinispan-dev mailing list
>>>>>>>>> infinispan-dev(a)lists.jboss.org
>>>>>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> infinispan-dev mailing list
>>>>>>>> infinispan-dev(a)lists.jboss.org
>>>>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev(a)lists.jboss.org
>>>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev(a)lists.jboss.org
>>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>> Cheers,
>>>>> --
>>>>> Mircea Markus
>>>>> Infinispan lead (
www.infinispan.org)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev(a)lists.jboss.org
>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev(a)lists.jboss.org
>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> Cheers,
>>> --
>>> Mircea Markus
>>> Infinispan lead (
www.infinispan.org)
>>>
>>>
>>>
>>>
>>
>> Cheers,
>> --
>> Mircea Markus
>> Infinispan lead (
www.infinispan.org)
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev