[infinispan-dev] Integration between HotRod and OGM

Mircea Markus mmarkus at redhat.com
Tue Jan 21 11:57:50 EST 2014 

On Jan 21, 2014, at 4:08 PM, Sanne Grinovero <sanne at infinispan.org> wrote:

> Hi Mircea,
> could you explain how Grouping is different than AtomicMaps ?

Here's the original thread where this has been discussed: http://goo.gl/WNs6KY 
I would add to that that the AtomicMap requires transactions, which grouping doesn't. Also in the context of hotrod (i.e. this email thread) FGAM is a structure that would be harder to migrate over 

> I understand you're all suggesting to move to AtomicMaps as "the
> implementation is better"

we're suggesting to move from the AM to grouping

> but is that an implementation detail, or how
> is it inherently different so that we can build something more
> reliable on it?

They both are doing pretty much the same thing, so it's more a matter of choosing one instead of the other. Grouping fits way nicer into the picture, both as a concept and the  implementation.

> 
>> From the limited knowledge I have in this area, I have been assuming -
> since they have very similar properties - that this was essentially a
> different syntax to get to the same semantics but obviously I'm wrong.
> 
> It would be especially helpfull to have a clear comparison on the
> different semantics in terms of transactions, atomicity and visibility
> of state across the three kinds: AtomicMaps, FineGrainedAtomicMaps,
> Grouping.
> 
> Let's also keep in mind that Hibernate OGM uses a carefully selected
> combination of *both* AtomicMap and FGAM instances - depending on the
> desired semantics we want to achieve, so since those two where clearly
> different and we actually build on those differences - I'm not seeing
> how we could migrate two different things to the same construct
> without having to move "fishy locking details" out of Infinispan but
> in OGM, and I wouldn't be too happy with that as such logic would
> belong in Infinispan to provide.

I wasn't aware that OGM still uses AtomicMap, but the only case in which I imagine that would be useful is in order to force a lock on the whole AtomicMap. Is that so or some other aspect that I'm missing? 

> 
> - Sanne
> 
> 
> On 21 January 2014 15:07, Mircea Markus <mmarkus at redhat.com> wrote:
>> Hi Emmanuel,
>> 
>> Just had a good chat with Davide on this and one solution to overcome the shortcoming you mentioned in the above email would be to enhance the hotrod client to support grouping:
>> 
>> RemoteClient.put(G g, K k, V v); //first param is the group
>> RemoteClinet.getGroup(G g) : Map<K,V>;
>> 
>> It requires an enhancement on our local grouping API: EmbeddedCache.getGroup(G). This is something useful for us in a broader context, as it is the step needed to be able to deprecated AtomicMaps and get suggest them being replaced with Grouping.
>> 
>> This approach still has some limitations compared to the current embedded integration:
>> - performance caused by the lack of transactions: this means increased TCP chattiness between the Hot Rod client and the server.
>> - you'd have to handle atomicity, potentially by retrying an operation
>> 
>> What do you think?
>> 
>> 
>> On Dec 3, 2013, at 3:10 AM, Mircea Markus <mmarkus at redhat.com> wrote:
>> 
>>> 
>>> On Nov 19, 2013, at 10:22 AM, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
>>> 
>>>> It's an interesting approach that would work fine-ish for entities
>>>> assuming the Hot Rod client is multi threaded and assuming the client
>>>> uses Future to parallelize the calls.
>>> 
>>> The Java Hotrod client is both multithreaded and exposes an Async API.
>>> 
>>>> 
>>>> But it won't work for associations as we have them designed today.
>>>> Each association - or more precisely the query results to go from an
>>>> entity A1 to the list of entities B associated to it - is represented by
>>>> an AtomicMap.
>>>> Each entry in this map does correspond to an entry in the association.
>>>> 
>>>> While we can "guess" the column names and build from the metadata the
>>>> list of composed keys for entities, we cannot do the same for
>>>> associations as the key is literally the (composite) id of the
>>>> association and we cannot guess that most of the time (we can in very
>>>> pathological cases).
>>>> We could imagine that we list the association row keys in a special
>>>> entry to work around that but this approach is just as problematic and
>>>> is conceptually the same.
>>>> The only solution would be to lock the whole association for each
>>>> operation and I guess impose some versioning / optimistic lock.
>>>> 
>>>> That is not a pattern that scales sufficiently from my experience.
>>> 
>>> I think so too :-)
>>> 
>>>> That's the problem with interconnected data :)
>>>> 
>>>> Emmanuel
>>>> 
>>>> On Mon 2013-11-18 23:05, Mircea Markus wrote:
>>>>> Neither the grouping API nor the AtomicMap work over hotrod.
>>>>> Between the grouping API and AtomicMap, I think the one that would make more sense migrating is the grouping API.
>>>>> One way or the other, I think the hotrod protocol would require an enhancement - mind raising a JIRA for that?
>>>>> For now I guess you can sacrifice performance and always sending the entire object across on every update instead of only the deltas?
>>>>> 
>>>>> On Nov 18, 2013, at 9:56 AM, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
>>>>> 
>>>>>> Someone mentioned the grouping API as some sort of alternative to
>>>>>> AtomicMap. Maybe we should use that?
>>>>>> Note that if we don't have a fine-grained approach we will need to
>>>>>> make sure we *copy* the complex data structure upon reads to mimic
>>>>>> proper transaction isolation.
>>>>>> 
>>>>>> On Tue 2013-11-12 15:14, Sanne Grinovero wrote:
>>>>>>> On 12 November 2013 14:54, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
>>>>>>>> On the transaction side, we can start without them.
>>>>>>> 
>>>>>>> +1 on omitting transactions for now.
>>>>>>> 
>>>>>>> And on the missing AtomicMaps, I hope the Infinispan will want to implement it?
>>>>>>> Would be good to eventually converge on similar featuresets on remote
>>>>>>> vs embedded APIs.
>>>>>>> 
>>>>>>> I know the embedded version relies on batching/transactions, but I
>>>>>>> guess we could obtain a similar effect with some ad-hoc commands in
>>>>>>> Hot Rod?
>>>>>>> 
>>>>>>> Sanne
>>>>>>> 
>>>>>>>> 
>>>>>>>> On Tue 2013-11-12 14:34, Davide D'Alto wrote:
>>>>>>>>> Hi,
>>>>>>>>> I'm working on the integration between HotRod and OGM.
>>>>>>>>> 
>>>>>>>>> We already have a dialect for Inifinispan and I'm trying to follow the same
>>>>>>>>> logic.
>>>>>>>>> At the moment I'm having two problems:
>>>>>>>>> 
>>>>>>>>> 1) In the Infinispan dialect we are using the AtomicMap and the
>>>>>>>>> AtomicMapLookup but this classes don't work with the RemoteCache. Is there
>>>>>>>>> an equivalent for HotRod?
>>>>>>>>> 
>>>>>>>>> 2) As far as I know HotRod does not support transactions. I've found a link
>>>>>>>>> to a branch on Mircea repository:
>>>>>>>>> https://github.com/mmarkus/ops_over_hotrod/wiki/Usage-guide
>>>>>>>>> Is this something I could/should use?
>>>>>>>>> 
>>>>>>>>> Any help is appreciated.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Davide
>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> infinispan-dev mailing list
>>>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> infinispan-dev mailing list
>>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> 
>>>>> Cheers,
>>>>> --
>>>>> Mircea Markus
>>>>> Infinispan lead (www.infinispan.org)
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> Cheers,
>>> --
>>> Mircea Markus
>>> Infinispan lead (www.infinispan.org)
>>> 
>>> 
>>> 
>>> 
>> 
>> Cheers,
>> --
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)

More information about the infinispan-dev mailing list