[hibernate-dev] Have Infinispan share Hibernate's TransactionManagerLookup strategy
by Sanne Grinovero
I'm sorry, this is especially relevant to infinispan-dev, but my
previous CC was bounced.
Sanne
---------- Forwarded message ----------
From: Sanne Grinovero <sanne(a)hibernate.org>
Date: 2011/2/2
Subject: Re: [hibernate-dev] Have Infinispan share Hibernate's
TransactionManagerLookup strategy
To: Galder Zamarreño <galder(a)jboss.org>
Cc: Emmanuel Bernard <emmanuel(a)hibernate.org>, Hibernate
<hibernate-dev(a)lists.jboss.org>, infinispan -Dev List
<infinispan-dev(a)lists.jboss.org>
2011/2/2 Galder Zamarreño <galder(a)jboss.org>:
> Hmmm, I already did a similar thing for Infinispan 2LC a while back:
>
> https://github.com/galderz/hibernate-core/blob/master/hibernate-infinispa...
Nice, you're taking it from the Settings, I like your approach more.
But be aware of ISPN-883, just solved.
>
> I take Hibernate's TM and hook it into Infinispan. The way I did it is create a HibernateTransactionManagerLookup class that implements org.infinispan.transaction.lookup.TransactionManagerLookup and looks up the transaction manager with Settings.getTransactionManagerLookup().getTransactionManager(properties)
>
> Maybe we should look into consolidating this code?
Definitely; in which project would you store the common
TransactionManagerLookup wrapper?
I would be fine in depending to the second level cache module, I'd
assume that anybody having setup Infinispan and Hibernate would use
the second level cache too, or at least have no issues having to
depend on it being classpath.
We could add a simplified EmbeddedCacheManager constructor to maintain
most of the complexity of building a cache via a configuration but
overriding components via provided instances, this relates to a brief
chat I had yesterday with Vladimir about a forum thread [1]. Maybe
easies way is to expose the GlobalComponentRegistry during the
configuration phase, as overrides for internal components?
1 - http://community.jboss.org/thread/161913?tstart=0
Cheers,
Sanne
>
> Cheers,
>
> On Feb 1, 2011, at 8:17 PM, Sanne Grinovero wrote:
>
>> As discussed, I have been playing with Infinispan's configuration
>> options to have it startup using hibernate's TransactionManagerLookup.
>> It wasn't working initially because of ISPN-883, but this is now
>> solved (thanks Mircea!).
>>
>> I'm not sure yet if this should be pushed to Hibernate Search, but it
>> might be useful to have a look for OGM:
>>
>> https://github.com/Sanne/hibernate-search/commits/InfinispanTransactionMa...
>>
>> it's quite verbose, if you have suggestions I can try pushing some of
>> the complexity into Infinispan, assuming they are fine with it.
>>
>> Sanne
>> _______________________________________________
>> hibernate-dev mailing list
>> hibernate-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
>
>
14 years, 8 months
Re: [infinispan-dev] [hibernate-dev] Have Infinispan share Hibernate's TransactionManagerLookup strategy
by Galder Zamarreño
On Feb 2, 2011, at 12:41 PM, Sanne Grinovero wrote:
> 2011/2/2 Galder Zamarreño <galder(a)jboss.org>:
>> Hmmm, I already did a similar thing for Infinispan 2LC a while back:
>>
>> https://github.com/galderz/hibernate-core/blob/master/hibernate-infinispa...
>
> Nice, you're taking it from the Settings, I like your approach more.
Exactly. The idea was that 2LC users should need to configure as little as possible of Infinispan, hence focusing on configuration options within Hibernate settings. In this case, the transaction manager set up for Hibernate can be detected by the 2LC and plug Infinispan with it.
> But be aware of ISPN-883, just solved.
>
>>
>> I take Hibernate's TM and hook it into Infinispan. The way I did it is create a HibernateTransactionManagerLookup class that implements org.infinispan.transaction.lookup.TransactionManagerLookup and looks up the transaction manager with Settings.getTransactionManagerLookup().getTransactionManager(properties)
>>
>> Maybe we should look into consolidating this code?
>
> Definitely; in which project would you store the common
> TransactionManagerLookup wrapper?
> I would be fine in depending to the second level cache module, I'd
> assume that anybody having setup Infinispan and Hibernate would use
> the second level cache too, or at least have no issues having to
> depend on it being classpath.
I don't think you should depend on the 2LC module. Instead, maybe we should have an infinispan-int/ or infinispan-core/ module for shared Infinispan stuff that several Hibernate modules might benefit from?
>
> We could add a simplified EmbeddedCacheManager constructor to maintain
> most of the complexity of building a cache via a configuration but
> overriding components via provided instances, this relates to a brief
> chat I had yesterday with Vladimir about a forum thread [1]. Maybe
> easies way is to expose the GlobalComponentRegistry during the
> configuration phase, as overrides for internal components?
>
> 1 - http://community.jboss.org/thread/161913?tstart=0
>
> Cheers,
> Sanne
>
>
>
>>
>> Cheers,
>>
>> On Feb 1, 2011, at 8:17 PM, Sanne Grinovero wrote:
>>
>>> As discussed, I have been playing with Infinispan's configuration
>>> options to have it startup using hibernate's TransactionManagerLookup.
>>> It wasn't working initially because of ISPN-883, but this is now
>>> solved (thanks Mircea!).
>>>
>>> I'm not sure yet if this should be pushed to Hibernate Search, but it
>>> might be useful to have a look for OGM:
>>>
>>> https://github.com/Sanne/hibernate-search/commits/InfinispanTransactionMa...
>>>
>>> it's quite verbose, if you have suggestions I can try pushing some of
>>> the complexity into Infinispan, assuming they are fine with it.
>>>
>>> Sanne
>>> _______________________________________________
>>> hibernate-dev mailing list
>>> hibernate-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>
>> --
>> Galder Zamarreño
>> Sr. Software Engineer
>> Infinispan, JBoss Cache
>>
>>
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years, 8 months
distribution's impact on performance
by Mircea Markus
Hi,
I've enhanced Radargun with an ideal distribution function: this is a perfectly correct behaving function that is aware of the keys passed to it and makes sure that all the nodes have the same share of keys(I'll send an email soon detailing this behavior),
I've just run a 3-9 test on the cluster{01..10}.mw.lab.eng.bos.redhat.com, results attached.
The ideal distribution causes the performance to be more linear. Also for puts we have an constant 10-35% performance increase.
In the case of default CH 3 nodes, the get has a peek, this can be explained by the fact that two of the nodes moved really fast, almost as in replicated mode, as they hold most the data (90%) (see bellow num of keys on each node);.
Below is the distribution of keys/nodes[1] :
Configuration : dist-sync.xml
Cluster size: 3 -> ( 2930 2612 458)
Cluster size: 5 -> ( 1922 1930 660 2969 2519)
Cluster size: 7 -> ( 4617 1271 720 1028 4394 328 1642)
Cluster size: 9 -> ( 2586 4507 1153 556 1292 274 582 2560 4490)
Configuration : idealdistribution/dist-sync-ideal-distribution.xml
Cluster size: 3 -> ( 2000 2000 2000)
Cluster size: 5 -> ( 2000 2000 2000 2000 2000)
Cluster size: 7 -> ( 2000 2000 2000 2000 2000 2000 2000)
Cluster size: 9 -> ( 2000 2000 2000 2000 2000 2000 2000 2000 2000)
[1] this can now be obtained by running (./bin/dist.sh - just added).
Cheers,
Mircea
14 years, 8 months
RELAY and Infinispan
by Bela Ban
RELAY [1] bridges separate local clusters into large virtual global
clusters, e.g. {A,B,C} and {X,Y,Z} into {A,B,C,X,Y,Z}.
This new global view has local members A, B and C and proxies X, Y, Z on
the {A,B,C} cluster, and vice versa.
When A sends a message, it is forwarded to the other cluster, but the
sender is wrapped into a ProxyAddress A/X. This means that the original
sender is A, but the local address in {X,Y,Z} is X. However, there is a
problem !
A ProxyAddress A/X's hashCode(), equals() and compareTo() use *X*, which
means that if we add X, Y, Z and A/X into a *HashSet* (or HashMap), X
and A/X map to the same address ! So if you have a View (on X) with
{A/X, B/X, C/X, X, Y, Z}, and add all members to a HashMap, we'll only
have a size of 3 !!
If we used *A* in A/X for hashCode(), equals() and compareTo(), then
this problem would not exist, however, this means that we would now
'know' about the other cluster, and therefore digest handling, flow
control etc would happen across both clusters, which is something we
don't want; we want the 2 local cluster to be completely autonomous !
E.g. we don't want cluster {X,Y,Z} to block on credits from B in the
other cluster...
So I was thinking of passing the local view {X,Y,Z} up to Infinispan
instead of the global view {A/X,B/X,C/X,X,Y,Z}. This would mean
Infinispan would know only about A, B and C in the cluster {A,B,C}, and
about X, Y and Z in {X,Y,Z}.
Now, I want to be able to have backups of keys from {A,B,C} in {X,Y,Z}
in DIST mode, e.g. with numOwners, key "name" should be stored on A and
C in the local cluster, and on Z in the remote cluster.
To do this, the consistent hash function would know about the local
cluster {A,B,C} and the remote cluster {X,Y,Z}. It would get view
changes by hooking into RELAY.
So when there is a local("name", 3), it would return A, C, A/Z, causing
Infinispan to fetch the data from or store the data to A,C and Z.
This should work fine I guess, because when Infinispan tries to send
data to A/Z, JGroups's RELAY will forward the message to the remote Z.
Q: does Infinispan assert that A, C and Z are in the local view, when
distributing data ? Because then my scheme above wouldn't work...
The other question I have is, can I force Infinispan to do a rehashing ?
For example, when the consistent hash function in {A,B,C} gets a view
change for the remote view, going from {X} to {X,Y}, then I'd like
Infinispan to do a rehashing, checking whether all keys are in the
correct location and - if not - call into the consistent hash function
to compute the new locations...
Thoughts ?
[1] http://www.jgroups.org/manual/html/user-advanced.html#RelayAdvanced
--
Bela Ban
Lead JGroups / Clustering Team
JBoss
14 years, 8 months
Delta operations
by Sanne Grinovero
Hello,
I'm going to implement some custom Delta operations, I'd appreciate
some explanations on the expectations of the API.
I got puzzled about the method
org.infinispan.atomic.Operation.rollback(Map<K, V>)
it doesn't seem to be ever invoked, but still some Operation
implementations collect information to properly handle an eventual
rollback request (such as ClearOperation).
In case it's not needed, can we remove the method and all unneded
logic in each operation?
In case it's needed, it looks like the existing operation
implementations are not able to honor a rollback invocation after
being marshalled & ummarshalled, as most needed information is not
externalized; could anybody spend two words on it?
thanks,
Sanne
14 years, 8 months
DistributionManagerImpl.rehash()
by Bela Ban
I see that this method calls MembershipArithmetic.getMemberJoined(),
which returns the newly joined member. This is done by removing the
existing members from the new members, and returning the first element
of the remaining list.
However, when we have view V1={A,B} and V2={A,B,C,D}, then the method
getMemberJoined() returns C, but skips D.
It seems to me that this logic is based on the assumption that JGroups
only ever joins and removes 1 member at a time, but this is not correct,
as JGroups can join multiple new members, or remove multiple members at
once, too.
Can you guys take a look ?
--
Bela Ban
Lead JGroups / Clustering Team
JBoss
14 years, 8 months
