Creating ByteArrayKey to solve client/server hashing differences in HotRod
by galder＠redhat.com
Hi all,
Re: https://jira.jboss.org/browse/ISPN-491
It's apparent that client and server were not calculating hashcode on the same thing. We don't want to force clients to do hash on Java classes cos that would make clients language independent and we can't use byte[] keys in the cache due to equality issues (array equality is based on reference, not Arrays.equals). So, we're forced to create a ByteArrayKey class in core/ to basically mimic what HotRod's CacheKey does in a more generic way. Diablo-D3 suggested using nio.ByteBuffer to wrap the key around, but this sounds like a heavy handed approach for simply having byte array keys that compare as expected, i.e. BB class brings in extra information such as position, limit...etc that has no use and would only take memory space.
We'll modify MurmurHash2.hash(Object o) so that if a ByteArrayKey is passed, it takes the underlying byte[] and calculates the hash on that. This way we'll make sure that both clients and servers hash on the same thing.
However, I think the long term goal should be https://jira.jboss.org/browse/ISPN-492. If we had underlying maps that did array equality based on contents, we would not have these problems and would not need for auxiliary classes.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
15 years, 3 months
Re: [infinispan-dev] Optimising default cache access in Hot Rod
by galder＠jboss.org
----- "Mircea Markus" <mircea.markus(a)jboss.com> wrote:
> On 9 Jun 2010, at 10:13, galder(a)redhat.com wrote:
>
> > Hi guys,
> >
> > This came to my mind while sleeping.
> You definitely over-geeked me!
> > While I was profiling the Hot Rod server, I noticed that some time
> (3%) was being used reading the string cache name from the wire and
> transforming it to UTF-8.
> How much of the time is being used to convert the byte[] to utf 8
> String? At least for writes we can use an cached byte[] rather than
> converting it each time.
About half. 1.5% reading the byte array and 1.5% converting it to UTF-8. The problem is that I need to get a cache out of it, so somehow you need to map that byte[] to the cache that it maps to. I have doubts whether creating a UTF-8 string and then getting a cache out of it is worse than having to maintain a mapping between byte[] and the target cache.
> >
> > Now, in the test I was acting against the default cache which we had
> established to be requests sent to "___defaultcache" would be
> considered to be directed to the default cache. However, I'm thinking
> that this is waste of space. We could make things even quicker when
> default cache is used by simply assuming that if the length of the
> String is 0, followed by nothing, then the default cache will be used.
> If a length other than 0 is passed, the String follows and we have to
> read it.
> +1
I've created: https://jira.jboss.org/browse/ISPN-489
> >
> > WDYT?
> > --
> > Galder Zamarreño
> > Sr. Software Engineer
> > Infinispan, JBoss Cache
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
15 years, 4 months
Re: [infinispan-dev] cache.put(), cache.stop() and suspected member
by galder＠jboss.org
First of all, this is a user question and you should direct them to http://community.jboss.org/en/infinispan?view=discussions&start=0
This list is to discuss development of Infinispan itself.
So, if you can open a user forum question, we can follow the conversation there :)
----- "Manni Bucau Romain" <romain.mannibucau(a)atosorigin.com> wrote:
> hi,
>
> i'm trying to use infinispan but when i'm running twice my program i
> have a suspected member exception.
>
> I'm doing it :
> 1) i'm running the program => P1
> 2) i'm running it again => P2
> 3) in P1 i'm writing a=>b
> 4) before the end of the a=>b command i'm closing P1 ('q')
>
> here is the code :
>
> public class InfiniSpanDemo {
> public static void main(String[] args) throws
> NotSupportedException, SystemException, SecurityException,
> IllegalStateException, RollbackException, HeuristicMixedException,
> HeuristicRollbackException, IOException {
> hello();
> }
>
> public static void hello() throws NotSupportedException,
> SystemException, SecurityException, IllegalStateException,
> RollbackException, HeuristicMixedException,
> HeuristicRollbackException, IOException {
> CacheManager mgr = new
> DefaultCacheManager(InfiniSpanDemo.class.getResourceAsStream("/config/infinispan-config.xml"));
> Cache<String, String> cache =
> mgr.getCache("distributedCache");
>
> do {
> System.out.println("type 'key=>value' to set a value or
> 'key?' to read a value to test a value or 'q' to quit\n");
> Scanner in = new Scanner(System.in);
> String action = in.nextLine();
>
> if (action.equalsIgnoreCase("q")) {
> break;
> } else if (action.contains("=>")) {
> String[] s = action.split("=>");
> cache.put(s[0], s[1]);
> for (int i = 0; i < 1000000; i++) {
> cache.put(i + "-" + s[0], i + "_" + s[1]);
> }
> } else if (action.length() > 1 &&
> action.charAt(action.length() - 1) == '?') {
> String s =action.substring(0, action.length() - 1);
> System.out.println(cache.get(s));
> } else {
> System.out.println("retry");
> }
> } while(true);
>
> cache.stop();
> mgr.stop();
> }
> }
>
> and here the exception:
> GRAVE: Execution error:
> org.infinispan.remoting.transport.jgroups.SuspectException: Suspected
> member: Jalapeno-25790
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:419)
> at
> org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:100)
> at
> org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:124)
> at
> org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:229)
> at
> org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:216)
> at
> org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:199)
> at
> org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:192)
> at
> org.infinispan.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:114)
> at
> org.infinispan.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:78)
> at
> org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
> at
> org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at
> org.infinispan.interceptors.LockingInterceptor.visitPutKeyValueCommand(LockingInterceptor.java:198)
> at
> org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
> at
> org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at
> org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at
> org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:57)
> at
> org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
> at
> org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at
> org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:183)
> at
> org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:132)
> at
> org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
> at
> org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at
> org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:58)
> at
> org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:39)
> at
> org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:57)
> at
> org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
> at
> org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:271)
> at org.infinispan.CacheDelegate.put(CacheDelegate.java:411)
> at org.infinispan.CacheSupport.put(CacheSupport.java:28)
> at pack.InfiniSpanDemo.hello(InfiniSpanDemo.java:36)
>
>
> ________________________________
>
> Ce message et les pi?ces jointes sont confidentiels et r?serv?s ?
> l'usage exclusif de ses destinataires. Il peut ?galement ?tre prot?g?
> par le secret professionnel. Si vous recevez ce message par erreur,
> merci d'en avertir imm?diatement l'exp?diteur et de le d?truire.
> L'int?grit? du message ne pouvant ?tre assur?e sur Internet, la
> responsabilit? du groupe Atos Origin ne pourra ?tre recherch?e quant
> au contenu de ce message. Bien que les meilleurs efforts soient faits
> pour maintenir cette transmission exempte de tout virus, l'exp?diteur
> ne donne aucune garantie ? cet ?gard et sa responsabilit? ne saurait
> ?tre recherch?e pour tout dommage r?sultant d'un virus transmis.
>
> This e-mail and the documents attached are confidential and intended
> solely for the addressee; it may also be privileged. If you receive
> this e-mail in error, please notify the sender immediately and destroy
> it. As its integrity cannot be secured on the Internet, the Atos
> Origin group liability cannot be triggered for the message content.
> Although the sender endeavours to maintain a computer virus-free
> network, the sender does not warrant that this transmission is
> virus-free and will not be liable for any damages resulting from any
> virus transmitted.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
15 years, 4 months
Optimising default cache access in Hot Rod
by galder＠redhat.com
Hi guys,
This came to my mind while sleeping. While I was profiling the Hot Rod server, I noticed that some time (3%) was being used reading the string cache name from the wire and transforming it to UTF-8.
Now, in the test I was acting against the default cache which we had established to be requests sent to "___defaultcache" would be considered to be directed to the default cache. However, I'm thinking that this is waste of space. We could make things even quicker when default cache is used by simply assuming that if the length of the String is 0, followed by nothing, then the default cache will be used. If a length other than 0 is passed, the String follows and we have to read it.
WDYT?
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
15 years, 4 months
cache.put(), cache.stop() and suspected member
by Manni Bucau Romain
hi,
i'm trying to use infinispan but when i'm running twice my program i have a suspected member exception.
I'm doing it :
1) i'm running the program => P1
2) i'm running it again => P2
3) in P1 i'm writing a=>b
4) before the end of the a=>b command i'm closing P1 ('q')
here is the code :
public class InfiniSpanDemo {
public static void main(String[] args) throws NotSupportedException, SystemException, SecurityException, IllegalStateException, RollbackException, HeuristicMixedException, HeuristicRollbackException, IOException {
hello();
}
public static void hello() throws NotSupportedException, SystemException, SecurityException, IllegalStateException, RollbackException, HeuristicMixedException, HeuristicRollbackException, IOException {
CacheManager mgr = new DefaultCacheManager(InfiniSpanDemo.class.getResourceAsStream("/config/infinispan-config.xml"));
Cache<String, String> cache = mgr.getCache("distributedCache");
do {
System.out.println("type 'key=>value' to set a value or 'key?' to read a value to test a value or 'q' to quit\n");
Scanner in = new Scanner(System.in);
String action = in.nextLine();
if (action.equalsIgnoreCase("q")) {
break;
} else if (action.contains("=>")) {
String[] s = action.split("=>");
cache.put(s[0], s[1]);
for (int i = 0; i < 1000000; i++) {
cache.put(i + "-" + s[0], i + "_" + s[1]);
}
} else if (action.length() > 1 && action.charAt(action.length() - 1) == '?') {
String s =action.substring(0, action.length() - 1);
System.out.println(cache.get(s));
} else {
System.out.println("retry");
}
} while(true);
cache.stop();
mgr.stop();
}
}
and here the exception:
GRAVE: Execution error:
org.infinispan.remoting.transport.jgroups.SuspectException: Suspected member: Jalapeno-25790
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:419)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:100)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:124)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:229)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:216)
at org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:199)
at org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:192)
at org.infinispan.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:114)
at org.infinispan.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:78)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.LockingInterceptor.visitPutKeyValueCommand(LockingInterceptor.java:198)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:57)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:183)
at org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:132)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:58)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:39)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:57)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:271)
at org.infinispan.CacheDelegate.put(CacheDelegate.java:411)
at org.infinispan.CacheSupport.put(CacheSupport.java:28)
at pack.InfiniSpanDemo.hello(InfiniSpanDemo.java:36)
________________________________
Ce message et les pi?ces jointes sont confidentiels et r?serv?s ? l'usage exclusif de ses destinataires. Il peut ?galement ?tre prot?g? par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir imm?diatement l'exp?diteur et de le d?truire. L'int?grit? du message ne pouvant ?tre assur?e sur Internet, la responsabilit? du groupe Atos Origin ne pourra ?tre recherch?e quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'exp?diteur ne donne aucune garantie ? cet ?gard et sa responsabilit? ne saurait ?tre recherch?e pour tout dommage r?sultant d'un virus transmis.
This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.
15 years, 4 months
Re: [infinispan-dev] infinispan and affinity
by galder＠jboss.org
Right, that's something you can already do with default Hibernate 2LC Infinispan settings which are explained in detail in http://community.jboss.org/docs/DOC-14105
I still do not understand your request to get hold of the 2LC cache. Try the default settings and if it doesn't work, please explain further.
----- "Romain Manni-Bucau" <manni.romain(a)gmail.com> wrote:
> I have one database and 5 computers. I want to limit db access which
> are slow.
>
> 2010/6/3, Galder Zamarreno <galder(a)jboss.org>:
> >
> > ----- "Mircea Markus" <mircea.markus(a)jboss.com> wrote:
> >
> >> Adding isp dev in the reply.
> >>
> >> On 3 Jun 2010, at 00:06, Romain Manni-Bucau wrote:
> >>
> >> > Hi,
> >> >
> >> > I would like to use infinispan (for data griding) with gridgain
> (for
> >> grid computing). My problem is to use the same "key" to run job
> where
> >> datas are located.
> >> >
> >> > I read your article about affinity but i don't understand 2
> things
> >> :
> >> > how to use it when infinispan is configured with hibernate as L2
> >> cache ?
> >> Galder might help on this - what we need to know is how do we get
> a
> >> reference to the cache used as L2 in Hibernate.
> >
> > Clients do not get access to underlying Infinispan cache instance
> and DIST
> > is not really a recommended mode for an L2 cache. What's your use
> case?
> >
> >> > how to use it correctly : a) i create a key K (generated) to be
> sure
> >> to keep my date on a node, b) if i do : put(K, value), i have to
> keep
> >> a map with my own key (understandable) to use generated key so i
> have
> >> a second "cache"
> >> Your scenario is a bit out of the scope of KeyAffinityService and
> will
> >> be implemented with https://jira.jboss.org/browse/ISPN-359
> >> An example use case for KAS is for web sessions: when you create a
> >> session id (you don't have to enforce a certain value on the
> session)
> >> you do want to create it so that it maps (distribution wise) on
> the
> >> local node. This way next calls from client (which uses session
> >> affinity) are local (at least as long as the cluster doesn't
> change
> >> its topology, but are optimists :) ).
> >> Now returning to your example, you can use an replicated cache
> for
> >> keeping the mapping (myMap in your example). This would have the
> >> drawback of one additional remote call at creation time - but all
> >> subsequent gets will be local, so if the read/write ratio is high
> >> (many reads on the same key) this will pay off.
> >> More, please be aware that a topology change will cause the
> existing
> >> collocated objects to be redistributed across nodes - even if you
> >> resurrect a failed node still applies.
> >> HTH,
> >> Mircea
> >> > ex :
> >> > K = service.getKeyForAddress(...);
> >> > myMap.put("myHumanKey", K);
> >> > cache.put(K, value);
> >> > readValue = cache.get(myMap.get("myHumanKey"));
> >> >
> >> > BUT
> >> > i have to put myMap in the cache to be able to read it on all
> nodes
> >> ...
> >> >
> >> > So as you can see i don't understand something so if you can
> help
> >> me. I can go on irc i you prefer, give me an hour (i'm in france).
> >> >
> >> > Regards,
> >> >
> >> > Romain Manni-Bucau
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev(a)lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
>
> --
> Envoyé avec mon mobile
>
> Romain Manni-Bucau
15 years, 4 months
Re: [infinispan-dev] infinispan and affinity
by Mircea Markus
Adding isp dev in the reply.
On 3 Jun 2010, at 00:06, Romain Manni-Bucau wrote:
> Hi,
>
> I would like to use infinispan (for data griding) with gridgain (for grid computing). My problem is to use the same "key" to run job where datas are located.
>
> I read your article about affinity but i don't understand 2 things :
> how to use it when infinispan is configured with hibernate as L2 cache ?
Galder might help on this - what we need to know is how do we get a reference to the cache used as L2 in Hibernate.
> how to use it correctly : a) i create a key K (generated) to be sure to keep my date on a node, b) if i do : put(K, value), i have to keep a map with my own key (understandable) to use generated key so i have a second "cache"
Your scenario is a bit out of the scope of KeyAffinityService and will be implemented with https://jira.jboss.org/browse/ISPN-359
An example use case for KAS is for web sessions: when you create a session id (you don't have to enforce a certain value on the session) you do want to create it so that it maps (distribution wise) on the local node. This way next calls from client (which uses session affinity) are local (at least as long as the cluster doesn't change its topology, but are optimists :) ).
Now returning to your example, you can use an replicated cache for keeping the mapping (myMap in your example). This would have the drawback of one additional remote call at creation time - but all subsequent gets will be local, so if the read/write ratio is high (many reads on the same key) this will pay off.
More, please be aware that a topology change will cause the existing collocated objects to be redistributed across nodes - even if you resurrect a failed node still applies.
HTH,
Mircea
> ex :
> K = service.getKeyForAddress(...);
> myMap.put("myHumanKey", K);
> cache.put(K, value);
> readValue = cache.get(myMap.get("myHumanKey"));
>
> BUT
> i have to put myMap in the cache to be able to read it on all nodes ...
>
> So as you can see i don't understand something so if you can help me. I can go on irc i you prefer, give me an hour (i'm in france).
>
> Regards,
>
> Romain Manni-Bucau
15 years, 4 months
CR1, BETA2 and Branching trunk
by Manik Surtani
Guys,
So we've decided to release Radegast BETA2 tomorrow rather than CR1, as it gives folks more time to feed back stuff. Hopefully there won't be any work done on Radegast between BETA2 and CR1 except fixing issues people may report.
And for this purpose, I propose branching trunk to a 4.1 branch, after we tag BETA2:
http://anonsvn.jboss.org/repos/infinispan/branches/4.1.x
This way trunk can contain the 5.0 codebase, and people can start working on 5.0 features.
Thoughts, opinions?
Cheers,
Manik
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
15 years, 4 months
Rehashing on leave
by Vladimir Blagojevic
Hey,
Our rehashing tasks assume that only one member at the time can leave (DistributionManagerImpl#rehash) between the subsequent views. JGroups does not enforce this model - i.e multiple members can leave/crash between subsequent views. Unless there is some disconnect that I am not aware of we have to handle multiple leaves/crashes in rehash.
Vladimir
15 years, 4 months
