11:04 a.m.
Hi infinispan-dev
I'm running 2 processes with 2 distributed caches each, standard
jgroups-tcp configutation. Both caches have a local dat file which is
loaded during startup, passivation is true. Each cache contains ~20Mio.
entries.
I'm writing with async put, peak is over 10'000 entries per second. It
performs well.
cache.getAdvancedCache()
.withFlags(Flag.SKIP_REMOTE_LOOKUP, Flag.SKIP_CACHE_LOAD)
.putIfAbsentAsync(Long.valueOf(a), Long.valueOf(b));
As soon as I launch a 3rd process to join the 2 caches, I'm getting the
following exception (see below).
Does anyone know what I need to tune. It looks like the OS doesn't offer
enough resources, or am I wrong? The server has plenty of RAM and CPU's.
I'm launching without -Xmx, but with -XX:+UseG1GC.
Any help is much appreciated
-andreas
Exception in thread "main"
org.infinispan.manager.EmbeddedCacheManagerStartupException:
org.infinispan.commons.CacheException: Unable to invoke method public
void org.infinispan.remoting.transport.jgroups.JGroupsTransport.start()
on object of type JGroupsTransport
at
org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:243)
at
org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:573)
at
org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
at
org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
at ch.nexustelecom.lbd.engine.ImsiCache.init(ImsiCache.java:49)
at
ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
at
ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
at
ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
at
ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
Caused by: org.infinispan.commons.CacheException: Unable to invoke
method public void
org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on
object of type JGroupsTransport
at
org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
at
org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
at
org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
at
org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
at
org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
at
org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:221)
... 8 more
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at
org.jgroups.protocols.FD_SOCK$ServerSocketHandler.start(FD_SOCK.java:1006)
at
org.jgroups.protocols.FD_SOCK$ServerSocketHandler.<init>(FD_SOCK.java:999)
at org.jgroups.protocols.FD_SOCK.init(FD_SOCK.java:188)
at
org.jgroups.stack.ProtocolStack.initProtocolStack(ProtocolStack.java:860)
at org.jgroups.stack.ProtocolStack.setup(ProtocolStack.java:481)
at org.jgroups.JChannel.init(JChannel.java:848)
at org.jgroups.JChannel.<init>(JChannel.java:159)
at org.jgroups.JChannel.<init>(JChannel.java:129)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.buildChannel(JGroupsTransport.java:381)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.initChannel(JGroupsTransport.java:286)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.initChannelAndRPCDispatcher(JGroupsTransport.java:330)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.start(JGroupsTransport.java:189)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
... 13 more
This email and any attachment may contain confidential information which is intended for
use only by the addressee(s) named above. If you received this email by mistake, please
notify the sender immediately, and delete the email from your system. You are prohibited
from copying, disseminating or otherwise using the email or any attachment.
11:47 a.m.
Hi,
my guess is that you're running Linux or OSX?
You might need to reconfigure your OS to allow running more threads,
we have a note about that here:
http://infinispan.org/docs/7.0.x/contributing/contributing.html#_running_...
Sanne
On 26 November 2014 at 16:04, Andreas Kruthoff
<andreas.kruthoff(a)nexustelecom.com> wrote:
Hi infinispan-dev
I'm running 2 processes with 2 distributed caches each, standard
jgroups-tcp configutation. Both caches have a local dat file which is
loaded during startup, passivation is true. Each cache contains ~20Mio.
entries.
I'm writing with async put, peak is over 10'000 entries per second. It
performs well.
cache.getAdvancedCache()
.withFlags(Flag.SKIP_REMOTE_LOOKUP, Flag.SKIP_CACHE_LOAD)
.putIfAbsentAsync(Long.valueOf(a), Long.valueOf(b));
As soon as I launch a 3rd process to join the 2 caches, I'm getting the
following exception (see below).
Does anyone know what I need to tune. It looks like the OS doesn't offer
enough resources, or am I wrong? The server has plenty of RAM and CPU's.
I'm launching without -Xmx, but with -XX:+UseG1GC.
Any help is much appreciated
-andreas
Exception in thread "main"
org.infinispan.manager.EmbeddedCacheManagerStartupException:
org.infinispan.commons.CacheException: Unable to invoke method public
void org.infinispan.remoting.transport.jgroups.JGroupsTransport.start()
on object of type JGroupsTransport
at
org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:243)
at
org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:573)
at
org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
at
org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
at ch.nexustelecom.lbd.engine.ImsiCache.init(ImsiCache.java:49)
at
ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
at
ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
at
ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
at
ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
Caused by: org.infinispan.commons.CacheException: Unable to invoke
method public void
org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on
object of type JGroupsTransport
at
org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
at
org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
at
org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
at
org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
at
org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
at
org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:221)
... 8 more
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at
org.jgroups.protocols.FD_SOCK$ServerSocketHandler.start(FD_SOCK.java:1006)
at
org.jgroups.protocols.FD_SOCK$ServerSocketHandler.<init>(FD_SOCK.java:999)
at org.jgroups.protocols.FD_SOCK.init(FD_SOCK.java:188)
at
org.jgroups.stack.ProtocolStack.initProtocolStack(ProtocolStack.java:860)
at org.jgroups.stack.ProtocolStack.setup(ProtocolStack.java:481)
at org.jgroups.JChannel.init(JChannel.java:848)
at org.jgroups.JChannel.<init>(JChannel.java:159)
at org.jgroups.JChannel.<init>(JChannel.java:129)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.buildChannel(JGroupsTransport.java:381)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.initChannel(JGroupsTransport.java:286)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.initChannelAndRPCDispatcher(JGroupsTransport.java:330)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.start(JGroupsTransport.java:189)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
... 13 more
This email and any attachment may contain confidential information which is intended for
use only by the addressee(s) named above. If you received this email by mistake, please
notify the sender immediately, and delete the email from your system. You are prohibited
from copying, disseminating or otherwise using the email or any attachment.
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
5:42 a.m.
RedHat Linux.
I've changed the parameters as suggested in the article. And it helped,
thank you!
I've to observe the system a bit longer, but it looks good so far.
I _needed_ to run the caches with mode="ASYNC", as I got timeout
problems with jgroups mode=GET_ALL... when launching all 3 processes.
I didn't see how to change that within the jgroups configuration, but I
switched to ASYNC in Infinispan, and the timeout messages were gone.
thx!
-andreas
On 11/26/2014 05:47 PM, Sanne Grinovero wrote:
Hi,
my guess is that you're running Linux or OSX?
You might need to reconfigure your OS to allow running more threads,
we have a note about that here:
http://infinispan.org/docs/7.0.x/contributing/contributing.html#_running_...
Sanne
On 26 November 2014 at 16:04, Andreas Kruthoff
<andreas.kruthoff(a)nexustelecom.com> wrote:
> Hi infinispan-dev
>
>
> I'm running 2 processes with 2 distributed caches each, standard
> jgroups-tcp configutation. Both caches have a local dat file which is
> loaded during startup, passivation is true. Each cache contains ~20Mio.
> entries.
>
> I'm writing with async put, peak is over 10'000 entries per second. It
> performs well.
>
> cache.getAdvancedCache()
> .withFlags(Flag.SKIP_REMOTE_LOOKUP, Flag.SKIP_CACHE_LOAD)
> .putIfAbsentAsync(Long.valueOf(a), Long.valueOf(b));
>
> As soon as I launch a 3rd process to join the 2 caches, I'm getting the
> following exception (see below).
>
> Does anyone know what I need to tune. It looks like the OS doesn't offer
> enough resources, or am I wrong? The server has plenty of RAM and CPU's.
> I'm launching without -Xmx, but with -XX:+UseG1GC.
>
>
> Any help is much appreciated
>
> -andreas
>
>
> Exception in thread "main"
> org.infinispan.manager.EmbeddedCacheManagerStartupException:
> org.infinispan.commons.CacheException: Unable to invoke method public
> void org.infinispan.remoting.transport.jgroups.JGroupsTransport.start()
> on object of type JGroupsTransport
> at
>
org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:243)
> at
>
org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:573)
> at
> org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
> at
> org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
> at ch.nexustelecom.lbd.engine.ImsiCache.init(ImsiCache.java:49)
> at
>
ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
> at
> ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
> at
> ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
> at
> ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
> Caused by: org.infinispan.commons.CacheException: Unable to invoke
> method public void
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on
> object of type JGroupsTransport
> at
> org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
> at
>
org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> at
>
org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> at
>
org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> at
>
org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> at
>
org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:221)
> ... 8 more
> Caused by: java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:714)
> at
> org.jgroups.protocols.FD_SOCK$ServerSocketHandler.start(FD_SOCK.java:1006)
> at
> org.jgroups.protocols.FD_SOCK$ServerSocketHandler.<init>(FD_SOCK.java:999)
> at org.jgroups.protocols.FD_SOCK.init(FD_SOCK.java:188)
> at
> org.jgroups.stack.ProtocolStack.initProtocolStack(ProtocolStack.java:860)
> at org.jgroups.stack.ProtocolStack.setup(ProtocolStack.java:481)
> at org.jgroups.JChannel.init(JChannel.java:848)
> at org.jgroups.JChannel.<init>(JChannel.java:159)
> at org.jgroups.JChannel.<init>(JChannel.java:129)
> at
>
org.infinispan.remoting.transport.jgroups.JGroupsTransport.buildChannel(JGroupsTransport.java:381)
> at
>
org.infinispan.remoting.transport.jgroups.JGroupsTransport.initChannel(JGroupsTransport.java:286)
> at
>
org.infinispan.remoting.transport.jgroups.JGroupsTransport.initChannelAndRPCDispatcher(JGroupsTransport.java:330)
> at
>
org.infinispan.remoting.transport.jgroups.JGroupsTransport.start(JGroupsTransport.java:189)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> ... 13 more
>
> This email and any attachment may contain confidential information which is intended
for use only by the addressee(s) named above. If you received this email by mistake,
please notify the sender immediately, and delete the email from your system. You are
prohibited from copying, disseminating or otherwise using the email or any attachment.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
This email and any attachment may contain confidential information which is intended for
use only by the addressee(s) named above. If you received this email by mistake, please
notify the sender immediately, and delete the email from your system. You are prohibited
from copying, disseminating or otherwise using the email or any attachment.
3476
days inactive
3477
days old
infinispan-dev@lists.jboss.org
2 comments
2 participants
participants (2)
-
Andreas Kruthoff -
Sanne Grinovero