[JBoss JIRA] (ISPN-4242) LeveDB cache store does not work in OSGi due to CNFE: org.fusesource.leveldbjni.JniDBFactory
by Martin Gencur (JIRA)
Martin Gencur created ISPN-4242:
-----------------------------------
Summary: LeveDB cache store does not work in OSGi due to CNFE: org.fusesource.leveldbjni.JniDBFactory
Key: ISPN-4242
URL: https://issues.jboss.org/browse/ISPN-4242
Project: Infinispan
Issue Type: Bug
Reporter: Martin Gencur
Assignee: Ion Savin
This might be related to https://issues.jboss.org/browse/ISPN-4241. Infinispan does not see custom classes in third-party bundles.
{code}
java.lang.ClassNotFoundException: org.fusesource.leveldbjni.JniDBFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:266)
at org.infinispan.commons.util.Util.loadClassStrict(Util.java:123)
at org.infinispan.commons.util.Util.loadClass(Util.java:83)
at org.infinispan.commons.util.Util.getInstance(Util.java:207)
at org.infinispan.persistence.leveldb.LevelDBStore.newDbFactory(LevelDBStore.java:73)
at org.infinispan.persistence.leveldb.LevelDBStore.init(LevelDBStore.java:56)
at org.infinispan.persistence.manager.PersistenceManagerImpl.createLoadersAndWriters(PersistenceManagerImpl.java:568)
at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:688)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:567)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:522)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:402)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:375)
at org.infinispan.it.osgi.persistence.BaseStoreFunctionalTestOSGi.testPreloadStoredAsBinary(BaseStoreFunctionalTestOSGi.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:67)
at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:37)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.invokeViaJUnit(JUnitProbeInvoker.java:125)
at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.findAndInvoke(JUnitProbeInvoker.java:98)
at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.call(JUnitProbeInvoker.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.ops4j.pax.exam.rbc.internal.RemoteBundleContextImpl.remoteCall(RemoteBundleContextImpl.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] (ISPN-4239) Commands received before the first topology where the local node is a member are not ignored
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4239?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-4239:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2525
> Commands received before the first topology where the local node is a member are not ignored
> --------------------------------------------------------------------------------------------
>
> Key: ISPN-4239
> URL: https://issues.jboss.org/browse/ISPN-4239
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 6.0.2.Final, 7.0.0.Alpha3
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 7.0.0.Alpha4
>
>
> The ISPN-3731 fix was supposed to prevent a joiner from running commands broadcasted by other members before the joiner became a member in the cache topology.
> However, the fix was incomplete, and commands received before the joiner became a member were still executed. This is causing failures in {{StateTransferReplicationQueueTest}}, which is issuing a put(k, v) and a remove(k) and expecting the key to be removed. It can happen that the put(k, v) command is executed before the joiner becomes a member, but the remove(k) command is rejected because it came from a topology where the joiner was not a member:
> {noformat}
> 12:54:41,412 TRACE (testng-StateTransferReplicationQueueTest:nbst-replqueue) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[StateTransferReplicationQueueTest-NodeA-8686], numSegments=1, primaryOwners=[0]}, pendingCH=null} on cache nbst-replqueue
> 12:54:41,412 TRACE (testng-StateTransferReplicationQueueTest:nbst-replqueue) [StateTransferLockImpl] Signalling topology 3 is installed
> 12:54:41,433 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:) [InboundInvocationHandlerImpl] Calling perform() on MultipleRpcCommand{commands=[..., PrepareCommand {modifications=[PutKeyValueCommand{key=test58, value=org.infinispan.statetransfer.StateTransferReplicationQueueTest$PojoValue@59, flags=null, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}], onePhaseCommit=true, gtx=GlobalTransaction:<StateTransferReplicationQueueTest-NodeA-8686>:6969:local, cacheName='nbst-replqueue', topologyId=3}], cacheName='nbst-replqueue'}
> 12:54:41,439 TRACE (remote-thread-StateTransferReplicationQueueTest-NodeC-p3980-t1:nbst-replqueue) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[StateTransferReplicationQueueTest-NodeA-8686], numSegments=1, primaryOwners=[0]}, pendingCH=ReplicatedConsistentHash{members=[StateTransferReplicationQueueTest-NodeA-8686, StateTransferReplicationQueueTest-NodeC-12962], numSegments=1, primaryOwners=[0]}} on cache nbst-replqueue
> 12:54:41,439 TRACE (remote-thread-StateTransferReplicationQueueTest-NodeC-p3980-t1:nbst-replqueue) [StateTransferManagerImpl] This is the first topology in which the local node is a member
> 12:54:41,462 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:nbst-replqueue) [ReadCommittedEntry] Updating entry (key=test58 removed=false valid=true changed=true created=true loaded=false value=org.infinispan.statetransfer.StateTransferReplicationQueueTest$PojoValue@59 metadata=EmbeddedMetadata{version=null}, providedMetadata=null)
> 12:54:41,491 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:) [CommandAwareRpcDispatcher] Attempting to execute command: MultipleRpcCommand{commands=[PrepareCommand {modifications=[RemoveCommand{key=test58, value=null, flags=null, valueMatcher=MATCH_ALWAYS}], onePhaseCommit=true, gtx=GlobalTransaction:<StateTransferReplicationQueueTest-NodeA-8686>:6970:local, cacheName='nbst-replqueue', topologyId=3}, ...], cacheName='nbst-replqueue'} [sender=StateTransferReplicationQueueTest-NodeA-8686]
> 12:54:41,493 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:) [StateTransferLockImpl] Waiting for transaction data for topology 3, current topology is 3
> 12:54:41,493 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:) [InboundInvocationHandlerImpl] Ignoring command sent before the local node was a member (command topology id is 3)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] (ISPN-4241) Distexec cannot clone tasks for distributed executions in Karaf
by Martin Gencur (JIRA)
[ https://issues.jboss.org/browse/ISPN-4241?page=com.atlassian.jira.plugin.... ]
Martin Gencur commented on ISPN-4241:
-------------------------------------
So this problem is more general. It affects also HotRod client when storing/reading custom objects and I guess it will also affect embedded mode whenever working with custom types. JBoss Marshalling is not able to find custom classes. It looks like the problem is in GenericJBossMarshaller which does not have the right class loader. I tried passing OsgiClassLoader.newInstance() to its constructor but it does not work either. The OsgiClassLoader only scans ACTIVE bundles if OsgiClassLoader.class.getClassLoader().getClass().getName().equals("org.osgi.framework.BundleReference")) but this is not true in Karaf. In Karaf, the name of the classloader's class is org.apache.felix.framework.BundleWiringImpl$BundleClassLoaderJava5
> Distexec cannot clone tasks for distributed executions in Karaf
> ---------------------------------------------------------------
>
> Key: ISPN-4241
> URL: https://issues.jboss.org/browse/ISPN-4241
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Execution and Map/Reduce
> Reporter: Martin Gencur
> Assignee: Ion Savin
>
> DistExec uses JBoss Marshalling to clone a task when running in clustered mode but it is not able to load the task class. I don't think this is a problem of classpath in PAX EXAM (as the log below might signify) because DistribuedExecutorService.submit works while DistribuedExecutorService.submitEverywhere fails to load the task class.
> I'll provide the test later.
> Stacktrace:
> {code}
> java.lang.ClassNotFoundException: org.infinispan.it.osgi.distexec.LocalDistributedExecutorTest$ExceptionThrowingCallable not found by org.ops4j.pax.exam.rbc [77]
> at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1460)
> at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)
> at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:266)
> at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:131)
> at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:112)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:943)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1239)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
> at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
> at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:135)
> at org.infinispan.marshall.core.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:101)
> at org.infinispan.commons.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:82)
> at org.infinispan.commons.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:75)
> at org.infinispan.commons.util.Util.cloneWithMarshaller(Util.java:240)
> at org.infinispan.distexec.DefaultExecutorService.clone(DefaultExecutorService.java:507)
> at org.infinispan.distexec.DefaultExecutorService.submitEverywhere(DefaultExecutorService.java:462)
> at org.infinispan.distexec.DefaultExecutorService.submitEverywhere(DefaultExecutorService.java:449)
> at org.infinispan.it.osgi.distexec.LocalDistributedExecutorTest.testExceptionInvocation(LocalDistributedExecutorTest.java:119)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:67)
> at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:37)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
> at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.invokeViaJUnit(JUnitProbeInvoker.java:125)
> at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.findAndInvoke(JUnitProbeInvoker.java:98)
> at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.call(JUnitProbeInvoker.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.ops4j.pax.exam.rbc.internal.RemoteBundleContextImpl.remoteCall(RemoteBundleContextImpl.java:80)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
> at sun.rmi.transport.Transport$1.run(Transport.java:177)
> at sun.rmi.transport.Transport$1.run(Transport.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-4241) Distexec cannot clone tasks for distributed executions in Karaf
by Martin Gencur (JIRA)
[ https://issues.jboss.org/browse/ISPN-4241?page=com.atlassian.jira.plugin.... ]
Martin Gencur commented on ISPN-4241:
-------------------------------------
The same problem happens with Map/Reduce tasks. The call to org.infinispan.distexec.mapreduce.MapReduceTask.clone fails with similar exception - WordCountMapper (or any other Mapper/Reducer/Collator) class not found.
> Distexec cannot clone tasks for distributed executions in Karaf
> ---------------------------------------------------------------
>
> Key: ISPN-4241
> URL: https://issues.jboss.org/browse/ISPN-4241
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Execution and Map/Reduce
> Reporter: Martin Gencur
> Assignee: Ion Savin
>
> DistExec uses JBoss Marshalling to clone a task when running in clustered mode but it is not able to load the task class. I don't think this is a problem of classpath in PAX EXAM (as the log below might signify) because DistribuedExecutorService.submit works while DistribuedExecutorService.submitEverywhere fails to load the task class.
> I'll provide the test later.
> Stacktrace:
> {code}
> java.lang.ClassNotFoundException: org.infinispan.it.osgi.distexec.LocalDistributedExecutorTest$ExceptionThrowingCallable not found by org.ops4j.pax.exam.rbc [77]
> at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1460)
> at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)
> at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:266)
> at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:131)
> at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:112)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:943)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1239)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
> at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
> at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:135)
> at org.infinispan.marshall.core.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:101)
> at org.infinispan.commons.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:82)
> at org.infinispan.commons.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:75)
> at org.infinispan.commons.util.Util.cloneWithMarshaller(Util.java:240)
> at org.infinispan.distexec.DefaultExecutorService.clone(DefaultExecutorService.java:507)
> at org.infinispan.distexec.DefaultExecutorService.submitEverywhere(DefaultExecutorService.java:462)
> at org.infinispan.distexec.DefaultExecutorService.submitEverywhere(DefaultExecutorService.java:449)
> at org.infinispan.it.osgi.distexec.LocalDistributedExecutorTest.testExceptionInvocation(LocalDistributedExecutorTest.java:119)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:67)
> at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:37)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
> at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.invokeViaJUnit(JUnitProbeInvoker.java:125)
> at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.findAndInvoke(JUnitProbeInvoker.java:98)
> at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.call(JUnitProbeInvoker.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.ops4j.pax.exam.rbc.internal.RemoteBundleContextImpl.remoteCall(RemoteBundleContextImpl.java:80)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
> at sun.rmi.transport.Transport$1.run(Transport.java:177)
> at sun.rmi.transport.Transport$1.run(Transport.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-4241) Distexec cannot clone tasks for distributed executions in Karaf
by Martin Gencur (JIRA)
[ https://issues.jboss.org/browse/ISPN-4241?page=com.atlassian.jira.plugin.... ]
Martin Gencur commented on ISPN-4241:
-------------------------------------
I suppose this is caused by the fact that the task's packages/classes are not visible to JBoss Marshaling. Certainly, JBoss Marshalling does NOT use OSGi's Import-Packages directive to import the task classes.
> Distexec cannot clone tasks for distributed executions in Karaf
> ---------------------------------------------------------------
>
> Key: ISPN-4241
> URL: https://issues.jboss.org/browse/ISPN-4241
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Execution and Map/Reduce
> Reporter: Martin Gencur
> Assignee: Ion Savin
>
> DistExec uses JBoss Marshalling to clone a task when running in clustered mode but it is not able to load the task class. I don't think this is a problem of classpath in PAX EXAM (as the log below might signify) because DistribuedExecutorService.submit works while DistribuedExecutorService.submitEverywhere fails to load the task class.
> I'll provide the test later.
> Stacktrace:
> {code}
> java.lang.ClassNotFoundException: org.infinispan.it.osgi.distexec.LocalDistributedExecutorTest$ExceptionThrowingCallable not found by org.ops4j.pax.exam.rbc [77]
> at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1460)
> at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)
> at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:266)
> at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:131)
> at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:112)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:943)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1239)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
> at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
> at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:135)
> at org.infinispan.marshall.core.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:101)
> at org.infinispan.commons.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:82)
> at org.infinispan.commons.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:75)
> at org.infinispan.commons.util.Util.cloneWithMarshaller(Util.java:240)
> at org.infinispan.distexec.DefaultExecutorService.clone(DefaultExecutorService.java:507)
> at org.infinispan.distexec.DefaultExecutorService.submitEverywhere(DefaultExecutorService.java:462)
> at org.infinispan.distexec.DefaultExecutorService.submitEverywhere(DefaultExecutorService.java:449)
> at org.infinispan.it.osgi.distexec.LocalDistributedExecutorTest.testExceptionInvocation(LocalDistributedExecutorTest.java:119)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:67)
> at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:37)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
> at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.invokeViaJUnit(JUnitProbeInvoker.java:125)
> at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.findAndInvoke(JUnitProbeInvoker.java:98)
> at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.call(JUnitProbeInvoker.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.ops4j.pax.exam.rbc.internal.RemoteBundleContextImpl.remoteCall(RemoteBundleContextImpl.java:80)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
> at sun.rmi.transport.Transport$1.run(Transport.java:177)
> at sun.rmi.transport.Transport$1.run(Transport.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-4241) Distexec cannot clone tasks for distributed executions in Karaf
by Martin Gencur (JIRA)
Martin Gencur created ISPN-4241:
-----------------------------------
Summary: Distexec cannot clone tasks for distributed executions in Karaf
Key: ISPN-4241
URL: https://issues.jboss.org/browse/ISPN-4241
Project: Infinispan
Issue Type: Bug
Components: Distributed Execution and Map/Reduce
Reporter: Martin Gencur
Assignee: Ion Savin
DistExec uses JBoss Marshalling to clone a task when running in clustered mode but it is not able to load the task class. I don't think this is a problem of classpath in PAX EXAM (as the log below might signify) because DistribuedExecutorService.submit works while DistribuedExecutorService.submitEverywhere fails to load the task class.
I'll provide the test later.
Stacktrace:
{code}
java.lang.ClassNotFoundException: org.infinispan.it.osgi.distexec.LocalDistributedExecutorTest$ExceptionThrowingCallable not found by org.ops4j.pax.exam.rbc [77]
at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1460)
at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)
at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:266)
at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:131)
at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:112)
at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:943)
at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1239)
at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:135)
at org.infinispan.marshall.core.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:101)
at org.infinispan.commons.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:82)
at org.infinispan.commons.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:75)
at org.infinispan.commons.util.Util.cloneWithMarshaller(Util.java:240)
at org.infinispan.distexec.DefaultExecutorService.clone(DefaultExecutorService.java:507)
at org.infinispan.distexec.DefaultExecutorService.submitEverywhere(DefaultExecutorService.java:462)
at org.infinispan.distexec.DefaultExecutorService.submitEverywhere(DefaultExecutorService.java:449)
at org.infinispan.it.osgi.distexec.LocalDistributedExecutorTest.testExceptionInvocation(LocalDistributedExecutorTest.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:67)
at org.ops4j.pax.exam.invoker.junit.internal.ContainerTestRunner.runChild(ContainerTestRunner.java:37)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.invokeViaJUnit(JUnitProbeInvoker.java:125)
at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.findAndInvoke(JUnitProbeInvoker.java:98)
at org.ops4j.pax.exam.invoker.junit.internal.JUnitProbeInvoker.call(JUnitProbeInvoker.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.ops4j.pax.exam.rbc.internal.RemoteBundleContextImpl.remoteCall(RemoteBundleContextImpl.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-4240) Migrate one intermediate key/value per maxCollectorSize threshold
by Vladimir Blagojevic (JIRA)
Vladimir Blagojevic created ISPN-4240:
-----------------------------------------
Summary: Migrate one intermediate key/value per maxCollectorSize threshold
Key: ISPN-4240
URL: https://issues.jboss.org/browse/ISPN-4240
Project: Infinispan
Issue Type: Enhancement
Components: Distributed Execution and Map/Reduce
Reporter: Vladimir Blagojevic
Assignee: Vladimir Blagojevic
When we hit the maxCollectorSize threshold, we remove all the values from the collector and we start transferring them into the intermediary cache. But the other mapper threads keep working, and event with large collector sizes, it's very likely that the collector will be full again before we finished sending the previous batch. So we could end up with a lot of threads trying to insert maxCollectorSize values in the intermediary cache in parallel, and a lot more intermediary values in memory on each mapper node than the user would expect.
In order to alleviate this observed phenomena Dan proposed an idea to keep the maxCollectorSize threshold, but to only move a single key at a time from the collector to the intermediary cache when the threshold is hit (the least recently used one). That way, the other keys would have time to collect more values (or just run the combiner a few more times).
This way we still transfer some intermediate keys/values during map/combine phase rather than all at once at the end of that phase thus alleviating possibility of OOM. At the end of map/combine phase all remaining keys/values are of course transferred. Application users can fine tune their requirements and a specific use case using maxCollectorSize threshold.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months