[JBoss JIRA] (ISPN-3873) Race condition in AbstractInvocationContextFactory (results in NullPointerException in EquivalentHashMap when receiving an invalidate command during startup)
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-3873?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-3873:
--------------------------------
Assignee: Dan Berindei (was: Mircea Markus)
> Race condition in AbstractInvocationContextFactory (results in NullPointerException in EquivalentHashMap when receiving an invalidate command during startup)
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-3873
> URL: https://issues.jboss.org/browse/ISPN-3873
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 6.0.1.Final
> Environment: Ubuntu Linux 12.04_LTS, Oracle Java SE 1.7.0_45
> Reporter: Karl von Randow
> Assignee: Dan Berindei
> Labels: 630
> Fix For: 7.0.0.Alpha1, 7.0.0.Final
>
> Attachments: infinispan.trace.gz
>
>
> Sometimes when I start one server in a cluster I see the NullPointerException reproduced below. This only occurs if there is another server running. I run multiple caches with clustering using invalidation. JGroups using JDBC_PING and TCP. The caches are non-transactional.
> The NPE is because the keyEq (Equivalence instance for keys) is null. I believe this is because the command is processed before the start() method has been called on the AbstractInvocationContextFactory, as the start() method sets the keyEq ivar.
> At this point I'm not sure how to fix it!
> java.lang.NullPointerException
> at org.infinispan.commons.equivalence.EquivalentHashMap.put(EquivalentHashMap.java:176)
> at org.infinispan.commons.equivalence.EquivalentHashSet.add(EquivalentHashSet.java:109)
> at org.infinispan.context.impl.NonTxInvocationContext.addLockedKey(NonTxInvocationContext.java:84)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:190)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLockNoCheck(LockManagerImpl.java:181)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:149)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitInvalidateCommand(AbstractLockingInterceptor.java:85)
> at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitInvalidateCommand(AbstractVisitor.java:111)
> at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitInvalidateCommand(AbstractVisitor.java:111)
> at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)
> at org.infinispan.commands.AbstractVisitor.visitInvalidateCommand(AbstractVisitor.java:111)
> at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:39)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:95)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:186)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:84)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:259)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:211)
> at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:460)
> at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:665)
> at org.jgroups.JChannel.up(JChannel.java:730)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1019)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:182)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:434)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:434)
> at org.jgroups.stack.Protocol.up(Protocol.java:409)
> at org.jgroups.stack.Protocol.up(Protocol.java:409)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
> at org.jgroups.protocols.RSVP.up(RSVP.java:221)
> at org.jgroups.protocols.UNICAST2.removeAndPassUp(UNICAST2.java:919)
> at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:800)
> at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:415)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600)
> at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:147)
> at org.jgroups.protocols.FD.up(FD.java:255)
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:301)
> at org.jgroups.protocols.MERGE2.up(MERGE2.java:209)
> at org.jgroups.protocols.Discovery.up(Discovery.java:379)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1399)
> at org.jgroups.protocols.TP$MyHandler.run(TP.java:1585)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-3916) Expiration reaper is not enabled until lifespan or max-idle is set
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3916?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-3916:
------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1083147
> Expiration reaper is not enabled until lifespan or max-idle is set
> ------------------------------------------------------------------
>
> Key: ISPN-3916
> URL: https://issues.jboss.org/browse/ISPN-3916
> Project: Infinispan
> Issue Type: Bug
> Components: Server
> Affects Versions: 6.0.1.Final
> Reporter: Wolf-Dieter Fink
> Assignee: Mircea Markus
> Labels: 630
> Fix For: 7.0.0.Alpha1, 7.0.0.Final
>
>
> If a cache is added where entities are with and without expiration by external processes it is expected that the reaper purge the data from memory and datastore.
> As the reaper is only enabled for the cache if the element
> <expiration interval="millis" lifespan="millis" max-idle="millis"/>
> has one of the attributes lifespan or max-idle set to a value >0
> The expired entities will not be removed from the memory or datastore.
> This is an unexpected behaviour and might end in a memory bottleneck if a server is up for a long time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-3916) Expiration reaper is not enabled until lifespan or max-idle is set
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3916?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-3916:
-----------------------------------------------
Mircea Markus <mmarkus(a)redhat.com> changed the Status of [bug 1083147|https://bugzilla.redhat.com/show_bug.cgi?id=1083147] from NEW to ASSIGNED
> Expiration reaper is not enabled until lifespan or max-idle is set
> ------------------------------------------------------------------
>
> Key: ISPN-3916
> URL: https://issues.jboss.org/browse/ISPN-3916
> Project: Infinispan
> Issue Type: Bug
> Components: Server
> Affects Versions: 6.0.1.Final
> Reporter: Wolf-Dieter Fink
> Assignee: Mircea Markus
> Labels: 630
> Fix For: 7.0.0.Alpha1, 7.0.0.Final
>
>
> If a cache is added where entities are with and without expiration by external processes it is expected that the reaper purge the data from memory and datastore.
> As the reaper is only enabled for the cache if the element
> <expiration interval="millis" lifespan="millis" max-idle="millis"/>
> has one of the attributes lifespan or max-idle set to a value >0
> The expired entities will not be removed from the memory or datastore.
> This is an unexpected behaviour and might end in a memory bottleneck if a server is up for a long time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-4085) Random failures in StateProviderTest due to race condition
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4085?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4085:
-----------------------------------------------
Adrian Nistor <anistor(a)redhat.com> changed the Status of [bug 1083112|https://bugzilla.redhat.com/show_bug.cgi?id=1083112] from POST to MODIFIED
> Random failures in StateProviderTest due to race condition
> ----------------------------------------------------------
>
> Key: ISPN-4085
> URL: https://issues.jboss.org/browse/ISPN-4085
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 7.0.0.Alpha1
> Environment: jgroups.bind_addr = 127.0.0.1
> java.runtime.version = 1.7.0_51-b13
> java.runtime.name =Java(TM) SE Runtime Environment
> java.vm.version = 24.51-b03
> java.vm.vendor = Oracle Corporation
> os.name = Mac OS X
> os.version = 10.9.2
> sun.arch.data.model = 64
> sun.cpu.endian = little
> protocol.stack = null
> infinispan.test.jgroups.protocol = tcp
> infinispan.unsafe.allow_jdk8_chm = true
> java.net.preferIPv4Stack = true
> java.net.preferIPv6Stack = null
> log4.configuration = null
> MAVEN_OPTS = null
> Reporter: Gustavo Fernandes
> Assignee: Adrian Nistor
> Labels: 630
> Fix For: 7.0.0.Alpha2, 7.0.0.Final
>
>
> In my environment the StateProviderTest .test2() fails sometimes (about 10% of the time) with the following error(s):
> {code}
> Tests run: 4233, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 428.06 sec <<< FAILURE!
> test2(org.infinispan.statetransfer.StateProviderTest) Time elapsed: 0.042 sec <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:92)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertTrue(Assert.java:54)
> at org.infinispan.statetransfer.StateProviderTest.test2(StateProviderTest.java:316)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> {code}
> The reason why is that test2() feeds the StateProvider a ThreadPoolExecutorService to execute a OutboundTransfer task asynchronously and right after forcing a state transfer
> asserts that there is a StateTransfer in progress. Sometimes the executor service manages to execute the task and as a result it clear the ‘transfersByDestination’ map, and thus the test cannot assert that the state transfer is happening
> OTOH, the method test1() never fails because it users a mock executor service which never executes the task, so the state transfer map will always contain the outbound task after initiating the state transfer and thus always visible from outside
> The quick fix is to also use a mock executor test for the test2()
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-4085) Random failures in StateProviderTest due to race condition
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4085?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4085:
-----------------------------------------------
Adrian Nistor <anistor(a)redhat.com> changed the Status of [bug 1083112|https://bugzilla.redhat.com/show_bug.cgi?id=1083112] from NEW to POST
> Random failures in StateProviderTest due to race condition
> ----------------------------------------------------------
>
> Key: ISPN-4085
> URL: https://issues.jboss.org/browse/ISPN-4085
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 7.0.0.Alpha1
> Environment: jgroups.bind_addr = 127.0.0.1
> java.runtime.version = 1.7.0_51-b13
> java.runtime.name =Java(TM) SE Runtime Environment
> java.vm.version = 24.51-b03
> java.vm.vendor = Oracle Corporation
> os.name = Mac OS X
> os.version = 10.9.2
> sun.arch.data.model = 64
> sun.cpu.endian = little
> protocol.stack = null
> infinispan.test.jgroups.protocol = tcp
> infinispan.unsafe.allow_jdk8_chm = true
> java.net.preferIPv4Stack = true
> java.net.preferIPv6Stack = null
> log4.configuration = null
> MAVEN_OPTS = null
> Reporter: Gustavo Fernandes
> Assignee: Adrian Nistor
> Labels: 630
> Fix For: 7.0.0.Alpha2, 7.0.0.Final
>
>
> In my environment the StateProviderTest .test2() fails sometimes (about 10% of the time) with the following error(s):
> {code}
> Tests run: 4233, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 428.06 sec <<< FAILURE!
> test2(org.infinispan.statetransfer.StateProviderTest) Time elapsed: 0.042 sec <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:92)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertTrue(Assert.java:54)
> at org.infinispan.statetransfer.StateProviderTest.test2(StateProviderTest.java:316)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> {code}
> The reason why is that test2() feeds the StateProvider a ThreadPoolExecutorService to execute a OutboundTransfer task asynchronously and right after forcing a state transfer
> asserts that there is a StateTransfer in progress. Sometimes the executor service manages to execute the task and as a result it clear the ‘transfersByDestination’ map, and thus the test cannot assert that the state transfer is happening
> OTOH, the method test1() never fails because it users a mock executor service which never executes the task, so the state transfer map will always contain the outbound task after initiating the state transfer and thus always visible from outside
> The quick fix is to also use a mock executor test for the test2()
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-4085) Random failures in StateProviderTest due to race condition
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4085?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-4085:
------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1083112
> Random failures in StateProviderTest due to race condition
> ----------------------------------------------------------
>
> Key: ISPN-4085
> URL: https://issues.jboss.org/browse/ISPN-4085
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 7.0.0.Alpha1
> Environment: jgroups.bind_addr = 127.0.0.1
> java.runtime.version = 1.7.0_51-b13
> java.runtime.name =Java(TM) SE Runtime Environment
> java.vm.version = 24.51-b03
> java.vm.vendor = Oracle Corporation
> os.name = Mac OS X
> os.version = 10.9.2
> sun.arch.data.model = 64
> sun.cpu.endian = little
> protocol.stack = null
> infinispan.test.jgroups.protocol = tcp
> infinispan.unsafe.allow_jdk8_chm = true
> java.net.preferIPv4Stack = true
> java.net.preferIPv6Stack = null
> log4.configuration = null
> MAVEN_OPTS = null
> Reporter: Gustavo Fernandes
> Assignee: Adrian Nistor
> Labels: 630
> Fix For: 7.0.0.Alpha2, 7.0.0.Final
>
>
> In my environment the StateProviderTest .test2() fails sometimes (about 10% of the time) with the following error(s):
> {code}
> Tests run: 4233, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 428.06 sec <<< FAILURE!
> test2(org.infinispan.statetransfer.StateProviderTest) Time elapsed: 0.042 sec <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:92)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertTrue(Assert.java:54)
> at org.infinispan.statetransfer.StateProviderTest.test2(StateProviderTest.java:316)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> {code}
> The reason why is that test2() feeds the StateProvider a ThreadPoolExecutorService to execute a OutboundTransfer task asynchronously and right after forcing a state transfer
> asserts that there is a StateTransfer in progress. Sometimes the executor service manages to execute the task and as a result it clear the ‘transfersByDestination’ map, and thus the test cannot assert that the state transfer is happening
> OTOH, the method test1() never fails because it users a mock executor service which never executes the task, so the state transfer map will always contain the outbound task after initiating the state transfer and thus always visible from outside
> The quick fix is to also use a mock executor test for the test2()
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months
[JBoss JIRA] (ISPN-4000) Delta view logging
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4000?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4000:
-----------------------------------------------
Mircea Markus <mmarkus(a)redhat.com> changed the Status of [bug 1083094|https://bugzilla.redhat.com/show_bug.cgi?id=1083094] from NEW to ASSIGNED
> Delta view logging
> ------------------
>
> Key: ISPN-4000
> URL: https://issues.jboss.org/browse/ISPN-4000
> Project: Infinispan
> Issue Type: Feature Request
> Components: Core
> Affects Versions: 6.0.1.Final
> Reporter: Takayoshi Kimura
> Assignee: Mircea Markus
> Labels: 630
> Fix For: 7.0.0.Alpha1
>
>
> Currently on view change the only log we have is "INFO ISPN000094: Received new cluster view".
> {noformat}
> 11:36:27,355 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,localhost-13545) ISPN000094: Received new cluster view: [localhost-7255|23] (24) [localhost-7255, localhost-51221, localhost-12479, localhost-1550, localhost-10300, localhost-11620, localhost-35337, localhost-40886, localhost-3020, localhost-51201, localhost-32626, localhost-17205, localhost-2984, localhost-45021, localhost-23189, localhost-9189, localhost-12902, localhost-38468, localhost-36454, localhost-63088 ...]
> {noformat}
> In a large cluster environment the view list is truncated at max.list.print_size sys prop value, 20 by default. We can increase it but it's still hard to see who is joined/left.
> It would be nice to see the delta on view change, like the following:
> {noformat}
> 11:36:27,355 DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,localhost-13545) Joined: [localhost-8582], Left: []
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 8 months