[JBoss JIRA] (ISPN-11569) DistributedStreamIteratorTest.testIterationDuringInitialTransfer random failures
by Dan Berindei (Jira)
[ https://issues.redhat.com/browse/ISPN-11569?page=com.atlassian.jira.plugi... ]
Dan Berindei updated ISPN-11569:
--------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/8135
> DistributedStreamIteratorTest.testIterationDuringInitialTransfer random failures
> --------------------------------------------------------------------------------
>
> Key: ISPN-11569
> URL: https://issues.redhat.com/browse/ISPN-11569
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Streams, Test Suite
> Affects Versions: 11.0.0.Dev03
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Major
> Labels: testsuite_stability
> Fix For: 11.0.0.Dev04
>
>
> Very similar to ISPN-11518, {{DistributedStreamIteratorTest.testIterationDuringInitialTransfer}} blocks {{StateProvider.startOutboundTransfer()}}, and with the recent non-blocking changes it also blocks the {{StateTransferStartCommand}} response, and the invocation of {{StateTransferLockImpl.transactionDataReceived()}} on the node that sent the {{StateTransferStartCommand}}.
> {noformat}
> 14:44:11,141 TRACE (non-blocking-thread-Test-NodeA-p24343-t1:[]) [StateTransferLockImpl] Signalling topology 15 is installed
> 14:44:11,147 TRACE (non-blocking-thread-Test-NodeA-p24343-t1:[]) [TrianglePerCacheInboundInvocationHandler] Calling perform() on StateTransferStartCommand{topologyId=15, segments={0}, cacheName=testCache}
> 14:44:11,150 TRACE (non-blocking-thread-Test-NodeA-p24343-t1:[]) [CheckPoint] Triggering event pre_invoked * 1 (available = 1, total = 1)
> 14:44:11,150 TRACE (non-blocking-thread-Test-NodeA-p24343-t1:[]) [CheckPoint] Waiting for event pre_released * 1
> 14:44:11,163 TRACE (testng-Test:[]) [JGroupsTransport] Test-NodeD sending single request 9 to Test-NodeA: InitialPublisherCommand{cacheName='testCache'}
> 14:44:11,164 TRACE (jgroups-9,Test-NodeA:[]) [JGroupsTransport] Test-NodeA received request 9 from Test-NodeD: InitialPublisherCommand{cacheName='testCache'}
> 14:44:11,164 TRACE (jgroups-9,Test-NodeA:[]) [StateTransferLockImpl] Waiting for transaction data for topology 15, current topology is 14
> 14:44:31,155 WARN (non-blocking-thread-Test-NodeA-p24343-t1:[]) [CLUSTER] ISPN000071: Caught exception when handling command StateTransferStartCommand{topologyId=15, segments={0}, cacheName=testCache}
> java.util.concurrent.TimeoutException: Timed out waiting for event pre_released
> at org.infinispan.test.fwk.CheckPoint.awaitStrict(CheckPoint.java:50) ~[test-classes/:?]
> at org.infinispan.test.fwk.CheckPoint.awaitStrict(CheckPoint.java:40) ~[test-classes/:?]
> at org.infinispan.test.Mocks.lambda$blockingAnswer$0(Mocks.java:156) ~[test-classes/:?]
> at org.mockito.internal.stubbing.StubbedInvocationMatcher.answer(StubbedInvocationMatcher.java:39) ~[mockito-core-2.27.0.jar:?]
> at org.mockito.internal.handler.MockHandlerImpl.handle(MockHandlerImpl.java:96) ~[mockito-core-2.27.0.jar:?]
> at org.mockito.internal.handler.NullResultGuardian.handle(NullResultGuardian.java:29) ~[mockito-core-2.27.0.jar:?]
> at org.mockito.internal.handler.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:35) ~[mockito-core-2.27.0.jar:?]
> at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:61) ~[mockito-core-2.27.0.jar:?]
> at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:49) ~[mockito-core-2.27.0.jar:?]
> at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor$DispatcherDefaultingToRealMethod.interceptAbstract(MockMethodInterceptor.java:126) ~[mockito-core-2.27.0.jar:?]
> at org.infinispan.statetransfer.StateProvider$MockitoMock$193392642.startOutboundTransfer(Unknown Source) ~[classes/:?]
> at org.infinispan.commands.statetransfer.StateTransferStartCommand.invokeAsync(StateTransferStartCommand.java:37) ~[classes/:?]
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokeCommand(BasePerCacheInboundInvocationHandler.java:115) ~[classes/:?]
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.invoke(BaseBlockingRunnable.java:100) ~[classes/:?]
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.lambda$runAsync$0(BaseBlockingRunnable.java:91) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
> at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) ~[?:?]
> at org.infinispan.statetransfer.StateTransferLockImpl.notifyTopologyInstalled(StateTransferLockImpl.java:137) ~[classes/:?]
> at org.infinispan.statetransfer.StateConsumerImpl.lambda$onTopologyUpdate$1(StateConsumerImpl.java:341) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) ~[?:?]
> at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235) ~[?:?]
> at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143) ~[?:?]
> at org.infinispan.statetransfer.StateConsumerImpl.onTopologyUpdate(StateConsumerImpl.java:321) ~[classes/:?]
> at org.infinispan.statetransfer.StateTransferManagerImpl.updateProviderAndConsumer(StateTransferManagerImpl.java:200) ~[classes/:?]
> at org.infinispan.statetransfer.StateTransferManagerImpl.lambda$doTopologyUpdate$0(StateTransferManagerImpl.java:188) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) ~[?:?]
> at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235) ~[?:?]
> at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143) ~[?:?]
> at org.infinispan.statetransfer.StateTransferManagerImpl.doTopologyUpdate(StateTransferManagerImpl.java:187) ~[classes/:?]
> at org.infinispan.statetransfer.StateTransferManagerImpl.access$000(StateTransferManagerImpl.java:66) ~[classes/:?]
> at org.infinispan.statetransfer.StateTransferManagerImpl$1.rebalance(StateTransferManagerImpl.java:124) ~[classes/:?]
> at org.infinispan.topology.LocalTopologyManagerImpl.lambda$doHandleRebalance$19(LocalTopologyManagerImpl.java:578) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) ~[?:?]
> at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235) ~[?:?]
> at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143) ~[?:?]
> at org.infinispan.topology.LocalTopologyManagerImpl.doHandleRebalance(LocalTopologyManagerImpl.java:574) ~[classes/:?]
> at org.infinispan.topology.LocalTopologyManagerImpl.lambda$handleRebalance$16(LocalTopologyManagerImpl.java:532) ~[classes/:?]
> at org.infinispan.topology.LocalTopologyManagerImpl.lambda$orderOnCache$24(LocalTopologyManagerImpl.java:737) ~[classes/:?]
> at org.infinispan.util.concurrent.ActionSequencer.safeNonBlockingCall(ActionSequencer.java:57) [classes/:?]
> at org.infinispan.util.concurrent.ActionSequencer.access$400(ActionSequencer.java:32) [classes/:?]
> at org.infinispan.util.concurrent.ActionSequencer$SequenceEntry.run(ActionSequencer.java:226) [classes/:?]
> 14:44:31,180 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.stream.DistributedStreamIteratorTest.testIterationDuringInitialTransfer[DIST_SYNC, tx=false]
> java.lang.IllegalStateException: Thread already timed out waiting for event pre_released
> at org.infinispan.test.fwk.CheckPoint.trigger(CheckPoint.java:177) ~[test-classes/:?]
> at org.infinispan.test.fwk.CheckPoint.triggerForever(CheckPoint.java:166) ~[test-classes/:?]
> at org.infinispan.stream.DistributedStreamIteratorTest.testIterationDuringInitialTransfer(DistributedStreamIteratorTest.java:115) ~[test-classes/:?]
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (ISPN-11574) Automatic server authentication configuration
by Tristan Tarrant (Jira)
[ https://issues.redhat.com/browse/ISPN-11574?page=com.atlassian.jira.plugi... ]
Tristan Tarrant updated ISPN-11574:
-----------------------------------
Status: Open (was: New)
> Automatic server authentication configuration
> ---------------------------------------------
>
> Key: ISPN-11574
> URL: https://issues.redhat.com/browse/ISPN-11574
> Project: Infinispan
> Issue Type: Enhancement
> Components: Security, Server
> Affects Versions: 11.0.0.Dev03
> Reporter: Tristan Tarrant
> Assignee: Tristan Tarrant
> Priority: Major
> Fix For: 11.0.0.Dev04
>
>
> if the <endpoint> is configured with a security realm, authentication will be enabled
> the authentication mechanisms will be determined by the security realm stores
> Default mechs for Hot Rod: SCRAM-SHA-512 SCRAM-SHA-384 SCRAM-SHA-256 SCRAM-SHA-1 DIGEST-SHA-512 DIGEST-SHA-384 DIGEST-SHA-256 DIGEST-SHA DIGEST-MD5
> Default mechs for HTTP: DIGEST
> *PLAIN/BASIC will only be enabled if TLS is also enabled
> * EXTERNAL/CLIENT_CERT will only be enabled if TLS is enabled and require-ssl-client-auth="true"
> * GSSAPI/GS2-KRB5/SPNEGO will only be enabled if a kerberos realm is present
> * OAUTHBEARER/BEARER_TOKEN will only be enabled if a token realm is present
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (ISPN-11574) Automatic server authentication configuration
by Tristan Tarrant (Jira)
[ https://issues.redhat.com/browse/ISPN-11574?page=com.atlassian.jira.plugi... ]
Tristan Tarrant updated ISPN-11574:
-----------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/8134
> Automatic server authentication configuration
> ---------------------------------------------
>
> Key: ISPN-11574
> URL: https://issues.redhat.com/browse/ISPN-11574
> Project: Infinispan
> Issue Type: Enhancement
> Components: Security, Server
> Affects Versions: 11.0.0.Dev03
> Reporter: Tristan Tarrant
> Assignee: Tristan Tarrant
> Priority: Major
> Fix For: 11.0.0.Dev04
>
>
> if the <endpoint> is configured with a security realm, authentication will be enabled
> the authentication mechanisms will be determined by the security realm stores
> Default mechs for Hot Rod: SCRAM-SHA-512 SCRAM-SHA-384 SCRAM-SHA-256 SCRAM-SHA-1 DIGEST-SHA-512 DIGEST-SHA-384 DIGEST-SHA-256 DIGEST-SHA DIGEST-MD5
> Default mechs for HTTP: DIGEST
> *PLAIN/BASIC will only be enabled if TLS is also enabled
> * EXTERNAL/CLIENT_CERT will only be enabled if TLS is enabled and require-ssl-client-auth="true"
> * GSSAPI/GS2-KRB5/SPNEGO will only be enabled if a kerberos realm is present
> * OAUTHBEARER/BEARER_TOKEN will only be enabled if a token realm is present
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (ISPN-11574) Automatic server authentication configuration
by Tristan Tarrant (Jira)
[ https://issues.redhat.com/browse/ISPN-11574?page=com.atlassian.jira.plugi... ]
Tristan Tarrant updated ISPN-11574:
-----------------------------------
Description:
if the <endpoint> is configured with a security realm, authentication should be enabled automatically based on the security realm stores.
Default mechs for Hot Rod:
* SCRAM-SHA-512 SCRAM-SHA-384 SCRAM-SHA-256 SCRAM-SHA-1 DIGEST-SHA-512 DIGEST-SHA-384 DIGEST-SHA-256 DIGEST-SHA DIGEST-MD5
Default mechs for HTTP:
* DIGEST
*PLAIN/BASIC will only be enabled if TLS is also enabled
* EXTERNAL/CLIENT_CERT will only be enabled if TLS is enabled and require-ssl-client-auth="true"
* GSSAPI/GS2-KRB5/SPNEGO will only be enabled if a kerberos realm is present
* OAUTHBEARER/BEARER_TOKEN will only be enabled if a token realm is present
was:
if the <endpoint> is configured with a security realm, authentication will be enabled
the authentication mechanisms will be determined by the security realm stores
Default mechs for Hot Rod: SCRAM-SHA-512 SCRAM-SHA-384 SCRAM-SHA-256 SCRAM-SHA-1 DIGEST-SHA-512 DIGEST-SHA-384 DIGEST-SHA-256 DIGEST-SHA DIGEST-MD5
Default mechs for HTTP: DIGEST
*PLAIN/BASIC will only be enabled if TLS is also enabled
* EXTERNAL/CLIENT_CERT will only be enabled if TLS is enabled and require-ssl-client-auth="true"
* GSSAPI/GS2-KRB5/SPNEGO will only be enabled if a kerberos realm is present
* OAUTHBEARER/BEARER_TOKEN will only be enabled if a token realm is present
> Automatic server authentication configuration
> ---------------------------------------------
>
> Key: ISPN-11574
> URL: https://issues.redhat.com/browse/ISPN-11574
> Project: Infinispan
> Issue Type: Enhancement
> Components: Security, Server
> Affects Versions: 11.0.0.Dev03
> Reporter: Tristan Tarrant
> Assignee: Tristan Tarrant
> Priority: Major
> Fix For: 11.0.0.Dev04
>
>
> if the <endpoint> is configured with a security realm, authentication should be enabled automatically based on the security realm stores.
> Default mechs for Hot Rod:
> * SCRAM-SHA-512 SCRAM-SHA-384 SCRAM-SHA-256 SCRAM-SHA-1 DIGEST-SHA-512 DIGEST-SHA-384 DIGEST-SHA-256 DIGEST-SHA DIGEST-MD5
> Default mechs for HTTP:
> * DIGEST
> *PLAIN/BASIC will only be enabled if TLS is also enabled
> * EXTERNAL/CLIENT_CERT will only be enabled if TLS is enabled and require-ssl-client-auth="true"
> * GSSAPI/GS2-KRB5/SPNEGO will only be enabled if a kerberos realm is present
> * OAUTHBEARER/BEARER_TOKEN will only be enabled if a token realm is present
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (ISPN-11574) Automatic server authentication configuration
by Tristan Tarrant (Jira)
Tristan Tarrant created ISPN-11574:
--------------------------------------
Summary: Automatic server authentication configuration
Key: ISPN-11574
URL: https://issues.redhat.com/browse/ISPN-11574
Project: Infinispan
Issue Type: Enhancement
Components: Security, Server
Affects Versions: 11.0.0.Dev03
Reporter: Tristan Tarrant
Assignee: Tristan Tarrant
Fix For: 11.0.0.Dev04
if the <endpoint> is configured with a security realm, authentication will be enabled
the authentication mechanisms will be determined by the security realm stores
Default mechs for Hot Rod: SCRAM-SHA-512 SCRAM-SHA-384 SCRAM-SHA-256 SCRAM-SHA-1 DIGEST-SHA-512 DIGEST-SHA-384 DIGEST-SHA-256 DIGEST-SHA DIGEST-MD5
Default mechs for HTTP: DIGEST
*PLAIN/BASIC will only be enabled if TLS is also enabled
* EXTERNAL/CLIENT_CERT will only be enabled if TLS is enabled and require-ssl-client-auth="true"
* GSSAPI/GS2-KRB5/SPNEGO will only be enabled if a kerberos realm is present
* OAUTHBEARER/BEARER_TOKEN will only be enabled if a token realm is present
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (ISPN-11422) ClusterExpirationFunctionalTest.testWriteExpiredEntry random failures
by Will Burns (Jira)
[ https://issues.redhat.com/browse/ISPN-11422?page=com.atlassian.jira.plugi... ]
Will Burns commented on ISPN-11422:
-----------------------------------
Okay looking closer, this is caused by sending a write command to a key that is expired, but the primary owner didn't see it expired but the backup does.
This can be caused by a few different things including time mismatch in clocks.
> ClusterExpirationFunctionalTest.testWriteExpiredEntry random failures
> ---------------------------------------------------------------------
>
> Key: ISPN-11422
> URL: https://issues.redhat.com/browse/ISPN-11422
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite
> Affects Versions: 11.0.0.Alpha2
> Reporter: Dan Berindei
> Assignee: Will Burns
> Priority: Major
> Labels: testsuite_stability
> Fix For: 11.0.0.Final
>
> Attachments: ISPN-11367_default_configuration_20200301-2115_ClusterExpirationFunctionalTest-infinispan-core.log.gz
>
>
> {noformat}
> 21:20:48,799 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command PutKeyValueCommand{key=key, value=value, flags=[], commandInvocationId=CommandInvocation:Test-NodeA:15530, putIfAbsent=true, valueMatcher=MATCH_EXPECTED, metadata=EmbeddedExpirableMetadata{version=null, lifespan=1000, maxIdle=-1}, successful=true, topologyId=-1} and InvocationContext [SingleKeyNonTxInvocationContext{isLocked=false, key=null, cacheEntry=null, origin=null, lockOwner=CommandInvocation:Test-NodeA:15530}]
> 21:20:48,800 TRACE (jgroups-5,Test-NodeA:[]) [TriangleDistributionInterceptor] Command CommandInvocation:Test-NodeA:15530 send to backup owner [Test-NodeB].
> 21:20:48,800 TRACE (jgroups-5,Test-NodeA:[]) [TriangleDistributionInterceptor] Command CommandInvocation:Test-NodeA:15530 got sequence 56 for segment 99
> 21:20:48,801 TRACE (jgroups-6,Test-NodeB:[]) [TriangleOrderManager] Receiver old topology. Current sequence (27:56), command sequence (27:56)
> 21:20:48,802 TRACE (remote-thread-Test-NodeB-p15169-t1:[]) [ClusterExpirationManager] Submitting expiration removal for key: key which is maxIdle: false of: 1000
> 21:20:48,802 TRACE (remote-thread-Test-NodeB-p15169-t1:[]) [CommandAckCollector] Created new collector for 15533. BackupOwners=[Test-NodeB]
> 21:20:48,802 TRACE (remote-thread-Test-NodeB-p15169-t1:[]) [JGroupsTransport] Test-NodeB sending single request 48 to Test-NodeA: SingleRpcCommand{cacheName='defaultcache', command=RemoveExpiredCommand{key=key, value=value, lifespan=1000, maxIde=false}}
> 21:20:48,803 TRACE (remote-thread-Test-NodeA-p15163-t3:[]) [TriangleDistributionInterceptor] Command CommandInvocation:Test-NodeB:15533 got sequence 57 for segment 99
> 21:20:48,804 TRACE (jgroups-5,Test-NodeB:[]) [CommandAckCollector] Pending backup acks: [Test-NodeB]
> 21:20:48,804 TRACE (jgroups-6,Test-NodeB:[]) [TriangleOrderAction] Checking if next for segment 99 and sequence 57
> 21:20:48,804 TRACE (jgroups-6,Test-NodeB:[]) [TriangleOrderManager] Receiver old topology. Current sequence (27:56), command sequence (27:57)
> 21:20:48,804 TRACE (Controller-remote-thread-Test-NodeB:[]) [TriangleOrderManager] Receiver old topology. Current sequence (27:56), command sequence (27:57)
> 21:20:48,804 TRACE (Controller-remote-thread-Test-NodeB:[]) [BlockingTaskAwareExecutorServiceImpl] Tasks to be executed=0, still pending=~1
> 21:21:03,801 ERROR (timeout-thread-Test-NodeA-p15164-t1:[]) [InvocationContextInterceptor] ISPN000136: Error executing command PutKeyValueCommand on Cache 'defaultcache', writing keys [key]
> org.infinispan.util.concurrent.TimeoutException: ISPN000427: Timeout after 15 seconds waiting for acks. Id=15530
> 21:21:03,805 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.expiration.impl.ClusterExpirationFunctionalTest.testWriteExpiredEntry[DIST_SYNC, tx=false, storageType=OBJECT]
> org.infinispan.util.concurrent.TimeoutException: ISPN000427: Timeout after 15 seconds waiting for acks. Id=15530
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:255) ~[classes/:?]
> at org.infinispan.cache.impl.InvocationHelper.doInvoke(InvocationHelper.java:296) ~[classes/:?]
> at org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:100) ~[classes/:?]
> at org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:82) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:1331) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:1325) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:1321) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:232) ~[classes/:?]
> at org.infinispan.cache.impl.AbstractDelegatingCache.putIfAbsent(AbstractDelegatingCache.java:127) ~[classes/:?]
> at org.infinispan.cache.impl.EncoderCache.putIfAbsent(EncoderCache.java:215) ~[classes/:?]
> at org.infinispan.expiration.impl.ClusterExpirationFunctionalTest.testWriteExpiredEntry(ClusterExpirationFunctionalTest.java:374) ~[test-classes/:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) ~[testng-6.14.3.jar:?]
> at org.testng.internal.MethodInvocationHelper$1.runTestMethod(MethodInvocationHelper.java:230) ~[testng-6.14.3.jar:?]
> at org.infinispan.commons.test.TestNGLongTestsHook.run(TestNGLongTestsHook.java:24) ~[infinispan-commons-test-11.0.0-SNAPSHOT.jar:11.0.0-SNAPSHOT]
> at org.testng.internal.MethodInvocationHelper.invokeHookable(MethodInvocationHelper.java:242) ~[testng-6.14.3.jar:?]
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:579) [testng-6.14.3.jar:?]
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719) [testng-6.14.3.jar:?]
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989) [testng-6.14.3.jar:?]
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) [testng-6.14.3.jar:?]
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) [testng-6.14.3.jar:?]
> at org.testng.TestRunner.privateRun(TestRunner.java:648) [testng-6.14.3.jar:?]
> at org.testng.TestRunner.run(TestRunner.java:505) [testng-6.14.3.jar:?]
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:455) [testng-6.14.3.jar:?]
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:40) [testng-6.14.3.jar:?]
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:489) [testng-6.14.3.jar:?]
> at org.testng.internal.thread.ThreadUtil$1.call(ThreadUtil.java:52) [testng-6.14.3.jar:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
> at java.lang.Thread.run(Thread.java:834) [?:?]
> Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000427: Timeout after 15 seconds waiting for acks. Id=15530
> at org.infinispan.util.concurrent.CommandAckCollector.createTimeoutException(CommandAckCollector.java:222) ~[classes/:?]
> at org.infinispan.util.concurrent.CommandAckCollector.access$700(CommandAckCollector.java:58) ~[classes/:?]
> at org.infinispan.util.concurrent.CommandAckCollector$BaseCollector.call(CommandAckCollector.java:275) ~[classes/:?]
> at org.infinispan.util.concurrent.CommandAckCollector$BaseCollector.call(CommandAckCollector.java:255) ~[classes/:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
> ... 3 more
> Suppressed: org.infinispan.util.logging.TraceException
> at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:39) ~[classes/:?]
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246) ~[classes/:?]
> at org.infinispan.cache.impl.InvocationHelper.doInvoke(InvocationHelper.java:296) ~[classes/:?]
> at org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:100) ~[classes/:?]
> at org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:82) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:1331) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:1325) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:1321) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:232) ~[classes/:?]
> at org.infinispan.cache.impl.AbstractDelegatingCache.putIfAbsent(AbstractDelegatingCache.java:127) ~[classes/:?]
> at org.infinispan.cache.impl.EncoderCache.putIfAbsent(EncoderCache.java:215) ~[classes/:?]
> at org.infinispan.expiration.impl.ClusterExpirationFunctionalTest.testWriteExpiredEntry(ClusterExpirationFunctionalTest.java:374) ~[test-classes/:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) ~[testng-6.14.3.jar:?]
> at org.testng.internal.MethodInvocationHelper$1.runTestMethod(MethodInvocationHelper.java:230) ~[testng-6.14.3.jar:?]
> at org.infinispan.commons.test.TestNGLongTestsHook.run(TestNGLongTestsHook.java:24) ~[infinispan-commons-test-11.0.0-SNAPSHOT.jar:11.0.0-SNAPSHOT]
> at org.testng.internal.MethodInvocationHelper.invokeHookable(MethodInvocationHelper.java:242) ~[testng-6.14.3.jar:?]
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:579) [testng-6.14.3.jar:?]
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719) [testng-6.14.3.jar:?]
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989) [testng-6.14.3.jar:?]
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) [testng-6.14.3.jar:?]
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) [testng-6.14.3.jar:?]
> at org.testng.TestRunner.privateRun(TestRunner.java:648) [testng-6.14.3.jar:?]
> at org.testng.TestRunner.run(TestRunner.java:505) [testng-6.14.3.jar:?]
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:455) [testng-6.14.3.jar:?]
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:40) [testng-6.14.3.jar:?]
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:489) [testng-6.14.3.jar:?]
> at org.testng.internal.thread.ThreadUtil$1.call(ThreadUtil.java:52) [testng-6.14.3.jar:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
> at java.lang.Thread.run(Thread.java:834) [?:?]
> {noformat}
> Looks like a deadlock: the {{RemoveExpiredCommand}} can't run on backup B because the sequence number is still 56 (the seqno of the {{PutKeyValueCommand}}).
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years