[JBoss JIRA] (ISPN-7070) CacheNotifierImplInitialTransferDistTest.testIterationBeganAndSegmentNotComplete random failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-7070?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-7070:
------------------------------------
{{testPrimaryOwnerGoesDownBeforeSendingEvent}} can also fail with the same error:
{noformat}
00:06:41,609 ERROR (testng-CacheNotifierImplInitialTransferDistTest:[]) [TestSuiteProgress] Test failed: org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testModificationAfterIterationBeganAndSegmentNotCompleteValueNonOwnerClustered
java.lang.AssertionError: expected [12] but found [11]
at org.testng.Assert.fail(Assert.java:94) ~[testng-6.8.8.jar:?]
at org.testng.Assert.failNotEquals(Assert.java:494) ~[testng-6.8.8.jar:?]
at org.testng.Assert.assertEquals(Assert.java:123) ~[testng-6.8.8.jar:?]
at org.testng.Assert.assertEquals(Assert.java:370) ~[testng-6.8.8.jar:?]
at org.testng.Assert.assertEquals(Assert.java:380) ~[testng-6.8.8.jar:?]
at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testIterationBeganAndSegmentNotComplete(CacheNotifierImplInitialTransferDistTest.java:509) ~[test-classes/:?]
at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testModificationAfterIterationBeganAndSegmentNotCompleteValueNonOwnerClustered(CacheNotifierImplInitialTransferDistTest.java:622) ~[test-classes/:?]
{noformat}
> CacheNotifierImplInitialTransferDistTest.testIterationBeganAndSegmentNotComplete random failures
> ------------------------------------------------------------------------------------------------
>
> Key: ISPN-7070
> URL: https://issues.jboss.org/browse/ISPN-7070
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 9.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: William Burns
> Priority: Critical
> Labels: testsuite_stability
> Fix For: 9.0.0.Beta1
>
> Attachments: CacheNotifierImplInitialTransferDistTest_ISPN-5467_CompletableFuture-like_API_20161003.log
>
>
> Looks like sometimes the originator listener ignores the CACHE_ENTRY_REMOVED notification:
> {noformat}
> 09:58:57,877 INFO (testng-Test:[]) [TestSuiteProgress] Test starting: org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testRemoveAfterIterationBeganAndSegmentNotCompleteValueNonOwnerClustered
> 09:58:58,913 DEBUG (testng-Test:[]) [Test] Found key key-to-change-0 with primary owner != Test-NodeA-7051, segment 60
> 09:58:58,930 TRACE (remote-thread-Test-NodeA-p23726-t6:[]) [ReadCommittedEntry] Updating entry (key=key-to-change-0 removed=false valid=true changed=true created=true value=initial-value metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, providedMetadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null})
> 09:58:58,972 TRACE (remote-thread-Test-NodeC-p23887-t6:[]) [ReadCommittedEntry] Updating entry (key=key-to-change-0 removed=false valid=true changed=true created=true value=initial-value metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, providedMetadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null})
> 09:58:59,014 TRACE (ForkThread-1,Test:[]) [Test] Adding segment listener 2a41ec3f-c79f-4b7d-bd0e-8754170d8fc1 : org.infinispan.notifications.cachelistener.DistributedQueueingSegmentListener@68007f8
> 09:58:59,014 TRACE (ForkThread-1,Test:[]) [CacheNotifierImpl] Replicating cluster listener to other nodes [Test-NodeA-7051, Test-NodeB-10318, Test-NodeC-52778] for cluster listener with id 2a41ec3f-c79f-4b7d-bd0e-8754170d8fc1
> 09:58:58,998 TRACE (testng-Test:[]) [CheckPoint] Waiting for event pre_complete_segment_invoked * 1
> 09:58:59,063 TRACE (remote-thread-Test-NodeC-p23887-t6:[]) [ClusterListenerReplicateCallable] Registered local cluster listener for remote cluster listener from origin Test-NodeA-7051 with id 2a41ec3f-c79f-4b7d-bd0e-8754170d8fc1
> 09:58:59,098 TRACE (remote-thread-Test-NodeB-p23775-t6:[]) [ClusterListenerReplicateCallable] Registered local cluster listener for remote cluster listener from origin Test-NodeA-7051 with id 2a41ec3f-c79f-4b7d-bd0e-8754170d8fc1
> 09:58:59,145 TRACE (ForkThread-1,Test:[]) [CacheNotifierImpl] Listener 2a41ec3f-c79f-4b7d-bd0e-8754170d8fc1 requests initial state for cache
> 09:58:59,373 TRACE (ForkThread-1,Test:[]) [Test] Completed segments [48, 2, 229, 118, 71, 72, 233, 60]
> 09:58:59,373 TRACE (ForkThread-1,Test:[]) [Test] Listener received event EventImpl{type=CACHE_ENTRY_CREATED, pre=false, cache=Cache 'DistInitialTransferListener'@Test-NodeA-7051, key=key-to-change-0, value=initial-value, oldValue=null, transaction=null, originLocal=true, transactionSuccessful=false, entries=null, created=false}
> 09:58:59,373 TRACE (ForkThread-1,Test:[]) [Test] Notified for key key-to-change-0
> 09:58:59,373 TRACE (ForkThread-1,Test:[]) [CheckPoint] Triggering event pre_complete_segment_invoked * 1 (available = 1, total = 1)
> 09:58:59,373 TRACE (ForkThread-1,Test:[]) [CheckPoint] Waiting for event pre_complete_segment_released * 1
> 09:58:59,373 TRACE (testng-Test:[]) [CheckPoint] Received event pre_complete_segment_invoked * 1 (available = 0, total = 1)
> 09:58:59,577 TRACE (remote-thread-Test-NodeA-p23726-t6:[]) [ReadCommittedEntry] Updating entry (key=key-to-change-0 removed=true valid=false changed=true created=false value=initial-value metadata=EmbeddedMetadata{version=null}, providedMetadata=null)
> 09:58:59,661 TRACE (remote-thread-Test-NodeC-p23887-t5:[]) [ReadCommittedEntry] Updating entry (key=key-to-change-0 removed=true valid=false changed=true created=false value=initial-value metadata=EmbeddedMetadata{version=null}, providedMetadata=null)
> 09:58:59,661 TRACE (remote-thread-Test-NodeC-p23887-t5:[]) [RequestCorrelator] Test-NodeC-52778: invoking unicast RPC [req-id=91126] on Test-NodeA-7051
> 09:58:59,684 TRACE (OOB-2,Test-NodeA-7051:[]) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 91126
> 09:58:59,779 TRACE (remote-thread-Test-NodeA-p23726-t6:[]) [NonTotalOrderPerCacheInboundInvocationHandler] Calling perform() on DistributedExecuteCommand [cache=Cache 'DistInitialTransferListener'@Test-NodeA-7051, keys=[], callable=org.infinispan.notifications.cachelistener.cluster.ClusterEventCallable@640b0234]
> 09:58:59,780 TRACE (remote-thread-Test-NodeA-p23726-t6:[]) [ClusterEventCallable] Received cluster event(s) [ClusterEvent {type=CACHE_ENTRY_REMOVED, cache=Cache 'DistInitialTransferListener'@Test-NodeA-7051, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, retryCommand=false, origin=Test-NodeC-52778}], notifying cluster listener with id 2a41ec3f-c79f-4b7d-bd0e-8754170d8fc1
> 09:58:59,837 TRACE (testng-Test:[]) [CheckPoint] Triggering event pre_complete_segment_released * 999999999 (available = 999999999, total = 999999999)
> 09:58:59,837 TRACE (ForkThread-1,Test:[]) [CheckPoint] Received event pre_complete_segment_released * 1 (available = 999999998, total = 999999999)
> 09:58:59,898 TRACE (ForkThread-1,Test:[]) [CacheNotifierImpl] Listener 2a41ec3f-c79f-4b7d-bd0e-8754170d8fc1 initial state for cache completed
> 09:58:59,916 TRACE (remote-thread-Test-NodeB-p23775-t6:[]) [ClusterListenerRemoveCallable] Removing local cluster listener due to parent cluster listener was removed : 2a41ec3f-c79f-4b7d-bd0e-8754170d8fc1
> 09:58:59,921 TRACE (remote-thread-Test-NodeC-p23887-t5:[]) [ClusterListenerRemoveCallable] Removing local cluster listener due to parent cluster listener was removed : 2a41ec3f-c79f-4b7d-bd0e-8754170d8fc1
> 09:58:59,943 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testRemoveAfterIterationBeganAndSegmentNotCompleteValueNonOwnerClustered
> java.lang.AssertionError: expected [12] but found [11]
> {noformat}
> {{CacheNotifierImplInitialTransferDistTest}} also had a different problem: it replaced the {{CacheNotifier}} component with a clone before adding the listener, but replaced it again with the original before removing the listener. Because the listener's uuid was only registered in the clone, {{removeListener}} couldn't find the uuid and disn't try to remove the listener on the other nodes. This meant the listener was leaked, resulting in logs like this:
> {noformat}
> 20:12:04,098 TRACE (remote-thread-Test-NodeB-p24127-t6:[]) [ReadCommittedEntry] Updating entry (key=key-to-change-0 removed=true valid=false changed=true created=false value=initial-value metadata=EmbeddedMetadata{version=null}, providedMetadata=null)
> 20:12:04,099 TRACE (remote-thread-Test-NodeB-p24127-t6:[]) [RemoteClusterListener] Passing Event to manager EventImpl{type=CACHE_ENTRY_REMOVED, pre=false, cache=Cache 'DistInitialTransferListener'@Test-NodeB-56142, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, originLocal=false, transactionSuccessful=false, entries=null, created=false} to send to Test-NodeA-38047
> 20:12:04,100 TRACE (remote-thread-Test-NodeB-p24127-t6:[]) [RemoteClusterListener] Passing Event to manager EventImpl{type=CACHE_ENTRY_REMOVED, pre=false, cache=Cache 'DistInitialTransferListener'@Test-NodeB-56142, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, originLocal=false, transactionSuccessful=false, entries=null, created=false} to send to Test-NodeA-38047
> 20:12:04,100 TRACE (remote-thread-Test-NodeB-p24127-t6:[]) [RemoteClusterListener] Passing Event to manager EventImpl{type=CACHE_ENTRY_REMOVED, pre=false, cache=Cache 'DistInitialTransferListener'@Test-NodeB-56142, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, originLocal=false, transactionSuccessful=false, entries=null, created=false} to send to Test-NodeA-38047
> 20:12:04,100 TRACE (remote-thread-Test-NodeB-p24127-t6:[]) [RemoteClusterListener] Passing Event to manager EventImpl{type=CACHE_ENTRY_REMOVED, pre=false, cache=Cache 'DistInitialTransferListener'@Test-NodeB-56142, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, originLocal=false, transactionSuccessful=false, entries=null, created=false} to send to Test-NodeA-38047
> 20:12:04,100 TRACE (remote-thread-Test-NodeB-p24127-t6:[]) [RemoteClusterListener] Passing Event to manager EventImpl{type=CACHE_ENTRY_REMOVED, pre=false, cache=Cache 'DistInitialTransferListener'@Test-NodeB-56142, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, originLocal=false, transactionSuccessful=false, entries=null, created=false} to send to Test-NodeA-38047
> 20:12:04,106 TRACE (remote-thread-Test-NodeA-p24103-t6:[]) [MultiClusterEventCallable] Received multiple cluster event(s) {86f77630-b1bb-4a21-97bb-b03efeae39d6=[ClusterEvent {type=CACHE_ENTRY_REMOVED, cache=Cache 'DistInitialTransferListener'@Test-NodeA-38047, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, retryCommand=false, origin=Test-NodeB-56142}], 91d05a50-4a35-4e37-8c5c-bffbabc6154b=[ClusterEvent {type=CACHE_ENTRY_REMOVED, cache=Cache 'DistInitialTransferListener'@Test-NodeA-38047, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, retryCommand=false, origin=Test-NodeB-56142}], ba09480f-c698-431b-8fe6-213f1bdefc7b=[ClusterEvent {type=CACHE_ENTRY_REMOVED, cache=Cache 'DistInitialTransferListener'@Test-NodeA-38047, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, retryCommand=false, origin=Test-NodeB-56142}], 992da014-0516-4f2a-8093-50ba3b1f358a=[ClusterEvent {type=CACHE_ENTRY_REMOVED, cache=Cache 'DistInitialTransferListener'@Test-NodeA-38047, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, retryCommand=false, origin=Test-NodeB-56142}], b4eeb9b3-85c4-4ff9-865e-076b7cf93faa=[ClusterEvent {type=CACHE_ENTRY_REMOVED, cache=Cache 'DistInitialTransferListener'@Test-NodeA-38047, key=key-to-change-0, value=null, oldValue=initial-value, transaction=null, retryCommand=false, origin=Test-NodeB-56142}]}
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 5 months
[JBoss JIRA] (ISPN-5021) Nodes that finish the rebalance later can see outdated values
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-5021?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-5021:
------------------------------
Description:
Copied from [ISPN-4444|https://issues.jboss.org/browse/ISPN-4444?focusedCommentId=1302...]
If the CH_UPDATE command is delayed on the old owner, the new owners might update the key without the old owner knowing, and a locality check on the old owner won't help.
I remember one thing that struck me when reading the Raft algorithm was that they install configuration changes symmetrically, in 3 phases. We might need to do the same for our rebalance:
1. T0: read_ch=old, write_ch=old
2. start a rebalance
3. T1: read_ch=old, write_ch=old+new
4. new owners have all the data
5. T2: read_ch=new, write_ch=old+new
6. remove old cache entries and ignore further writes
7. T3: read_ch=new, write_ch=new
was:
Copied from [ISPN-4444|https://issues.jboss.org/browse/ISPN-4444?focusedCommentId=1302...]
If the CH_UPDATE command is delayed on the old owner, the new owners might update the key without the old owner knowing, and a locality check on the old owner won't help.
I remember one thing that struck me when reading the Raft algorithm was that they install configuration changes symmetrically, in 3 phases. We might need to do the same for our rebalance: start a rebalance with read_ch=old, write_ch=old+new, when the new owners have all the data install read_ch=new, write_ch=old+new, and finally read_ch=new, write_ch=new. Old cache entries are removed during the 2nd topology update, and further writes should be ignored, in order for this to work.
> Nodes that finish the rebalance later can see outdated values
> -------------------------------------------------------------
>
> Key: ISPN-5021
> URL: https://issues.jboss.org/browse/ISPN-5021
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.0.2.Final
> Reporter: Dan Berindei
> Assignee: Pedro Ruivo
> Priority: Critical
>
> Copied from [ISPN-4444|https://issues.jboss.org/browse/ISPN-4444?focusedCommentId=1302...]
> If the CH_UPDATE command is delayed on the old owner, the new owners might update the key without the old owner knowing, and a locality check on the old owner won't help.
> I remember one thing that struck me when reading the Raft algorithm was that they install configuration changes symmetrically, in 3 phases. We might need to do the same for our rebalance:
> 1. T0: read_ch=old, write_ch=old
> 2. start a rebalance
> 3. T1: read_ch=old, write_ch=old+new
> 4. new owners have all the data
> 5. T2: read_ch=new, write_ch=old+new
> 6. remove old cache entries and ignore further writes
> 7. T3: read_ch=new, write_ch=new
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 5 months
[JBoss JIRA] (ISPN-7165) CACHE_MODE_LOCAL is unsafe with L1 enabled
by Dan Berindei (JIRA)
Dan Berindei created ISPN-7165:
----------------------------------
Summary: CACHE_MODE_LOCAL is unsafe with L1 enabled
Key: ISPN-7165
URL: https://issues.jboss.org/browse/ISPN-7165
Project: Infinispan
Issue Type: Bug
Components: Core
Affects Versions: 8.2.4.Final, 9.0.0.Alpha4
Reporter: Dan Berindei
Priority: Minor
The {{CACHE_MODE_LOCAL}} javadoc says:
{quote}
In distributed mode, the modifications performed
during an operation in a non-owner node are going to L1,
if it is enabled, otherwise the operation is a no-op in
that node.
{quote}
I see 2 problems with this:
# Remote get-triggered L1 writes go through an {{L1Synchronizer}} to ensure stale entries don't overwrite newer ones. {{CACHE_MODE_LOCAL}} ones don't have any synchronization, so they could overwrite an existing value with a newer one. To be fair, when reading values from an external source and inserting it in the cache, keeping the cache in sync is kind of tricky even with regular writes.
# Remote get-triggered L1 writes also register on one or more of the key owners as an "L1 requestor", so the owner knows whether it should broadcast an invalidation message when the value changes. (That's in the default configuration; with {{invalidationThreshold > 1}}, the owner will only send unicast invalidations to the individual requestors.) {{CACHE_MODE_LOCAL}} writes do not register on any owner, so their values may never be invalidated.
We should change {{CACHE_MODE_LOCAL}} to never write to L1. If the application wants to write an entry to the local node regardless of ownership, it should use {{SKIP_OWNERSHIP_CHECK}} too, and it should be aware that the cache will not perform the regular L1 invalidation for that entry.
Note that {{putForExternalRead()}} doesn't use {{CACHE_MODE_LOCAL}}, so it shouldn't be affected.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 5 months
[JBoss JIRA] (ISPN-7080) NPE in CacheNotifierImpl by LIRS eviction listener
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-7080?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-7080:
-------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.0.0.Beta1
Assignee: William Burns
Resolution: Done
> NPE in CacheNotifierImpl by LIRS eviction listener
> ---------------------------------------------------
>
> Key: ISPN-7080
> URL: https://issues.jboss.org/browse/ISPN-7080
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 9.0.0.Alpha4
> Environment: * Infinispan branch: Upstream master
> Reporter: ted won
> Assignee: William Burns
> Labels: core, eviction, lirs
> Fix For: 9.0.0.Beta1
>
>
> The LIRSEvictionFunctionalTest.testSimpleEvictionMaxEntries unit test method in core module fails and throws NPE in CacheNotifierImpl.notifyCacheEntriesEvicted method when trying to reinsert the first keys again.
> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/o...
> It reproduces only with LIRS eviction strategy and eviction listener.
> However, LRU and UNORDERED eviction strategies are working properly.
> It makes impossible the use of eviction listeners with LIRS eviction strategy.
> It's able to reproduce by:
> - inserting 20 keys - values in a 10 sized cache ( keys from key-1 to key-20),
> - and then reinsert the first 10 keys (key-1 to key-10)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 5 months
[JBoss JIRA] (ISPN-7164) infinispan exception during wildfly app server startup
by Purush Reddy (JIRA)
Purush Reddy created ISPN-7164:
----------------------------------
Summary: infinispan exception during wildfly app server startup
Key: ISPN-7164
URL: https://issues.jboss.org/browse/ISPN-7164
Project: Infinispan
Issue Type: Bug
Environment: RHEL 7, VMWare ESX 5.5,
Wildfly 8.2.1 cluster with 6 nodes
Reporter: Purush Reddy
We are seeing the following exception in the cluster node startup. Please note that this issue is sporadic.
This seems like race condition. the application on one node is calling evictAll() before the other node (throwing the exception) fully starts up and initializes the region.
2016-10-26 17:57:56,606 ERROR [OOB-18,shared=udp]-[org.infinispan.remoting.InboundInvocationHandlerImpl] ISPN000260: Exception executing command: java.lang.NullPointerException
at org.hibernate.cache.infinispan.util.EvictAllCommand.perform(EvictAllCommand.java:64)
at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:95)
at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:186)
at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:84)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:259)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:211)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:460)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:667)
at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
at org.jgroups.JChannel.up(JChannel.java:708)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1015)
at org.jgroups.protocols.RSVP.up(RSVP.java:187)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1010)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:391)
at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:774)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:570)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:147)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:185)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:301)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:303)
at org.jgroups.protocols.Discovery.up(Discovery.java:379)
at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2641)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1429)
at org.jgroups.protocols.TP$MyHandler.run(TP.java:1615)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 5 months
[JBoss JIRA] (ISPN-7163) InfinispanNodeFailureTest.killedNodeDoesNotBreakReplaceCommand random failures
by Dan Berindei (JIRA)
Dan Berindei created ISPN-7163:
----------------------------------
Summary: InfinispanNodeFailureTest.killedNodeDoesNotBreakReplaceCommand random failures
Key: ISPN-7163
URL: https://issues.jboss.org/browse/ISPN-7163
Project: Infinispan
Issue Type: Bug
Components: Test Suite - Core
Reporter: Dan Berindei
Priority: Critical
Fix For: 9.0.0.Beta1
Attachments: InfinispanNodeFailureTest_20161021.log
{noformat}
00:07:56,440 ERROR (testng-InfinispanNodeFailureTest:[]) [TestSuiteProgress] Test failed: org.infinispan.tx.InfinispanNodeFailureTest.killedNodeDoesNotBreakReplaceCommand
java.lang.AssertionError: expected:<false> but was:<true>
at org.testng.AssertJUnit.fail(AssertJUnit.java:59) ~[testng-6.8.8.jar:?]
at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364) ~[testng-6.8.8.jar:?]
at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80) ~[testng-6.8.8.jar:?]
at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:185) ~[testng-6.8.8.jar:?]
at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:192) ~[testng-6.8.8.jar:?]
at org.infinispan.tx.InfinispanNodeFailureTest.killedNodeDoesNotBreakReplaceCommand(InfinispanNodeFailureTest.java:135) ~[test-classes/:?]
{noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 5 months
[JBoss JIRA] (ISPN-7162) SiteManualSwitchTest random failures
by Dan Berindei (JIRA)
Dan Berindei created ISPN-7162:
----------------------------------
Summary: SiteManualSwitchTest random failures
Key: ISPN-7162
URL: https://issues.jboss.org/browse/ISPN-7162
Project: Infinispan
Issue Type: Bug
Reporter: Dan Berindei
Priority: Critical
The hotrod-client X-site tests can't run concurrently because all join the same RELAY2 bridge cluster:
{noformat}
23:00:03,696 DEBUG (Incoming-1,global,_SiteManualSwitchTest-NodeA-9028:LON:[]) [GMS] _SiteManualSwitchTest-NodeA-9028:LON: installing view [_SiteManualSwitchTest-NodeA-9028:LON|2] (3) [_SiteManualSwitchTest-NodeA-9028:LON, _SiteDownFailoverTest-NodeA-45170:LON, _SiteManualSwitchTest-NodeC-31279:NYC]
{noformat}
This means some writes end up backed up to the wrong site, leading to failures like this:
{noformat}
23:00:23,966 ERROR (testng-SiteManualSwitchTest) [TestSuiteProgress] Test failed: org.infinispan.client.hotrod.xsite.SiteManualSwitchTest.testManualClusterSwitch java.lang.AssertionError: expected:<1> but was:<2> at org.testng.AssertJUnit.fail(AssertJUnit.java:59) at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364) at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80) at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:170) at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:177) at org.infinispan.client.hotrod.xsite.AbstractHotRodSiteFailoverTest.assertSiteHit(AbstractHotRodSiteFailoverTest.java:147) at org.infinispan.client.hotrod.xsite.SiteManualSwitchTest.assertSingleSiteHit(SiteManualSwitchTest.java:47) at org.infinispan.client.hotrod.xsite.SiteManualSwitchTest.testManualClusterSwitch(SiteManualSwitchTest.java:33)
{noformat}
{noformat}
00:35:31,372 ERROR (testng-SiteManualSwitchTest:[]) [TestSuiteProgress] Test failed: org.infinispan.client.hotrod.xsite.SiteManualSwitchTest.testManualClusterSwitch
java.lang.AssertionError: expected:<0> but was:<1>
at org.testng.AssertJUnit.fail(AssertJUnit.java:59) ~[testng-6.8.8.jar:?]
at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364) ~[testng-6.8.8.jar:?]
at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80) ~[testng-6.8.8.jar:?]
at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:245) ~[testng-6.8.8.jar:?]
at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:252) ~[testng-6.8.8.jar:?]
at org.infinispan.client.hotrod.xsite.AbstractHotRodSiteFailoverTest.lambda$null$47(AbstractHotRodSiteFailoverTest.java:126) ~[test-classes/:?]
at org.infinispan.client.hotrod.xsite.AbstractHotRodSiteFailoverTest$$Lambda$651/559455232.accept(Unknown Source) ~[?:?]
at java.util.ArrayList.forEach(ArrayList.java:1249) ~[?:1.8.0_45]
at org.infinispan.client.hotrod.xsite.AbstractHotRodSiteFailoverTest.lambda$assertNoHits$48(AbstractHotRodSiteFailoverTest.java:123) ~[test-classes/:?]
at org.infinispan.client.hotrod.xsite.AbstractHotRodSiteFailoverTest$$Lambda$650/2126831304.accept(Unknown Source) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1280) ~[?:1.8.0_45]
at org.infinispan.client.hotrod.xsite.AbstractHotRodSiteFailoverTest.assertNoHits(AbstractHotRodSiteFailoverTest.java:122) ~[test-classes/:?]
at org.infinispan.client.hotrod.xsite.SiteManualSwitchTest.testManualClusterSwitch(SiteManualSwitchTest.java:29) ~[test-classes/:?]
{noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 5 months
[JBoss JIRA] (ISPN-7148) Unable to create/edit Cache Configurations when Security is not enabled
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-7148?page=com.atlassian.jira.plugin.... ]
Ryan Emerson resolved ISPN-7148.
--------------------------------
Fix Version/s: 9.0.0.Beta1
Assignee: Vladimir Blagojevic (was: Ryan Emerson)
Resolution: Done
> Unable to create/edit Cache Configurations when Security is not enabled
> -----------------------------------------------------------------------
>
> Key: ISPN-7148
> URL: https://issues.jboss.org/browse/ISPN-7148
> Project: Infinispan
> Issue Type: Bug
> Components: Console
> Affects Versions: 9.0.0.Alpha4
> Reporter: Ryan Emerson
> Assignee: Vladimir Blagojevic
> Fix For: 9.0.0.Beta1
>
>
> Unable to create/edit cache configurations/templates as the security/authorization DMR is not created when required:
> "WFLYCTL0175: Resource [
> ("profile" => "clustered"),
> ("subsystem" => "datagrid-infinispan"),
> ("cache-container" => "clustered"),
> ("configurations" => "CONFIGURATIONS"),
> ("distributed-cache-configuration" => "asdasdad"),
> ("security" => "SECURITY")
> ] does not exist; a resource at address [
> ("profile" => "clustered"),
> ("subsystem" => "datagrid-infinispan"),
> ("cache-container" => "clustered"),
> ("configurations" => "CONFIGURATIONS"),
> ("distributed-cache-configuration" => "asdasdad"),
> ("security" => "SECURITY"),
> ("authorization" => "AUTHORIZATION")
> ] cannot be created until all ancestor resources have been added"
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 5 months
[JBoss JIRA] (ISPN-7161) RemoteGetFailureTest.testOneOwnerSuspectedNoFilter[staggered=false] random failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-7161?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-7161:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/4642
> RemoteGetFailureTest.testOneOwnerSuspectedNoFilter[staggered=false] random failures
> -----------------------------------------------------------------------------------
>
> Key: ISPN-7161
> URL: https://issues.jboss.org/browse/ISPN-7161
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Labels: testsuite_stability
> Fix For: 9.0.0.Beta1
>
>
> {noformat}
> java.lang.AssertionError:
> at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
> at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24)
> at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:33)
> at org.infinispan.distribution.RemoteGetFailureTest.testOneOwnerSuspectedNoFilter(RemoteGetFailureTest.java:223)
> {noformat}
> The test installs a new view and then immediately unblocks the latch on node 1. Since ISPN-5469, {{RequestCorrelator}} doesn't receive the view synchronously any more, so node 0 could receive the response from node 1 before receiving the view and suspecting node 2.
--
This message was sent by Atlassian JIRA
(v7.2.2#72004)
9 years, 5 months
[JBoss JIRA] (ISPN-7161) RemoteGetFailureTest.testOneOwnerSuspectedNoFilter[staggered=false] random failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-7161?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-7161:
-------------------------------
Description:
{noformat}
java.lang.AssertionError:
at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24)
at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:33)
at org.infinispan.distribution.RemoteGetFailureTest.testOneOwnerSuspectedNoFilter(RemoteGetFailureTest.java:223)
{noformat}
The test installs a new view and then immediately unblocks the latch on node 1. Since ISPN-5469, {{RequestCorrelator}} doesn't receive the view synchronously any more, so node 0 could receive the response from node 1 before receiving the view and suspecting node 2.
was:
{noformat}
java.lang.AssertionError: at org.testng.AssertJUnit.fail(AssertJUnit.java:59) at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24) at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:33) at org.infinispan.distribution.RemoteGetFailureTest.testOneOwnerSuspectedNoFilter(RemoteGetFailureTest.java:223)
{noformat}
The test installs a new view and then immediately unblocks the latch on node 1. Since ISPN-5469, {{RequestCorrelator}} doesn't receive the view synchronously any more, so node 0 could receive the response from node 1 before receiving the view and suspecting node 2.
> RemoteGetFailureTest.testOneOwnerSuspectedNoFilter[staggered=false] random failures
> -----------------------------------------------------------------------------------
>
> Key: ISPN-7161
> URL: https://issues.jboss.org/browse/ISPN-7161
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Labels: testsuite_stability
> Fix For: 9.0.0.Beta1
>
>
> {noformat}
> java.lang.AssertionError:
> at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
> at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24)
> at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:33)
> at org.infinispan.distribution.RemoteGetFailureTest.testOneOwnerSuspectedNoFilter(RemoteGetFailureTest.java:223)
> {noformat}
> The test installs a new view and then immediately unblocks the latch on node 1. Since ISPN-5469, {{RequestCorrelator}} doesn't receive the view synchronously any more, so node 0 could receive the response from node 1 before receiving the view and suspecting node 2.
--
This message was sent by Atlassian JIRA
(v7.2.2#72004)
9 years, 5 months