[JBoss JIRA] (ISPN-9264) StringBasedStoreInvalidationCacheSyncIT Failures
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-9264?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-9264:
------------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> StringBasedStoreInvalidationCacheSyncIT Failures
> ------------------------------------------------
>
> Key: ISPN-9264
> URL: https://issues.jboss.org/browse/ISPN-9264
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 9.3.0.CR1
> Reporter: Ryan Emerson
> Assignee: Ryan Emerson
> Fix For: 9.3.0.Final
>
>
> {code:java}
> testRemoveNonExistentEntry
> java.lang.AssertionError: Read timeout at org.infinispan.server.test.cs.jdbc.multinode.StringBasedStoreInvalidationCacheSyncIT.testRemoveNonExistentEntry(StringBasedStoreInvalidationCacheSyncIT.java:82)
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (ISPN-9188) Cancelling the last segment doesn't remove transfer task
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-9188?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-9188:
-------------------------------
Status: Open (was: New)
> Cancelling the last segment doesn't remove transfer task
> --------------------------------------------------------
>
> Key: ISPN-9188
> URL: https://issues.jboss.org/browse/ISPN-9188
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.3.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 9.3.0.Final
>
>
> If an {{InboundTransferTask}} has some completed segments and the last segment in an is cancelled with {{cancelSegments()}} (because the local node is no longer an owner), the task is never completed or removed from the list of active transfers.
> In the log below, NodeA adds segment 243 in topology 10. The transfer fails because NodeB shut down, so it's retried in topology 11 from NodeC.,
> {noformat}
> 13:19:17,057 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] Received new topology for cache cluster-listener, isRebalance = true, isMember = true, topology = CacheTopology{id=10, phase=READ_OLD_WRITE_ALL, rebalanceId=4, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (3)[Test-NodeA-53146: 79+82, Test-NodeB-56589: 90+84, Test-NodeC-22289: 87+90]}, pendingCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (4)[Test-NodeA-53146: 55+54, Test-NodeB-56589: 67+60, Test-NodeC-22289: 67+69, Test-NodeD-8561: 67+73]}, unionCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (4)[Test-NodeA-53146: 79+87, Test-NodeB-56589: 90+85, Test-NodeC-22289: 87+93, Test-NodeD-8561: 0+140]}, actualMembers=[Test-NodeA-53146, Test-NodeB-56589, Test-NodeC-22289, Test-NodeD-8561], persistentUUIDs=[dff2268e-e59e-48cc-809d-dcfe0774d1d8, 01379420-7999-4abf-abd8-2777454c035a, e9090b94-dc69-49d8-a2dc-ae0ad7f18fe0, 8edade14-6827-4b67-93d2-49b93844d8ad]}
> 13:19:17,097 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] On cache cluster-listener we have: added segments: {18 71 91-92 243}; removed segments: {}
> 13:19:17,097 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeB-56589 for segments {18 71 91 243}
> 13:19:17,098 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeC-22289 for segments {92}
> 13:19:17,107 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Received new topology for cache cluster-listener, isRebalance = false, isMember = true, topology = CacheTopology{id=11, phase=READ_OLD_WRITE_ALL, rebalanceId=4, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 122+39, Test-NodeC-22289: 134+43]}, pendingCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 120+31, Test-NodeC-22289: 136+42]}, unionCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 122+60, Test-NodeC-22289: 134+71]}, actualMembers=[Test-NodeA-53146, Test-NodeC-22289, Test-NodeD-8561], persistentUUIDs=[dff2268e-e59e-48cc-809d-dcfe0774d1d8, e9090b94-dc69-49d8-a2dc-ae0ad7f18fe0, 8edade14-6827-4b67-93d2-49b93844d8ad]}
> 13:19:17,140 TRACE (stateTransferExecutor-thread-Test-NodeA-p22713-t1:[StateRequest-cluster-listener]) [StateConsumerImpl] Inbound transfer finished: InboundTransferTask{segments={18 71 91 243}, finishedSegments={}, unfinishedSegments={18 71 91 243}, source=Test-NodeB-56589, isCancelled=false, completionFuture=java.util.concurrent.CompletableFuture@5237f653[Completed exceptionally], topologyId=10, timeout=240000, cacheName=cluster-listener}
> 13:19:17,156 TRACE (remote-thread-Test-NodeA-p22710-t5:[cluster-listener]) [StateConsumerImpl] Segments not received yet for cache cluster-listener: {Test-NodeB-56589=[InboundTransferTask{segments={18 71 91 243}, finishedSegments={}, unfinishedSegments={18 71 91 243}, source=Test-NodeB-56589, isCancelled=false, completionFuture=java.util.concurrent.CompletableFuture@5237f653[Completed exceptionally], topologyId=10, timeout=240000, cacheName=cluster-listener}]}
> 13:19:17,273 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Removing inbound transfers from node Test-NodeB-56589 for segments {18 71 91 243}
> 13:19:17,274 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeC-22289 for segments {18 71 91 243}
> 13:19:17,283 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Received new topology for cache cluster-listener, isRebalance = true, isMember = true, topology = CacheTopology{id=12, phase=READ_OLD_WRITE_ALL, rebalanceId=5, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 122+39, Test-NodeC-22289: 134+43]}, pendingCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (3)[Test-NodeA-53146: 76+78, Test-NodeC-22289: 90+80, Test-NodeD-8561: 90+98]}, unionCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (3)[Test-NodeA-53146: 122+75, Test-NodeC-22289: 134+73, Test-NodeD-8561: 0+188]}, actualMembers=[Test-NodeA-53146, Test-NodeC-22289, Test-NodeD-8561], persistentUUIDs=[dff2268e-e59e-48cc-809d-dcfe0774d1d8, e9090b94-dc69-49d8-a2dc-ae0ad7f18fe0, 8edade14-6827-4b67-93d2-49b93844d8ad]}
> 13:19:17,285 WARN (remote-thread-Test-NodeA-p22710-t6:[cluster-listener]) [StateConsumerImpl] Discarding received cache entries for segment 243 of cache cluster-listener because they do not belong to this node.
> 13:19:17,286 TRACE (remote-thread-Test-NodeA-p22710-t6:[cluster-listener]) [StateConsumerImpl] Segments not received yet for cache cluster-listener: {Test-NodeC-22289=[InboundTransferTask{segments={18 71 91 243}, finishedSegments={18 71 91}, unfinishedSegments={243}, source=Test-NodeC-22289, isCancelled=false, completionFuture=java.util.concurrent.CompletableFuture@2755cd[Not completed, 1 dependents], topologyId=11, timeout=240000, cacheName=cluster-listener}]}
> 13:19:17,428 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] On cache cluster-listener we have: added segments: {45 54-55 63 90 95-98 114-115 119 138-140 155-156 167 174-175 205 220-222}; removed segments: {30 66-67 77 93 190-192 243}
> 13:19:17,429 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [InboundTransferTask] Partially cancelling inbound state transfer from node Test-NodeC-22289, segments {243}
> 13:19:17,429 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeC-22289 for segments {45 54-55 63 90 95-98 114-115 119 138-140 155-156 167 174-175 205 220-222}
> 13:19:17,430 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] notifyEndOfStateTransferIfNeeded: no active transfers
> {noformat}
> The last message is incorrect: the rebalance phase is not confirmed because there are 2 active transfers from NodeC: the 243 one, which wasn't completely cancelled, and the 45...222 one, which hasn't yet started.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (ISPN-9188) Cancelling the last segment doesn't remove transfer task
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-9188?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-9188:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/6059
> Cancelling the last segment doesn't remove transfer task
> --------------------------------------------------------
>
> Key: ISPN-9188
> URL: https://issues.jboss.org/browse/ISPN-9188
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.3.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 9.3.0.Final
>
>
> If an {{InboundTransferTask}} has some completed segments and the last segment in an is cancelled with {{cancelSegments()}} (because the local node is no longer an owner), the task is never completed or removed from the list of active transfers.
> In the log below, NodeA adds segment 243 in topology 10. The transfer fails because NodeB shut down, so it's retried in topology 11 from NodeC.,
> {noformat}
> 13:19:17,057 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] Received new topology for cache cluster-listener, isRebalance = true, isMember = true, topology = CacheTopology{id=10, phase=READ_OLD_WRITE_ALL, rebalanceId=4, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (3)[Test-NodeA-53146: 79+82, Test-NodeB-56589: 90+84, Test-NodeC-22289: 87+90]}, pendingCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (4)[Test-NodeA-53146: 55+54, Test-NodeB-56589: 67+60, Test-NodeC-22289: 67+69, Test-NodeD-8561: 67+73]}, unionCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (4)[Test-NodeA-53146: 79+87, Test-NodeB-56589: 90+85, Test-NodeC-22289: 87+93, Test-NodeD-8561: 0+140]}, actualMembers=[Test-NodeA-53146, Test-NodeB-56589, Test-NodeC-22289, Test-NodeD-8561], persistentUUIDs=[dff2268e-e59e-48cc-809d-dcfe0774d1d8, 01379420-7999-4abf-abd8-2777454c035a, e9090b94-dc69-49d8-a2dc-ae0ad7f18fe0, 8edade14-6827-4b67-93d2-49b93844d8ad]}
> 13:19:17,097 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] On cache cluster-listener we have: added segments: {18 71 91-92 243}; removed segments: {}
> 13:19:17,097 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeB-56589 for segments {18 71 91 243}
> 13:19:17,098 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeC-22289 for segments {92}
> 13:19:17,107 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Received new topology for cache cluster-listener, isRebalance = false, isMember = true, topology = CacheTopology{id=11, phase=READ_OLD_WRITE_ALL, rebalanceId=4, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 122+39, Test-NodeC-22289: 134+43]}, pendingCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 120+31, Test-NodeC-22289: 136+42]}, unionCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 122+60, Test-NodeC-22289: 134+71]}, actualMembers=[Test-NodeA-53146, Test-NodeC-22289, Test-NodeD-8561], persistentUUIDs=[dff2268e-e59e-48cc-809d-dcfe0774d1d8, e9090b94-dc69-49d8-a2dc-ae0ad7f18fe0, 8edade14-6827-4b67-93d2-49b93844d8ad]}
> 13:19:17,140 TRACE (stateTransferExecutor-thread-Test-NodeA-p22713-t1:[StateRequest-cluster-listener]) [StateConsumerImpl] Inbound transfer finished: InboundTransferTask{segments={18 71 91 243}, finishedSegments={}, unfinishedSegments={18 71 91 243}, source=Test-NodeB-56589, isCancelled=false, completionFuture=java.util.concurrent.CompletableFuture@5237f653[Completed exceptionally], topologyId=10, timeout=240000, cacheName=cluster-listener}
> 13:19:17,156 TRACE (remote-thread-Test-NodeA-p22710-t5:[cluster-listener]) [StateConsumerImpl] Segments not received yet for cache cluster-listener: {Test-NodeB-56589=[InboundTransferTask{segments={18 71 91 243}, finishedSegments={}, unfinishedSegments={18 71 91 243}, source=Test-NodeB-56589, isCancelled=false, completionFuture=java.util.concurrent.CompletableFuture@5237f653[Completed exceptionally], topologyId=10, timeout=240000, cacheName=cluster-listener}]}
> 13:19:17,273 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Removing inbound transfers from node Test-NodeB-56589 for segments {18 71 91 243}
> 13:19:17,274 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeC-22289 for segments {18 71 91 243}
> 13:19:17,283 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Received new topology for cache cluster-listener, isRebalance = true, isMember = true, topology = CacheTopology{id=12, phase=READ_OLD_WRITE_ALL, rebalanceId=5, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 122+39, Test-NodeC-22289: 134+43]}, pendingCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (3)[Test-NodeA-53146: 76+78, Test-NodeC-22289: 90+80, Test-NodeD-8561: 90+98]}, unionCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (3)[Test-NodeA-53146: 122+75, Test-NodeC-22289: 134+73, Test-NodeD-8561: 0+188]}, actualMembers=[Test-NodeA-53146, Test-NodeC-22289, Test-NodeD-8561], persistentUUIDs=[dff2268e-e59e-48cc-809d-dcfe0774d1d8, e9090b94-dc69-49d8-a2dc-ae0ad7f18fe0, 8edade14-6827-4b67-93d2-49b93844d8ad]}
> 13:19:17,285 WARN (remote-thread-Test-NodeA-p22710-t6:[cluster-listener]) [StateConsumerImpl] Discarding received cache entries for segment 243 of cache cluster-listener because they do not belong to this node.
> 13:19:17,286 TRACE (remote-thread-Test-NodeA-p22710-t6:[cluster-listener]) [StateConsumerImpl] Segments not received yet for cache cluster-listener: {Test-NodeC-22289=[InboundTransferTask{segments={18 71 91 243}, finishedSegments={18 71 91}, unfinishedSegments={243}, source=Test-NodeC-22289, isCancelled=false, completionFuture=java.util.concurrent.CompletableFuture@2755cd[Not completed, 1 dependents], topologyId=11, timeout=240000, cacheName=cluster-listener}]}
> 13:19:17,428 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] On cache cluster-listener we have: added segments: {45 54-55 63 90 95-98 114-115 119 138-140 155-156 167 174-175 205 220-222}; removed segments: {30 66-67 77 93 190-192 243}
> 13:19:17,429 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [InboundTransferTask] Partially cancelling inbound state transfer from node Test-NodeC-22289, segments {243}
> 13:19:17,429 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeC-22289 for segments {45 54-55 63 90 95-98 114-115 119 138-140 155-156 167 174-175 205 220-222}
> 13:19:17,430 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] notifyEndOfStateTransferIfNeeded: no active transfers
> {noformat}
> The last message is incorrect: the rebalance phase is not confirmed because there are 2 active transfers from NodeC: the 243 one, which wasn't completely cancelled, and the 45...222 one, which hasn't yet started.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (ISPN-9287) Integration test suite fails when JAVA_HOME is not set
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-9287?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-9287:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/6058
> Integration test suite fails when JAVA_HOME is not set
> ------------------------------------------------------
>
> Key: ISPN-9287
> URL: https://issues.jboss.org/browse/ISPN-9287
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 9.3.0.CR1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.3.0.Final
>
>
> The build tries to run the {{kill-server}} Ant target before and after running the integration test suite. The target assumes that the server is running, and if none of ps/jps/lsof/netstat find the server it fails the build:
> {noformat}
> Caused by: org.apache.tools.ant.BuildException: The following error occurred while executing this line:
> /home/jenkins/workspace/polarion-report/5fee2853/server/integration/src/main/ant/infinispan-server.xml:102: Not yet supported on UNIX/WINDOWS favour without working ps/lsof/jps/netstat
> {noformat}
> Normally this doesn't happen because the {{jps}} check is not as strict as the others, passing if {{jps}} finds any java process (and not necessarily a server). But if {{JAVA_HOME}} is not defined, the {{jps}} check also fails:
> {noformat}
> [exec] Execute failed: java.io.IOException: Cannot run program "${env.JAVA_HOME}/bin/jps" (in directory "/home/jenkins/workspace/polarion-report/5fee2853/integrationtests/wildfly-modules"): error=2, No such file or directory
> {noformat}
> We should change the {{kill-server}} target so that it doesn't assume the server is already running. Maybe it could also use ps/jps to find the pid of the server, and lsof/netstat to check that the server really is really stopped after it was killed.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (ISPN-9282) Near-Real-Time indexes lost when server is stopped
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-9282?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-9282:
------------------------------------
Affects Version/s: 9.2.5.Final
9.3.0.CR1
> Near-Real-Time indexes lost when server is stopped
> --------------------------------------------------
>
> Key: ISPN-9282
> URL: https://issues.jboss.org/browse/ISPN-9282
> Project: Infinispan
> Issue Type: Bug
> Components: Indexing, Server
> Affects Versions: 9.3.0.CR1, 9.2.5.Final
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
>
> The server does not respect the stop order of the caches dictated by cache manager and tries to stop all caches in random order.
> When a cache is indexed and uses the "near-real-time" index manager, it relies on the indexing caches being available during shutdown of the data cache so that it can properly flush the index to persistence storage, and this fails due to one or more index caches being unavailable.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (ISPN-9288) Use transform() to decorate the cache used by Hot Rod Transactions
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-9288?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-9288:
------------------------------
Description:
Use the {{transform()}} method instead of checking manually the instance of the class to decorate. Notes: the server wraps the cache to use!
was:
Use the {{transform()}} method instead of checking the instance of the class to decorate.
> Use transform() to decorate the cache used by Hot Rod Transactions
> ------------------------------------------------------------------
>
> Key: ISPN-9288
> URL: https://issues.jboss.org/browse/ISPN-9288
> Project: Infinispan
> Issue Type: Bug
> Components: Hot Rod, Transactions
> Reporter: Pedro Ruivo
> Assignee: Pedro Ruivo
> Fix For: 9.3.0.Final
>
>
> Use the {{transform()}} method instead of checking manually the instance of the class to decorate. Notes: the server wraps the cache to use!
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months