[JBoss JIRA] (ISPN-7489) org.jgroups.protocols.TCP emits errors when node leaves the cluster
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7489?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec commented on ISPN-7489:
-------------------------------------------
Another example:
{noformat}
[transactions-repository-2-pd4c7] 18:26:48,055 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] [GC (Allocation Failure) 808852K->535816K(1013632K), 0.0264148 secs]
[transactions-repository-2-1gk60] 18:26:48,384 ERROR [org.jgroups.protocols.TCP] (jgroups-36,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] [GC (Allocation Failure) 815432K->522173K(1013632K), 0.0094881 secs]
[transactions-repository-2-pd4c7] 18:26:48,857 ERROR [org.jgroups.protocols.TCP] (jgroups-32,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] [GC (Allocation Failure) 801789K->523993K(1013632K), 0.0111717 secs]
[transactions-repository-2-1gk60] 18:26:49,186 ERROR [org.jgroups.protocols.TCP] (jgroups-36,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:49,590 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t18) [Context=transactions][Context=transactions-repository-2-pd4c7]ISPN100003: Finished local rebalance
[transactions-repository-2-pd4c7] 18:26:49,659 ERROR [org.jgroups.protocols.TCP] (jgroups-23,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:49,987 ERROR [org.jgroups.protocols.TCP] (jgroups-32,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] 18:26:50,461 ERROR [org.jgroups.protocols.TCP] (jgroups-23,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:50,789 ERROR [org.jgroups.protocols.TCP] (jgroups-32,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] 18:26:51,262 ERROR [org.jgroups.protocols.TCP] (jgroups-23,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] [GC (Allocation Failure) 706815K->441667K(1013632K), 0.0366909 secs]
[transactions-repository-2-1gk60] 18:26:51,591 ERROR [org.jgroups.protocols.TCP] (jgroups-32,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] 18:26:52,063 ERROR [org.jgroups.protocols.TCP] (jgroups-23,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:52,395 ERROR [org.jgroups.protocols.TCP] (jgroups-32,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] 18:26:52,867 ERROR [org.jgroups.protocols.TCP] (jgroups-23,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:53,197 ERROR [org.jgroups.protocols.TCP] (jgroups-32,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] [GC (Allocation Failure) 803579K->533000K(1013632K), 0.0314847 secs]
[transactions-repository-2-pd4c7] 18:26:53,672 ERROR [org.jgroups.protocols.TCP] (jgroups-21,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:54,001 ERROR [org.jgroups.protocols.TCP] (jgroups-32,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] 18:26:54,474 ERROR [org.jgroups.protocols.TCP] (jgroups-23,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:54,803 ERROR [org.jgroups.protocols.TCP] (jgroups-24,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] 18:26:55,276 ERROR [org.jgroups.protocols.TCP] (jgroups-23,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:55,607 ERROR [org.jgroups.protocols.TCP] (jgroups-24,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:55,916 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (jgroups-32,transactions-repository-2-1gk60) ISPN000094: Received new cluster view for channel clustered: [transactions-repository-2-1gk60|11] (4) [transactions-repository-2-1gk60, transactions-repository-2-pd4c7, transactions-repository-3-6mzvd, transactions-repository-3-bv2mj]
[transactions-repository-2-1gk60] 18:26:55,917 INFO [org.infinispan.CLUSTER] (jgroups-32,transactions-repository-2-1gk60) ISPN100000: Node transactions-repository-3-bv2mj joined the cluster
[transactions-repository-2-pd4c7] 18:26:55,945 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (jgroups-23,transactions-repository-2-pd4c7) ISPN000094: Received new cluster view for channel clustered: [transactions-repository-2-1gk60|11] (4) [transactions-repository-2-1gk60, transactions-repository-2-pd4c7, transactions-repository-3-6mzvd, transactions-repository-3-bv2mj]
[transactions-repository-2-pd4c7] 18:26:55,946 INFO [org.infinispan.CLUSTER] (jgroups-23,transactions-repository-2-pd4c7) ISPN100000: Node transactions-repository-3-bv2mj joined the cluster
[transactions-repository-2-1gk60] 18:26:56,409 ERROR [org.jgroups.protocols.TCP] (jgroups-24,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
[transactions-repository-2-pd4c7] 18:26:56,747 ERROR [org.jgroups.protocols.TCP] (jgroups-32,transactions-repository-2-pd4c7) JGRP000029: transactions-repository-2-pd4c7: failed sending message to transactions-repository-2-jms3k (104 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2415, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=7070, conn_id=1, TP: [cluster_name=cluster]
[transactions-repository-2-1gk60] 18:26:57,210 ERROR [org.jgroups.protocols.TCP] (jgroups-24,transactions-repository-2-1gk60) JGRP000029: transactions-repository-2-1gk60: failed sending message to transactions-repository-2-jms3k (106 bytes): java.net.SocketTimeoutException: connect timed out, headers: RequestCorrelator: corr_id=200, type=RSP, req_id=2426, rsp_expected=true, FORK: cluster:clustered, UNICAST3: DATA, seqno=12125, conn_id=3, TP: [cluster_name=cluster]
{noformat}
Full logs: https://gist.github.com/slaskawi/d789431ab5f16d46136dafffd7d4a89e
> org.jgroups.protocols.TCP emits errors when node leaves the cluster
> -------------------------------------------------------------------
>
> Key: ISPN-7489
> URL: https://issues.jboss.org/browse/ISPN-7489
> Project: Infinispan
> Issue Type: Sub-task
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> [transactions-repository-1-04x09] 18:09:12,193 ERROR [org.jgroups.protocols.TCP] (jgroups-30,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:12,310 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> [transactions-repository-1-04x09] 18:09:12,997 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:13,113 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> {noformat}
> Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * invoke `cd transaction-creator && mvn fabric8:run`
> * Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
> * Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (ISPN-7491) When a node leaves the cluster it emits unsolicited state
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7491?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec updated ISPN-7491:
--------------------------------------
Description:
When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
One of the is this:
{noformat}
[transactions-repository-2-1gk60] 18:26:44,264 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t22) ISPN000336: Finished cluster-wide rebalance for cache ___protobuf_metadata, topology id = 18
[transactions-repository-2-pd4c7] [GC (Allocation Failure) 756168K->489129K(1013632K), 0.0439983 secs]
[transactions-repository-2-1gk60] [GC (Allocation Failure) 686020K->416468K(1013632K), 0.0514641 secs]
[transactions-repository-2-pd4c7] [GC (Allocation Failure) 768745K->501140K(1013632K), 0.0544712 secs]
[transactions-repository-2-1gk60] [GC (Allocation Failure) 696084K->427206K(1013632K), 0.0997416 secs]
[transactions-repository-2-pd4c7] [GC (Allocation Failure) 780756K->516161K(1013632K), 0.0674455 secs]
[transactions-repository-2-jms3k] *** JBossAS process (81) received TERM signal ***
[transactions-repository-2-jms3k] 18:26:46,371 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested via an OS signal
[transactions-repository-2-jms3k] 18:26:46,392 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-5) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
[transactions-repository-2-1gk60] 18:26:46,395 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t25) ISPN000310: Starting cluster-wide rebalance for cache sessions, topology CacheTopology{id=25, rebalanceId=9, currentCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 5+6, transactions-repository-2-pd4c7: 7+3, transactions-repository-3-6mzvd: 8+3]}, pendingCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 7+8, transactions-repository-2-pd4c7: 7+6, transactions-repository-3-6mzvd: 6+6]}, unionCH=null, actualMembers=[transactions-repository-2-1gk60, transactions-repository-2-pd4c7, transactions-repository-3-6mzvd], persistentUUIDs=[d47f6f69-97e8-4385-abd0-f1d43312d8c4, a17aa2e7-5137-4b5b-bf90-266207db7bb2, a000aa28-b1bd-453b-86e9-1628da898c36]}
[transactions-repository-2-1gk60] 18:26:46,395 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t25) [Context=sessions][Context=transactions-repository-2-1gk60]ISPN100002: Started local rebalance
[transactions-repository-2-pd4c7] 18:26:46,402 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,411 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,417 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-jms3k] 18:26:46,430 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-6) DGISPN0002: Stopped sessions cache from clustered container
[transactions-repository-2-pd4c7] 18:26:46,441 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,444 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,444 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,448 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,450 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,450 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,451 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,455 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,456 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,453 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,466 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,470 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,471 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,466 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,473 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,495 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,497 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,473 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t60) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,494 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t61) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,490 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t62) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,477 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,476 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,476 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,475 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t57) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,475 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t59) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,475 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,474 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-1gk60] 18:26:46,513 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t6) [Context=sessions][Context=transactions-repository-2-1gk60]ISPN100003: Finished local rebalance
[transactions-repository-2-pd4c7] 18:26:46,514 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t57) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,514 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t60) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,515 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t60) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,515 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,515 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,515 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,516 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,516 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t62) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,516 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-1gk60] 18:26:46,527 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t14) [Context=sessions][Context=transactions-repository-2-pd4c7]ISPN100003: Finished local rebalance
[transactions-repository-2-pd4c7] 18:26:46,517 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,517 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-jms3k] 18:26:46,529 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-8) WFLYJCA0019: Stopped Driver service with driver-name = h2
[transactions-repository-2-pd4c7] 18:26:46,536 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,541 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-1gk60] 18:26:46,548 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t19) [Context=sessions][Context=transactions-repository-3-6mzvd]ISPN100003: Finished local rebalance
[transactions-repository-2-1gk60] 18:26:46,549 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t19) ISPN000336: Finished cluster-wide rebalance for cache sessions, topology id = 25
[transactions-repository-2-pd4c7] 18:26:46,558 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,558 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-1gk60] 18:26:46,574 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) ISPN000310: Starting cluster-wide rebalance for cache default, topology CacheTopology{id=21, rebalanceId=10, currentCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 5+6, transactions-repository-2-pd4c7: 7+3, transactions-repository-3-6mzvd: 8+3]}, pendingCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 7+8, transactions-repository-2-pd4c7: 7+6, transactions-repository-3-6mzvd: 6+6]}, unionCH=null, actualMembers=[transactions-repository-2-1gk60, transactions-repository-2-pd4c7, transactions-repository-3-6mzvd], persistentUUIDs=[d47f6f69-97e8-4385-abd0-f1d43312d8c4, a17aa2e7-5137-4b5b-bf90-266207db7bb2, a000aa28-b1bd-453b-86e9-1628da898c36]}
[transactions-repository-2-1gk60] 18:26:46,574 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) [Context=default][Context=transactions-repository-2-1gk60]ISPN100002: Started local rebalance
[transactions-repository-2-pd4c7] 18:26:46,536 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,563 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,517 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,563 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,517 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-1gk60] 18:26:46,591 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t2) ISPN000310: Starting cluster-wide rebalance for cache memcachedCache, topology CacheTopology{id=21, rebalanceId=10, currentCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 5+6, transactions-repository-2-pd4c7: 7+3, transactions-repository-3-6mzvd: 8+3]}, pendingCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 7+8, transactions-repository-2-pd4c7: 7+6, transactions-repository-3-6mzvd: 6+6]}, unionCH=null, actualMembers=[transactions-repository-2-1gk60, transactions-repository-2-pd4c7, transactions-repository-3-6mzvd], persistentUUIDs=[d47f6f69-97e8-4385-abd0-f1d43312d8c4, a17aa2e7-5137-4b5b-bf90-266207db7bb2, a000aa28-b1bd-453b-86e9-1628da898c36]}
[transactions-repository-2-1gk60] 18:26:46,596 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t2) [Context=memcachedCache][Context=transactions-repository-2-1gk60]ISPN100002: Started local rebalance
[transactions-repository-2-1gk60] 18:26:46,602 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t25) [Context=default][Context=transactions-repository-2-1gk60]ISPN100003: Finished local rebalance
[transactions-repository-2-jms3k] 18:26:46,599 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) DGISPN0002: Stopped default cache from clustered container
[transactions-repository-2-pd4c7] 18:26:46,609 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,610 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,610 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,610 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,612 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,612 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-1gk60] 18:26:46,635 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) [Context=default][Context=transactions-repository-2-pd4c7]ISPN100003: Finished local rebalance
[transactions-repository-2-1gk60] 18:26:46,640 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t15) [Context=default][Context=transactions-repository-3-6mzvd]ISPN100003: Finished local rebalance
[transactions-repository-2-1gk60] 18:26:46,642 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t15) ISPN000336: Finished cluster-wide rebalance for cache default, topology id = 21
[transactions-repository-2-1gk60] 18:26:46,644 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t20) [Context=memcachedCache][Context=transactions-repository-2-pd4c7]ISPN100003: Finished local rebalance
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t62) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-1gk60] 18:26:46,661 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t9) [Context=memcachedCache][Context=transactions-repository-3-6mzvd]ISPN100003: Finished local rebalance
[transactions-repository-2-1gk60] 18:26:46,663 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t22) [Context=memcachedCache][Context=transactions-repository-2-1gk60]ISPN100003: Finished local rebalance
[transactions-repository-2-1gk60] 18:26:46,664 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t22) ISPN000336: Finished cluster-wide rebalance for cache memcachedCache, topology id = 21
[transactions-repository-2-pd4c7] 18:26:46,668 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,668 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-jms3k] 18:26:46,673 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0002: Stopped memcachedCache cache from clustered container
[transactions-repository-2-pd4c7] 18:26:46,674 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,675 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,679 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,679 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,686 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,687 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,713 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,715 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,722 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,722 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,728 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,728 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,729 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,729 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,747 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,748 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,753 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,753 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,755 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,756 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,757 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,758 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,759 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,760 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,760 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,760 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,763 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,765 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t62) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,765 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,766 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,766 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,765 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,763 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,764 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,774 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,774 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,777 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,777 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,778 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,778 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,778 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-jms3k] 18:26:46,778 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-2) DGISPN0002: Stopped transactions cache from clustered container
[transactions-repository-2-pd4c7] 18:26:46,778 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,784 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,784 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,787 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
[transactions-repository-2-pd4c7] 18:26:46,787 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
[transactions-repository-2-1gk60] 18:26:46,802 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 6 of cache transactions
{noformat}
Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/d789431ab5f16d46136dafffd7d4a89e
Steps to reproduce:
* Start local OpenShift Cluster
* invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
* invoke `cd transaction-creator && mvn fabric8:run`
* Start Spring Session Demo `cd session-demo && mvn fabric8:run`
* Create a client which inserts data (`watch -n 0.5 curl http://<spring-session-demo-pod-ip>/sessions`) and at the same time invoke the rolling update: `oc deploy transactions-repository --latest -n myproject`
* Observe logs `kubetail -l environment=infrastructure`
was:
When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
One of the is this:
{noformat}
[transactions-repository-1-2sn4z] 18:21:20,548 INFO [org.jboss.as.protocol] (management task-9) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 2,5,management-handler-thread]
{noformat}
The event seems to be emitted on steady cluster state without any reason.
Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
Steps to reproduce:
* Start local OpenShift Cluster
* invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
* invoke `cd transaction-creator && mvn fabric8:run`
* Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
* Observe logs `kubetail -l environment=infrastructure`
> When a node leaves the cluster it emits unsolicited state
> ---------------------------------------------------------
>
> Key: ISPN-7491
> URL: https://issues.jboss.org/browse/ISPN-7491
> Project: Infinispan
> Issue Type: Sub-task
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> [transactions-repository-2-1gk60] 18:26:44,264 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t22) ISPN000336: Finished cluster-wide rebalance for cache ___protobuf_metadata, topology id = 18
> [transactions-repository-2-pd4c7] [GC (Allocation Failure) 756168K->489129K(1013632K), 0.0439983 secs]
> [transactions-repository-2-1gk60] [GC (Allocation Failure) 686020K->416468K(1013632K), 0.0514641 secs]
> [transactions-repository-2-pd4c7] [GC (Allocation Failure) 768745K->501140K(1013632K), 0.0544712 secs]
> [transactions-repository-2-1gk60] [GC (Allocation Failure) 696084K->427206K(1013632K), 0.0997416 secs]
> [transactions-repository-2-pd4c7] [GC (Allocation Failure) 780756K->516161K(1013632K), 0.0674455 secs]
> [transactions-repository-2-jms3k] *** JBossAS process (81) received TERM signal ***
> [transactions-repository-2-jms3k] 18:26:46,371 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested via an OS signal
> [transactions-repository-2-jms3k] 18:26:46,392 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-5) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
> [transactions-repository-2-1gk60] 18:26:46,395 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t25) ISPN000310: Starting cluster-wide rebalance for cache sessions, topology CacheTopology{id=25, rebalanceId=9, currentCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 5+6, transactions-repository-2-pd4c7: 7+3, transactions-repository-3-6mzvd: 8+3]}, pendingCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 7+8, transactions-repository-2-pd4c7: 7+6, transactions-repository-3-6mzvd: 6+6]}, unionCH=null, actualMembers=[transactions-repository-2-1gk60, transactions-repository-2-pd4c7, transactions-repository-3-6mzvd], persistentUUIDs=[d47f6f69-97e8-4385-abd0-f1d43312d8c4, a17aa2e7-5137-4b5b-bf90-266207db7bb2, a000aa28-b1bd-453b-86e9-1628da898c36]}
> [transactions-repository-2-1gk60] 18:26:46,395 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t25) [Context=sessions][Context=transactions-repository-2-1gk60]ISPN100002: Started local rebalance
> [transactions-repository-2-pd4c7] 18:26:46,402 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,411 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,417 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-jms3k] 18:26:46,430 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-6) DGISPN0002: Stopped sessions cache from clustered container
> [transactions-repository-2-pd4c7] 18:26:46,441 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,444 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,444 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,448 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,450 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,450 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,451 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,455 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,456 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,453 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,466 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,470 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,471 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,466 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,473 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,495 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,497 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,473 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t60) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,494 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t61) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,490 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t62) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,477 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,476 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t24) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,476 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,475 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t57) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,475 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t59) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,475 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t54) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,474 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-1gk60] 18:26:46,513 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t6) [Context=sessions][Context=transactions-repository-2-1gk60]ISPN100003: Finished local rebalance
> [transactions-repository-2-pd4c7] 18:26:46,514 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t57) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,514 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t60) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,515 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t60) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,515 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,515 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,515 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,516 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,516 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t62) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,516 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-1gk60] 18:26:46,527 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t14) [Context=sessions][Context=transactions-repository-2-pd4c7]ISPN100003: Finished local rebalance
> [transactions-repository-2-pd4c7] 18:26:46,517 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,517 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-jms3k] 18:26:46,529 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-8) WFLYJCA0019: Stopped Driver service with driver-name = h2
> [transactions-repository-2-pd4c7] 18:26:46,536 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,541 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-1gk60] 18:26:46,548 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t19) [Context=sessions][Context=transactions-repository-3-6mzvd]ISPN100003: Finished local rebalance
> [transactions-repository-2-1gk60] 18:26:46,549 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t19) ISPN000336: Finished cluster-wide rebalance for cache sessions, topology id = 25
> [transactions-repository-2-pd4c7] 18:26:46,558 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,558 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-1gk60] 18:26:46,574 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) ISPN000310: Starting cluster-wide rebalance for cache default, topology CacheTopology{id=21, rebalanceId=10, currentCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 5+6, transactions-repository-2-pd4c7: 7+3, transactions-repository-3-6mzvd: 8+3]}, pendingCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 7+8, transactions-repository-2-pd4c7: 7+6, transactions-repository-3-6mzvd: 6+6]}, unionCH=null, actualMembers=[transactions-repository-2-1gk60, transactions-repository-2-pd4c7, transactions-repository-3-6mzvd], persistentUUIDs=[d47f6f69-97e8-4385-abd0-f1d43312d8c4, a17aa2e7-5137-4b5b-bf90-266207db7bb2, a000aa28-b1bd-453b-86e9-1628da898c36]}
> [transactions-repository-2-1gk60] 18:26:46,574 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) [Context=default][Context=transactions-repository-2-1gk60]ISPN100002: Started local rebalance
> [transactions-repository-2-pd4c7] 18:26:46,536 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,563 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,517 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,563 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,517 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-1gk60] 18:26:46,591 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t2) ISPN000310: Starting cluster-wide rebalance for cache memcachedCache, topology CacheTopology{id=21, rebalanceId=10, currentCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 5+6, transactions-repository-2-pd4c7: 7+3, transactions-repository-3-6mzvd: 8+3]}, pendingCH=DefaultConsistentHash{ns=20, owners = (3)[transactions-repository-2-1gk60: 7+8, transactions-repository-2-pd4c7: 7+6, transactions-repository-3-6mzvd: 6+6]}, unionCH=null, actualMembers=[transactions-repository-2-1gk60, transactions-repository-2-pd4c7, transactions-repository-3-6mzvd], persistentUUIDs=[d47f6f69-97e8-4385-abd0-f1d43312d8c4, a17aa2e7-5137-4b5b-bf90-266207db7bb2, a000aa28-b1bd-453b-86e9-1628da898c36]}
> [transactions-repository-2-1gk60] 18:26:46,596 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t2) [Context=memcachedCache][Context=transactions-repository-2-1gk60]ISPN100002: Started local rebalance
> [transactions-repository-2-1gk60] 18:26:46,602 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t25) [Context=default][Context=transactions-repository-2-1gk60]ISPN100003: Finished local rebalance
> [transactions-repository-2-jms3k] 18:26:46,599 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) DGISPN0002: Stopped default cache from clustered container
> [transactions-repository-2-pd4c7] 18:26:46,609 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,610 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,610 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,610 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,612 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,611 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,612 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-1gk60] 18:26:46,635 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) [Context=default][Context=transactions-repository-2-pd4c7]ISPN100003: Finished local rebalance
> [transactions-repository-2-1gk60] 18:26:46,640 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t15) [Context=default][Context=transactions-repository-3-6mzvd]ISPN100003: Finished local rebalance
> [transactions-repository-2-1gk60] 18:26:46,642 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t15) ISPN000336: Finished cluster-wide rebalance for cache default, topology id = 21
> [transactions-repository-2-1gk60] 18:26:46,644 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t20) [Context=memcachedCache][Context=transactions-repository-2-pd4c7]ISPN100003: Finished local rebalance
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t55) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,641 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t62) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-1gk60] 18:26:46,661 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t9) [Context=memcachedCache][Context=transactions-repository-3-6mzvd]ISPN100003: Finished local rebalance
> [transactions-repository-2-1gk60] 18:26:46,663 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t22) [Context=memcachedCache][Context=transactions-repository-2-1gk60]ISPN100003: Finished local rebalance
> [transactions-repository-2-1gk60] 18:26:46,664 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t22) ISPN000336: Finished cluster-wide rebalance for cache memcachedCache, topology id = 21
> [transactions-repository-2-pd4c7] 18:26:46,668 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,668 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-jms3k] 18:26:46,673 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0002: Stopped memcachedCache cache from clustered container
> [transactions-repository-2-pd4c7] 18:26:46,674 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,675 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,679 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,679 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,686 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,687 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,713 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,715 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,722 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,722 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,728 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,728 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,729 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,729 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,747 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,748 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,753 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,753 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,755 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,756 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,757 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,758 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,759 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,760 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,760 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,760 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,763 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t64) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,765 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t62) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,765 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t65) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,766 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t58) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,766 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,765 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,763 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,764 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,774 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,774 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,777 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,777 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,778 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t53) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,778 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,778 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-jms3k] 18:26:46,778 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-2) DGISPN0002: Stopped transactions cache from clustered container
> [transactions-repository-2-pd4c7] 18:26:46,778 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,784 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,784 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,787 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t63) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 19 of cache transactions
> [transactions-repository-2-pd4c7] 18:26:46,787 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t56) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 17 of cache transactions
> [transactions-repository-2-1gk60] 18:26:46,802 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p7-t66) ISPN000396: Received unsolicited state from node transactions-repository-2-jms3k for segment 6 of cache transactions
> {noformat}
> Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/d789431ab5f16d46136dafffd7d4a89e
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * invoke `cd transaction-creator && mvn fabric8:run`
> * Start Spring Session Demo `cd session-demo && mvn fabric8:run`
> * Create a client which inserts data (`watch -n 0.5 curl http://<spring-session-demo-pod-ip>/sessions`) and at the same time invoke the rolling update: `oc deploy transactions-repository --latest -n myproject`
> * Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (ISPN-7490) "cancelled task by interrupting thread Thread" is emitted without any reason
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7490?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec updated ISPN-7490:
--------------------------------------
Description:
When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
One of the is this:
{noformat}
[transactions-repository-1-2sn4z] 18:21:20,548 INFO [org.jboss.as.protocol] (management task-9) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 2,5,management-handler-thread]
{noformat}
The event seems to be emitted on steady cluster state without any reason.
Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
Steps to reproduce:
* Start local OpenShift Cluster
* invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
* invoke `cd transaction-creator && mvn fabric8:run`
* Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
* Observe logs `kubetail -l environment=infrastructure`
was:
When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
One of the is this:
{noformat}
[transactions-repository-1-04x09] 18:09:12,193 ERROR [org.jgroups.protocols.TCP] (jgroups-30,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
[transactions-repository-1-1f8dx] 18:09:12,310 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
[transactions-repository-1-04x09] 18:09:12,997 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
[transactions-repository-1-1f8dx] 18:09:13,113 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
{noformat}
Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
Steps to reproduce:
* Start local OpenShift Cluster
* invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
* invoke `cd transaction-creator && mvn fabric8:run`
* Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
* Observe logs `kubetail -l environment=infrastructure`
> "cancelled task by interrupting thread Thread" is emitted without any reason
> ----------------------------------------------------------------------------
>
> Key: ISPN-7490
> URL: https://issues.jboss.org/browse/ISPN-7490
> Project: Infinispan
> Issue Type: Sub-task
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> [transactions-repository-1-2sn4z] 18:21:20,548 INFO [org.jboss.as.protocol] (management task-9) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 2,5,management-handler-thread]
> {noformat}
> The event seems to be emitted on steady cluster state without any reason.
> Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * invoke `cd transaction-creator && mvn fabric8:run`
> * Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
> * Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (ISPN-7490) "cancelled task by interrupting thread Thread" is emitted without any reason
by Sebastian Łaskawiec (JIRA)
Sebastian Łaskawiec created ISPN-7490:
-----------------------------------------
Summary: "cancelled task by interrupting thread Thread" is emitted without any reason
Key: ISPN-7490
URL: https://issues.jboss.org/browse/ISPN-7490
Project: Infinispan
Issue Type: Sub-task
Components: Cloud Integrations, Core
Affects Versions: 9.0.0.CR1
Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
* Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
Reporter: Sebastian Łaskawiec
When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
One of the is this:
{noformat}
[transactions-repository-1-04x09] 18:09:12,193 ERROR [org.jgroups.protocols.TCP] (jgroups-30,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
[transactions-repository-1-1f8dx] 18:09:12,310 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
[transactions-repository-1-04x09] 18:09:12,997 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
[transactions-repository-1-1f8dx] 18:09:13,113 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
{noformat}
Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
Steps to reproduce:
* Start local OpenShift Cluster
* invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
* invoke `cd transaction-creator && mvn fabric8:run`
* Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
* Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (ISPN-7489) org.jgroups.protocols.TCP emits errors when node leaves the cluster
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7489?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec updated ISPN-7489:
--------------------------------------
Description:
When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
One of the is this:
{noformat}
[transactions-repository-1-04x09] 18:09:12,193 ERROR [org.jgroups.protocols.TCP] (jgroups-30,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
[transactions-repository-1-1f8dx] 18:09:12,310 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
[transactions-repository-1-04x09] 18:09:12,997 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
[transactions-repository-1-1f8dx] 18:09:13,113 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
{noformat}
Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
Steps to reproduce:
* Start local OpenShift Cluster
* invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
* invoke `cd transaction-creator && mvn fabric8:run`
* Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
* Observe logs `kubetail -l environment=infrastructure`
was:
When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
One of the is this:
{noformat}
[transactions-repository-1-4z05w] 18:09:06,122 WARN [org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler] (remote-thread--p2-t5) ISPN000071: Caught exception when handling command CacheTopologyControlCommand{cache=___script_cache, type=REBALANCE_CONFIRM, sender=transactions-repository-1-1f8dx, joinInfo=null, topologyId=9, rebalanceId=0, currentCH=null, pendingCH=null, availabilityMode=null, actualMembers=null, throwable=null, viewId=4}: org.infinispan.commons.CacheConfigurationException: Unable to configure component (type: class org.infinispan.topology.CacheTopologyControlCommand, instance CacheTopologyControlCommand{cache=___script_cache, type=REBALANCE_CONFIRM, sender=transactions-repository-1-1f8dx, joinInfo=null, topologyId=9, rebalanceId=0, currentCH=null, pendingCH=null, availabilityMode=null, actualMembers=null, throwable=null, viewId=4})
[transactions-repository-1-4z05w] at org.infinispan.factories.AbstractComponentRegistry.wireDependencies(AbstractComponentRegistry.java:152)
[transactions-repository-1-4z05w] at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$2.run(GlobalInboundInvocationHandler.java:160)
[transactions-repository-1-4z05w] at org.infinispan.util.concurrent.BlockingTaskAwareExecutorServiceImpl$RunnableWrapper.run(BlockingTaskAwareExecutorServiceImpl.java:203)
[transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[transactions-repository-1-4z05w] at java.lang.Thread.run(Thread.java:745)
[transactions-repository-1-4z05w] Caused by: org.infinispan.IllegalLifecycleStateException: Trying to register a component after stopping: org.infinispan.topology.LocalTopologyManagerFactory
[transactions-repository-1-4z05w] at org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:185)
[transactions-repository-1-4z05w] at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:172)
[transactions-repository-1-4z05w] at org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:168)
[transactions-repository-1-4z05w] at org.infinispan.factories.AbstractComponentRegistry.createComponentFactoryInternal(AbstractComponentRegistry.java:349)
[transactions-repository-1-4z05w] at org.infinispan.factories.AbstractComponentRegistry.getFactory(AbstractComponentRegistry.java:328)
[transactions-repository-1-4z05w] at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:294)
[transactions-repository-1-4z05w] at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:247)
[transactions-repository-1-4z05w] at org.infinispan.factories.AbstractComponentRegistry.wireDependencies(AbstractComponentRegistry.java:148)
[transactions-repository-1-4z05w] ... 5 more
{noformat}
Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
Steps to reproduce:
* Start local OpenShift Cluster
* invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
* invoke `cd transaction-creator && mvn fabric8:run`
* Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
* Observe logs `kubetail -l environment=infrastructure`
> org.jgroups.protocols.TCP emits errors when node leaves the cluster
> -------------------------------------------------------------------
>
> Key: ISPN-7489
> URL: https://issues.jboss.org/browse/ISPN-7489
> Project: Infinispan
> Issue Type: Sub-task
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> [transactions-repository-1-04x09] 18:09:12,193 ERROR [org.jgroups.protocols.TCP] (jgroups-30,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:12,310 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> [transactions-repository-1-04x09] 18:09:12,997 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:13,113 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> {noformat}
> Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * invoke `cd transaction-creator && mvn fabric8:run`
> * Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
> * Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month