[JBoss JIRA] (ISPN-7496) InitialStateTransfer should always retry
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-7496?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-7496:
------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.0.0.CR2
Resolution: Done
> InitialStateTransfer should always retry
> ----------------------------------------
>
> Key: ISPN-7496
> URL: https://issues.jboss.org/browse/ISPN-7496
> Project: Infinispan
> Issue Type: Bug
> Components: Cloud Integrations
> Affects Versions: 9.0.0.Beta2
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Infinispan Server 9.0.0.Beta2
> Reporter: Sebastian Łaskawiec
> Assignee: Sebastian Łaskawiec
> Priority: Blocker
> Fix For: 9.0.0.CR2
>
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> 13:21:40,816 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-6) MSC000001: Failed to start service jboss.datagrid-infinispan.clustered.transactions: org.jboss.msc.service.StartException in service jboss.datagrid-infinispan.clustered.transactions: Failed to start service
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.Exception on object of type StateTransferManagerImpl
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:172)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:867)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:633)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:622)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:547)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:231)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:803)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:641)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:589)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:453)
> at org.infinispan.manager.impl.AbstractDelegatingEmbeddedCacheManager.getCache(AbstractDelegatingEmbeddedCacheManager.java:139)
> at org.infinispan.server.infinispan.SecurityActions$5.run(SecurityActions.java:133)
> at org.infinispan.server.infinispan.SecurityActions$5.run(SecurityActions.java:130)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:66)
> at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:138)
> at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:86)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
> ... 3 more
> Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache transactions on transactions-repository-2-m3rqk
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:228)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> ... 22 more
> {noformat}
> Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/92b9fa309b1e07cbfdc7353db45ed754#file-gi...
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * invoke `cd transaction-creator && mvn fabric8:run`
> * Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0
[JBoss JIRA] (ISPN-7519) NPE and Deadlock during server start
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-7519?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-7519:
------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.0.0.CR2
Resolution: Done
> NPE and Deadlock during server start
> ------------------------------------
>
> Key: ISPN-7519
> URL: https://issues.jboss.org/browse/ISPN-7519
> Project: Infinispan
> Issue Type: Bug
> Components: Server, Test Suite - Server
> Affects Versions: 9.0.0.CR1
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Fix For: 9.0.0.CR2
>
> Attachments: server-trace.txt, surefire-trace.txt
>
>
> When running the full test suite, it deadlocked when running the test {{StateTransferSuppressIT.testRebalanceWithFirstNodeStop}}.
> The server startup printed
> {noformat}
> 12:13:07,787 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-4) ISPN000128: Infinispan version: Infinispan 'Ruppaner' 9.0.0-SNAPSHOT
> 12:13:08,159 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0001: Started memcachedCache cache from clustered container
> 12:13:08,159 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-2) DGISPN0001: Started default cache from clustered container
> 12:13:08,181 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10000: MemcachedServer starting
> 12:13:08,181 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10000: HotRodServer starting
> 12:13:08,182 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10001: MemcachedServer listening on 127.0.0.1:11211
> 12:13:08,182 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10001: HotRodServer listening on 127.0.0.1:11222
> 12:13:08,188 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10000: REST starting
> 12:13:08,259 WARNING [io.netty.channel.epoll.EpollEventLoop] (MemcachedServerMaster-1-1) Unexpected exception in the selector loop.: java.lang.NullPointerException
> at io.netty.util.internal.shaded.org.jctools.queues.MpscChunkedArrayQueue.poll(MpscChunkedArrayQueue.java:264)
> at io.netty.util.concurrent.SingleThreadEventExecutor.pollTaskFrom(SingleThreadEventExecutor.java:223)
> at io.netty.util.concurrent.SingleThreadEventExecutor.pollTask(SingleThreadEventExecutor.java:218)
> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:408)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:306)
> at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> at java.lang.Thread.run(Thread.java:745)
> 12:13:08,338 INFO [org.infinispan.rest.NettyRestServer] (MSC service thread 1-1) ISPN012003: REST server starting, listening on 127.0.0.1:8080
> 12:13:08,338 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10002: REST mapped to /rest
> {noformat}
> and then hanged. Attached is the stack trace of the server, and the forked test process.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0
[JBoss JIRA] (ISPN-7484) Upgrade to Apache Lucene 5.5.4
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-7484?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-7484:
------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Upgrade to Apache Lucene 5.5.4
> ------------------------------
>
> Key: ISPN-7484
> URL: https://issues.jboss.org/browse/ISPN-7484
> Project: Infinispan
> Issue Type: Component Upgrade
> Components: Embedded Querying, Lucene Directory
> Reporter: Sanne Grinovero
> Assignee: Sanne Grinovero
> Fix For: 9.0.0.CR2
>
>
> Various improvements in Lucene, not least the Java 9 compatibility was backported:
> - https://lucene.apache.org/core/5_5_4/changes/Changes.html
> Related to https://hibernate.atlassian.net/browse/HSEARCH-2591
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0
[JBoss JIRA] (ISPN-7506) Tests fail after upgrading Lucene version
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-7506?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-7506:
------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Tests fail after upgrading Lucene version
> -----------------------------------------
>
> Key: ISPN-7506
> URL: https://issues.jboss.org/browse/ISPN-7506
> Project: Infinispan
> Issue Type: Bug
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Fix For: 9.0.0.CR2
>
>
> Upgrading Lucene causes clash with the elasticsearch being used in tests:
> {noformat}
> org.infinispan.test.integration.as (1)
> query.ElasticsearchIndexManagerIT
> java.lang.AssertionError: Version must be upgraded to [5.5.4] is still set to [5.5.2]
> at org.elasticsearch.Version.(Version.java:316)
> at org.infinispan.query.test.elasticsearch.ElasticSearchCluster$TestNode.(ElasticSearchCluster.java:59)
> at org.infinispan.query.test.elasticsearch.ElasticSearchCluster$ElasticSearchClusterBuilder.buildNode(ElasticSearchCluster.java:123)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0
[JBoss JIRA] (ISPN-7520) Increase staggered get delay
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-7520?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-7520:
------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Increase staggered get delay
> ----------------------------
>
> Key: ISPN-7520
> URL: https://issues.jboss.org/browse/ISPN-7520
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 9.0.0.CR1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: performance
> Fix For: 9.0.0.CR2
>
>
> A stagger delay of 5 ms appears to be insufficient under heavy load, as more than 50% of the remote gets end up requesting the value from both owners.
> We have made some experiments with delays of 100ms and 5000ms, and while they're both vastly better than 5ms, the difference between seems to be very small. So we have decided to remove the system property and use {{remote-timeout / 10}} (1500ms with the default configuration) as the stagger timeout instead.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0
[JBoss JIRA] (ISPN-7522) LocalMode[No]PassivationTest fails randomly
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-7522?page=com.atlassian.jira.plugin.... ]
William Burns updated ISPN-7522:
--------------------------------
Status: Pull Request Sent (was: Coding In Progress)
Git Pull Request: https://github.com/infinispan/infinispan/pull/4899
> LocalMode[No]PassivationTest fails randomly
> -------------------------------------------
>
> Key: ISPN-7522
> URL: https://issues.jboss.org/browse/ISPN-7522
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Eviction, Loaders and Stores
> Reporter: William Burns
> Assignee: William Burns
>
> LocalModePassivationTest and LocalModeNoPassivationTest can fail randomly saying entries aren't present. All entries should be present since there is a configured store, this is a pretty big loss of data.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0
[JBoss JIRA] (ISPN-7487) Fix NPE in ClusterCacheStatus.java:147
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7487?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec updated ISPN-7487:
--------------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Fix NPE in ClusterCacheStatus.java:147
> --------------------------------------
>
> Key: ISPN-7487
> URL: https://issues.jboss.org/browse/ISPN-7487
> Project: Infinispan
> Issue Type: Bug
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
> Assignee: Sebastian Łaskawiec
> Priority: Blocker
> Fix For: 9.0.0.CR2, 9.0.0.Final
>
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> kubetail -l environment=infrastructure
> Will tail 3 logs...
> transactions-repository-1-04x09
> transactions-repository-1-1f8dx
> transactions-repository-1-4z05w
> [transactions-repository-1-1f8dx] JAVA_OPTS already set in environment; overriding default settings with values: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-1f8dx] =========================================================================
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] JBoss Bootstrap Environment
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] JBOSS_HOME: /opt/jboss/infinispan-server
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] JAVA: /usr/lib/jvm/java/bin/java
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] JAVA_OPTS: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] =========================================================================
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] 17:54:15,634 INFO [org.jboss.modules] (main) JBoss Modules version 1.5.2.Final
> [transactions-repository-1-1f8dx] 17:54:15,853 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
> [transactions-repository-1-1f8dx] 17:54:15,931 INFO [org.jboss.as] (MSC service thread 1-8) WFLYSRV0049: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) starting
> [transactions-repository-1-1f8dx] 17:54:17,232 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 7) WFLYCTL0028: Attribute 'default-stack' in the resource at address '/subsystem=datagrid-jgroups' is deprecated, and may be removed in future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
> [transactions-repository-1-1f8dx] [GC (CMS Initial Mark) 234913K(1013632K), 0.0473268 secs]
> [transactions-repository-1-1f8dx] 17:54:17,432 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
> [transactions-repository-1-1f8dx] 17:54:17,452 INFO [org.xnio] (MSC service thread 1-2) XNIO version 3.4.0.Final
> [transactions-repository-1-1f8dx] 17:54:17,464 INFO [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.4.0.Final
> [transactions-repository-1-1f8dx] 17:54:17,516 INFO [org.jboss.remoting] (MSC service thread 1-3) JBoss Remoting version 4.0.21.Final
> [transactions-repository-1-1f8dx] 17:54:17,553 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 19) Activating Infinispan subsystem.
> [transactions-repository-1-1f8dx] [GC (GCLocker Initiated GC) 279722K->29726K(1013632K), 0.0377206 secs]
> [transactions-repository-1-1f8dx] 17:54:17,611 INFO [org.infinispan.server.jgroups] (ServerService Thread Pool -- 21) DGJGRP0001: Activating JGroups subsystem.
> [transactions-repository-1-1f8dx] 17:54:17,630 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 29) WFLYTX0013: Node identifier property is set to the default value. Please make sure it is unique.
> [transactions-repository-1-1f8dx] 17:54:17,646 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 26) WFLYNAM0001: Activating Naming Subsystem
> [transactions-repository-1-1f8dx] 17:54:17,646 INFO [org.jboss.as.security] (ServerService Thread Pool -- 28) WFLYSEC0002: Activating Security Subsystem
> [transactions-repository-1-1f8dx] 17:54:17,652 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 22) WFLYIO001: Worker 'default' has auto-configured to 16 core threads with 128 task threads based on your 8 available processors
> [transactions-repository-1-1f8dx] 17:54:17,653 INFO [org.jboss.as.security] (MSC service thread 1-8) WFLYSEC0001: Current PicketBox version=4.9.6.Final
> [transactions-repository-1-1f8dx] 17:54:17,673 INFO [org.jboss.as.connector] (MSC service thread 1-4) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.3.4.Final)
> [transactions-repository-1-1f8dx] 17:54:17,701 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 18) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
> [transactions-repository-1-1f8dx] 17:54:17,703 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-6) WFLYJCA0018: Started Driver service with driver-name = h2
> [transactions-repository-1-1f8dx] 17:54:17,744 INFO [org.jboss.as.naming] (MSC service thread 1-7) WFLYNAM0003: Starting Naming Service
> [transactions-repository-1-1f8dx] 17:54:18,268 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-5) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]
> [transactions-repository-1-1f8dx] 17:54:18,425 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] 17:54:18,429 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-1f8dx] 17:54:18,779 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-5) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/jboss/infinispan-server/standalone/deployments
> [transactions-repository-1-04x09] JAVA_OPTS already set in environment; overriding default settings with values: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-04x09] =========================================================================
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] JBoss Bootstrap Environment
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] JBOSS_HOME: /opt/jboss/infinispan-server
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] JAVA: /usr/lib/jvm/java/bin/java
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] JAVA_OPTS: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] =========================================================================
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] 17:54:15,628 INFO [org.jboss.modules] (main) JBoss Modules version 1.5.2.Final
> [transactions-repository-1-04x09] 17:54:15,849 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
> [transactions-repository-1-04x09] 17:54:15,960 INFO [org.jboss.as] (MSC service thread 1-5) WFLYSRV0049: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) starting
> [transactions-repository-1-04x09] [GC (CMS Initial Mark) 218119K(1013632K), 0.0624692 secs]
> [transactions-repository-1-04x09] 17:54:17,475 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 13) WFLYCTL0028: Attribute 'default-stack' in the resource at address '/subsystem=datagrid-jgroups' is deprecated, and may be removed in future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
> [transactions-repository-1-04x09] 17:54:17,647 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
> [transactions-repository-1-04x09] 17:54:17,664 INFO [org.xnio] (MSC service thread 1-3) XNIO version 3.4.0.Final
> [transactions-repository-1-04x09] 17:54:17,683 INFO [org.xnio.nio] (MSC service thread 1-3) XNIO NIO Implementation Version 3.4.0.Final
> [transactions-repository-1-04x09] 17:54:17,763 INFO [org.jboss.remoting] (MSC service thread 1-4) JBoss Remoting version 4.0.21.Final
> [transactions-repository-1-04x09] [GC (Allocation Failure) 279616K->29652K(1013632K), 0.0690895 secs]
> [transactions-repository-1-04x09] 17:54:17,872 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 19) Activating Infinispan subsystem.
> [transactions-repository-1-04x09] 17:54:17,875 INFO [org.infinispan.server.jgroups] (ServerService Thread Pool -- 21) DGJGRP0001: Activating JGroups subsystem.
> [transactions-repository-1-04x09] [GC (CMS Final Remark) 68259K(1013632K), 0.0485831 secs]
> [transactions-repository-1-04x09] 17:54:17,933 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 22) WFLYIO001: Worker 'default' has auto-configured to 16 core threads with 128 task threads based on your 8 available processors
> [transactions-repository-1-04x09] 17:54:17,960 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 29) WFLYTX0013: Node identifier property is set to the default value. Please make sure it is unique.
> [transactions-repository-1-04x09] 17:54:17,972 INFO [org.jboss.as.security] (ServerService Thread Pool -- 28) WFLYSEC0002: Activating Security Subsystem
> [transactions-repository-1-04x09] 17:54:17,972 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 26) WFLYNAM0001: Activating Naming Subsystem
> [transactions-repository-1-04x09] 17:54:17,984 INFO [org.jboss.as.security] (MSC service thread 1-4) WFLYSEC0001: Current PicketBox version=4.9.6.Final
> [transactions-repository-1-04x09] 17:54:18,004 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 18) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
> [transactions-repository-1-04x09] 17:54:18,027 INFO [org.jboss.as.naming] (MSC service thread 1-5) WFLYNAM0003: Starting Naming Service
> [transactions-repository-1-04x09] 17:54:18,053 INFO [org.jboss.as.connector] (MSC service thread 1-8) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.3.4.Final)
> [transactions-repository-1-04x09] 17:54:18,065 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-6) WFLYJCA0018: Started Driver service with driver-name = h2
> [transactions-repository-1-04x09] 17:54:18,543 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-04x09] 17:54:18,561 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-04x09] 17:54:18,584 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-6) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]
> [transactions-repository-1-04x09] 17:54:18,795 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-2) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/jboss/infinispan-server/standalone/deployments
> [transactions-repository-1-4z05w] JAVA_OPTS already set in environment; overriding default settings with values: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-4z05w] =========================================================================
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] JBoss Bootstrap Environment
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] JBOSS_HOME: /opt/jboss/infinispan-server
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] JAVA: /usr/lib/jvm/java/bin/java
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] JAVA_OPTS: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] =========================================================================
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] 17:54:15,714 INFO [org.jboss.modules] (main) JBoss Modules version 1.5.2.Final
> [transactions-repository-1-4z05w] 17:54:15,950 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
> [transactions-repository-1-4z05w] 17:54:16,016 INFO [org.jboss.as] (MSC service thread 1-6) WFLYSRV0049: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) starting
> [transactions-repository-1-4z05w] 17:54:17,364 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 5) WFLYCTL0028: Attribute 'default-stack' in the resource at address '/subsystem=datagrid-jgroups' is deprecated, and may be removed in future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
> [transactions-repository-1-4z05w] [GC (CMS Initial Mark) 234897K(1013632K), 0.0587177 secs]
> [transactions-repository-1-4z05w] 17:54:17,599 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
> [transactions-repository-1-4z05w] 17:54:17,620 INFO [org.xnio] (MSC service thread 1-5) XNIO version 3.4.0.Final
> [transactions-repository-1-4z05w] 17:54:17,626 INFO [org.xnio.nio] (MSC service thread 1-5) XNIO NIO Implementation Version 3.4.0.Final
> [transactions-repository-1-4z05w] 17:54:17,674 INFO [org.jboss.remoting] (MSC service thread 1-5) JBoss Remoting version 4.0.21.Final
> [transactions-repository-1-4z05w] [GC (Allocation Failure) 279616K->29736K(1013632K), 0.0733498 secs]
> [transactions-repository-1-4z05w] 17:54:17,799 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 19) Activating Infinispan subsystem.
> [transactions-repository-1-4z05w] 17:54:17,802 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 29) WFLYTX0013: Node identifier property is set to the default value. Please make sure it is unique.
> [transactions-repository-1-4z05w] 17:54:17,829 INFO [org.jboss.as.security] (ServerService Thread Pool -- 28) WFLYSEC0002: Activating Security Subsystem
> [transactions-repository-1-4z05w] 17:54:17,831 INFO [org.infinispan.server.jgroups] (ServerService Thread Pool -- 21) DGJGRP0001: Activating JGroups subsystem.
> [transactions-repository-1-4z05w] 17:54:17,835 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 22) WFLYIO001: Worker 'default' has auto-configured to 16 core threads with 128 task threads based on your 8 available processors
> [transactions-repository-1-4z05w] 17:54:17,870 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 26) WFLYNAM0001: Activating Naming Subsystem
> [transactions-repository-1-4z05w] 17:54:17,881 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 18) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
> [transactions-repository-1-4z05w] 17:54:17,887 INFO [org.jboss.as.connector] (MSC service thread 1-2) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.3.4.Final)
> [transactions-repository-1-4z05w] 17:54:17,895 INFO [org.jboss.as.security] (MSC service thread 1-4) WFLYSEC0001: Current PicketBox version=4.9.6.Final
> [transactions-repository-1-4z05w] 17:54:17,900 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-8) WFLYJCA0018: Started Driver service with driver-name = h2
> [transactions-repository-1-4z05w] 17:54:18,037 INFO [org.jboss.as.naming] (MSC service thread 1-6) WFLYNAM0003: Starting Naming Service
> [transactions-repository-1-4z05w] 17:54:18,842 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-5) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]
> [transactions-repository-1-4z05w] 17:54:18,920 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-4z05w] 17:54:18,934 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-4z05w] 17:54:19,094 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-6) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/jboss/infinispan-server/standalone/deployments
> [transactions-repository-1-4z05w] 17:54:20,553 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-6) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-4z05w] 17:54:20,555 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-6) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-4z05w] [GC (CMS Final Remark) 171687K(1013632K), 0.0585468 secs]
> [transactions-repository-1-4z05w] 17:54:20,716 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000078: Starting JGroups channel clustered
> [transactions-repository-1-4z05w] 17:54:20,732 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000094: Received new cluster view for channel clustered: [transactions-repository-1-4z05w|2] (3) [transactions-repository-1-4z05w, transactions-repository-1-04x09, transactions-repository-1-1f8dx]
> [transactions-repository-1-4z05w] 17:54:20,742 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000079: Channel clustered local address is transactions-repository-1-4z05w, physical addresses are [172.17.0.3:7600]
> [transactions-repository-1-4z05w] 17:54:20,754 INFO [org.infinispan.globalstate.impl.GlobalStateManagerImpl] (MSC service thread 1-6) ISPN000390: Persisted state, version=9.0.0-SNAPSHOT timestamp=2017-02-18T17:54:20.748Z
> [transactions-repository-1-4z05w] 17:54:20,755 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-6) ISPN000128: Infinispan version: Infinispan 'Ruppaner' 9.0.0-SNAPSHOT
> [transactions-repository-1-4z05w] 17:54:20,933 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-6) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-4z05w] 17:54:20,933 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-6) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-4z05w] 17:54:20,936 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-5) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-4z05w] 17:54:20,939 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-4) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-4z05w] 17:54:20,943 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-4) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-4z05w] 17:54:20,954 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-4) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-4z05w] 17:54:20,950 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-5) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-4z05w] 17:54:20,956 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-5) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] [GC (CMS Final Remark) 172593K(1013632K), 0.0447491 secs]
> [transactions-repository-1-04x09] 17:54:20,995 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-3) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:20,996 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-3) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:21,109 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000078: Starting JGroups channel clustered
> [transactions-repository-1-04x09] 17:54:21,129 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000094: Received new cluster view for channel clustered: [transactions-repository-1-4z05w|2] (3) [transactions-repository-1-4z05w, transactions-repository-1-04x09, transactions-repository-1-1f8dx]
> [transactions-repository-1-04x09] 17:54:21,138 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000079: Channel clustered local address is transactions-repository-1-04x09, physical addresses are [172.17.0.4:7600]
> [transactions-repository-1-1f8dx] 17:54:21,158 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-8) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-1f8dx] 17:54:21,158 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-8) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:21,160 INFO [org.infinispan.globalstate.impl.GlobalStateManagerImpl] (MSC service thread 1-3) ISPN000390: Persisted state, version=9.0.0-SNAPSHOT timestamp=2017-02-18T17:54:21.156Z
> [transactions-repository-1-04x09] 17:54:21,160 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-3) ISPN000128: Infinispan version: Infinispan 'Ruppaner' 9.0.0-SNAPSHOT
> [transactions-repository-1-1f8dx] 17:54:21,254 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000078: Starting JGroups channel clustered
> [transactions-repository-1-1f8dx] 17:54:21,266 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000094: Received new cluster view for channel clustered: [transactions-repository-1-4z05w|2] (3) [transactions-repository-1-4z05w, transactions-repository-1-04x09, transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] 17:54:21,269 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000079: Channel clustered local address is transactions-repository-1-1f8dx, physical addresses are [172.17.0.6:7600]
> [transactions-repository-1-1f8dx] 17:54:21,282 INFO [org.infinispan.globalstate.impl.GlobalStateManagerImpl] (MSC service thread 1-8) ISPN000390: Persisted state, version=9.0.0-SNAPSHOT timestamp=2017-02-18T17:54:21.279Z
> [transactions-repository-1-1f8dx] 17:54:21,283 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-8) ISPN000128: Infinispan version: Infinispan 'Ruppaner' 9.0.0-SNAPSHOT
> [transactions-repository-1-04x09] 17:54:21,297 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-3) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:21,297 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-3) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:21,300 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-6) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-04x09] 17:54:21,303 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-2) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-04x09] 17:54:21,305 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-6) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-04x09] 17:54:21,310 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-6) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-04x09] 17:54:21,310 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-2) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-04x09] 17:54:21,311 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-2) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] 17:54:21,443 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-8) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-1f8dx] 17:54:21,443 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-8) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-1f8dx] 17:54:21,449 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-8) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-1f8dx] 17:54:21,450 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-8) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-1f8dx] 17:54:21,452 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-4) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] 17:54:21,462 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-4) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] 17:54:21,462 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-8) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-1f8dx] 17:54:21,468 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-4) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-04x09] [GC (CMS Initial Mark) 208341K(1013632K), 0.0101475 secs]
> [transactions-repository-1-4z05w] 17:54:23,974 ERROR [org.infinispan.topology.ClusterCacheStatus] (transport-thread--p4-t9) ISPN000228: Failed to recover cache transactions state after the current node became the coordinator: java.lang.NullPointerException
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterCacheStatus.updateAvailabilityMode(ClusterCacheStatus.java:147)
> [transactions-repository-1-4z05w] at org.infinispan.partitionhandling.impl.PreferConsistencyStrategy.updateMembersAndRebalance(PreferConsistencyStrategy.java:259)
> [transactions-repository-1-4z05w] at org.infinispan.partitionhandling.impl.PreferConsistencyStrategy.onPartitionMerge(PreferConsistencyStrategy.java:210)
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterCacheStatus.doMergePartitions(ClusterCacheStatus.java:554)
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$recoverClusterStatus$3(ClusterTopologyManagerImpl.java:499)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174)
> [transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [transactions-repository-1-4z05w] at java.lang.Thread.run(Thread.java:745)
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] 17:54:23,974 ERROR [org.infinispan.topology.ClusterCacheStatus] (transport-thread--p4-t7) ISPN000228: Failed to recover cache sessions state after the current node became the coordinator: java.lang.NullPointerException
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterCacheStatus.updateAvailabilityMode(ClusterCacheStatus.java:147)
> [transactions-repository-1-4z05w] at org.infinispan.partitionhandling.impl.PreferConsistencyStrategy.updateMembersAndRebalance(PreferConsistencyStrategy.java:259)
> [transactions-repository-1-4z05w] at org.infinispan.partitionhandling.impl.PreferConsistencyStrategy.onPartitionMerge(PreferConsistencyStrategy.java:210)
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterCacheStatus.doMergePartitions(ClusterCacheStatus.java:554)
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$recoverClusterStatus$3(ClusterTopologyManagerImpl.java:499)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174)
> [transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [transactions-repository-1-4z05w] at java.lang.Thread.run(Thread.java:745)
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] 17:54:24,005 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-8) DGISPN0001: Started transactions cache from clustered container
> [transactions-repository-1-4z05w] 17:54:24,005 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-1) DGISPN0001: Started default cache from clustered container
> [transactions-repository-1-04x09] 17:54:24,005 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) DGISPN0001: Started default cache from clustered container
> [transactions-repository-1-04x09] 17:54:24,006 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0001: Started memcachedCache cache from clustered container
> [transactions-repository-1-04x09] 17:54:24,006 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-1) DGISPN0001: Started transactions cache from clustered container
> [transactions-repository-1-4z05w] 17:54:24,008 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) DGISPN0001: Started memcachedCache cache from clustered container
> [transactions-repository-1-4z05w] 17:54:24,010 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-6) DGISPN0001: Started sessions cache from clustered container
> [transactions-repository-1-04x09] 17:54:24,011 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-3) DGISPN0001: Started sessions cache from clustered container
> [transactions-repository-1-1f8dx] 17:54:24,012 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-5) DGISPN0001: Started sessions cache from clustered container
> [transactions-repository-1-1f8dx] 17:54:24,012 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-6) DGISPN0001: Started memcachedCache cache from clustered container
> [transactions-repository-1-1f8dx] 17:54:24,012 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-2) DGISPN0001: Started default cache from clustered container
> [transactions-repository-1-1f8dx] 17:54:24,013 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) DGISPN0001: Started transactions cache from clustered container
> [transactions-repository-1-4z05w] 17:54:24,036 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) DGENDPT10000: MemcachedServer starting
> [transactions-repository-1-4z05w] 17:54:24,039 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10000: HotRodServer starting
> [transactions-repository-1-4z05w] 17:54:24,039 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10001: HotRodServer listening on 172.17.0.3:11222
> [transactions-repository-1-4z05w] 17:54:24,040 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10000: WebSocketServer starting
> [transactions-repository-1-4z05w] 17:54:24,040 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10001: WebSocketServer listening on 172.17.0.3:8181
> [transactions-repository-1-1f8dx] 17:54:24,046 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) DGENDPT10000: WebSocketServer starting
> [transactions-repository-1-1f8dx] 17:54:24,045 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10000: MemcachedServer starting
> [transactions-repository-1-1f8dx] 17:54:24,046 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10000: HotRodServer starting
> [transactions-repository-1-1f8dx] 17:54:24,047 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10001: HotRodServer listening on 172.17.0.6:11222
> [transactions-repository-1-1f8dx] 17:54:24,048 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10001: MemcachedServer listening on 172.17.0.6:11211
> [transactions-repository-1-04x09] 17:54:24,051 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10000: MemcachedServer starting
> [transactions-repository-1-04x09] 17:54:24,051 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10000: WebSocketServer starting
> [transactions-repository-1-04x09] 17:54:24,052 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10000: HotRodServer starting
> [transactions-repository-1-04x09] 17:54:24,055 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10001: HotRodServer listening on 172.17.0.4:11222
> [transactions-repository-1-04x09] 17:54:24,055 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10001: MemcachedServer listening on 172.17.0.4:11211
> [transactions-repository-1-04x09] 17:54:24,056 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10001: WebSocketServer listening on 172.17.0.4:8181
> [transactions-repository-1-4z05w] 17:54:24,063 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) DGENDPT10001: MemcachedServer listening on 172.17.0.3:11211
> [transactions-repository-1-1f8dx] 17:54:24,059 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) DGENDPT10001: WebSocketServer listening on 172.17.0.6:8181
> [transactions-repository-1-4z05w] 17:54:24,070 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10000: REST starting
> [transactions-repository-1-1f8dx] 17:54:24,079 INFO [org.infinispan.server.endpoint] (MSC service thread 1-6) DGENDPT10000: REST starting
> [transactions-repository-1-04x09] 17:54:24,071 INFO [org.infinispan.server.endpoint] (MSC service thread 1-8) DGENDPT10000: REST starting
> [transactions-repository-1-4z05w] 17:54:24,193 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t20) ISPN000310: Starting cluster-wide rebalance for cache ___protobuf_metadata, topology CacheTopology{id=2, rebalanceId=2, currentCH=ReplicatedConsistentHash{ns = 256, owners = (1)[transactions-repository-1-4z05w: 256]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 126, transactions-repository-1-1f8dx: 130]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-1f8dx], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, ced31139-f1a1-4ee8-96af-f37c5386ecc0]}
> [transactions-repository-1-4z05w] 17:54:24,197 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t20) [Context=___protobuf_metadata][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,246 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t24) [Context=___protobuf_metadata][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,255 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t22) ISPN000310: Starting cluster-wide rebalance for cache ___script_cache, topology CacheTopology{id=2, rebalanceId=2, currentCH=ReplicatedConsistentHash{ns = 256, owners = (1)[transactions-repository-1-4z05w: 256]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 126, transactions-repository-1-1f8dx: 130]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-1f8dx], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, ced31139-f1a1-4ee8-96af-f37c5386ecc0]}
> [transactions-repository-1-4z05w] 17:54:24,256 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t22) [Context=___script_cache][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,272 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t2) [Context=___script_cache][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,317 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t2) ISPN000310: Starting cluster-wide rebalance for cache ___hotRodTopologyCache, topology CacheTopology{id=2, rebalanceId=2, currentCH=ReplicatedConsistentHash{ns = 256, owners = (1)[transactions-repository-1-4z05w: 256]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 134, transactions-repository-1-04x09: 122]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-04x09], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, 5d246b27-021c-4425-b3c4-14017f378635]}
> [transactions-repository-1-4z05w] 17:54:24,318 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t2) [Context=___hotRodTopologyCache][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,321 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t11) [Context=___hotRodTopologyCache][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,364 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t8) [Context=___protobuf_metadata][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,366 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) [Context=___script_cache][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,382 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) [Context=___hotRodTopologyCache][Context=transactions-repository-1-04x09]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,385 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t8) ISPN000336: Finished cluster-wide rebalance for cache ___protobuf_metadata, topology id = 2
> [transactions-repository-1-4z05w] 17:54:24,387 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t8) ISPN000310: Starting cluster-wide rebalance for cache ___protobuf_metadata, topology CacheTopology{id=4, rebalanceId=3, currentCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 126, transactions-repository-1-1f8dx: 130]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (3)[transactions-repository-1-4z05w: 90, transactions-repository-1-1f8dx: 86, transactions-repository-1-04x09: 80]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-1f8dx, transactions-repository-1-04x09], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, ced31139-f1a1-4ee8-96af-f37c5386ecc0, 5d246b27-021c-4425-b3c4-14017f378635]}
> [transactions-repository-1-4z05w] 17:54:24,388 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t8) [Context=___protobuf_metadata][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,394 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) ISPN000336: Finished cluster-wide rebalance for cache ___hotRodTopologyCache, topology id = 2
> [transactions-repository-1-4z05w] 17:54:24,395 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) ISPN000336: Finished cluster-wide rebalance for cache ___script_cache, topology id = 2
> [transactions-repository-1-4z05w] 17:54:24,397 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t19) [Context=___protobuf_metadata][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,397 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t7) [Context=___protobuf_metadata][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,398 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) ISPN000310: Starting cluster-wide rebalance for cache ___script_cache, topology CacheTopology{id=4, rebalanceId=3, currentCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 126, transactions-repository-1-1f8dx: 130]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (3)[transactions-repository-1-4z05w: 90, transactions-repository-1-1f8dx: 86, transactions-repository-1-04x09: 80]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-1f8dx, transactions-repository-1-04x09], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, ced31139-f1a1-4ee8-96af-f37c5386ecc0, 5d246b27-021c-4425-b3c4-14017f378635]}
> [transactions-repository-1-4z05w] 17:54:24,398 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) [Context=___script_cache][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,401 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) ISPN000310: Starting cluster-wide rebalance for cache ___hotRodTopologyCache, topology CacheTopology{id=4, rebalanceId=3, currentCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 134, transactions-repository-1-04x09: 122]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (3)[transactions-repository-1-4z05w: 90, transactions-repository-1-04x09: 80, transactions-repository-1-1f8dx: 86]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-04x09, transactions-repository-1-1f8dx], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, 5d246b27-021c-4425-b3c4-14017f378635, ced31139-f1a1-4ee8-96af-f37c5386ecc0]}
> [transactions-repository-1-4z05w] 17:54:24,405 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) [Context=___hotRodTopologyCache][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,407 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t20) [Context=___hotRodTopologyCache][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,409 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t3) [Context=___script_cache][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,420 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t11) [Context=___script_cache][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,425 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t11) [Context=___hotRodTopologyCache][Context=transactions-repository-1-04x09]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,476 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t17) [Context=___hotRodTopologyCache][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,480 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t17) ISPN000336: Finished cluster-wide rebalance for cache ___hotRodTopologyCache, topology id = 4
> [transactions-repository-1-4z05w] 17:54:24,536 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t21) [Context=___protobuf_metadata][Context=transactions-repository-1-04x09]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,537 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t21) ISPN000336: Finished cluster-wide rebalance for cache ___protobuf_metadata, topology id = 4
> [transactions-repository-1-4z05w] 17:54:24,544 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t24) [Context=___script_cache][Context=transactions-repository-1-04x09]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,546 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t24) ISPN000336: Finished cluster-wide rebalance for cache ___script_cache, topology id = 4
> [transactions-repository-1-4z05w] 17:54:24,567 INFO [org.infinispan.rest.NettyRestServer] (MSC service thread 1-1) ISPN012003: REST server starting, listening on 172.17.0.3:8080
> [transactions-repository-1-4z05w] 17:54:24,568 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10002: REST mapped to /rest
> [transactions-repository-1-1f8dx] 17:54:24,576 INFO [org.infinispan.rest.NettyRestServer] (MSC service thread 1-6) ISPN012003: REST server starting, listening on 172.17.0.6:8080
> [transactions-repository-1-1f8dx] 17:54:24,577 INFO [org.infinispan.server.endpoint] (MSC service thread 1-6) DGENDPT10002: REST mapped to /rest
> [transactions-repository-1-04x09] 17:54:24,639 INFO [org.infinispan.rest.NettyRestServer] (MSC service thread 1-8) ISPN012003: REST server starting, listening on 172.17.0.4:8080
> [transactions-repository-1-04x09] 17:54:24,639 INFO [org.infinispan.server.endpoint] (MSC service thread 1-8) DGENDPT10002: REST mapped to /rest
> [transactions-repository-1-1f8dx] [GC (Allocation Failure) 309342K->63160K(1013632K), 0.1823599 secs]
> [transactions-repository-1-4z05w] [GC (Allocation Failure) 309352K->65174K(1013632K), 0.1616480 secs]
> [transactions-repository-1-04x09] [GC (Allocation Failure) 309268K->61649K(1013632K), 0.1507677 secs]
> [transactions-repository-1-1f8dx] 17:54:25,021 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management
> [transactions-repository-1-1f8dx] 17:54:25,022 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://0.0.0.0:9990
> [transactions-repository-1-1f8dx] 17:54:25,022 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) started in 9718ms - Started 197 of 282 services (164 services are lazy, passive or on-demand)
> [transactions-repository-1-4z05w] 17:54:25,021 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management
> [transactions-repository-1-4z05w] 17:54:25,024 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://0.0.0.0:9990
> [transactions-repository-1-4z05w] 17:54:25,027 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) started in 9643ms - Started 197 of 282 services (164 services are lazy, passive or on-demand)
> [transactions-repository-1-04x09] 17:54:25,063 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management
> [transactions-repository-1-04x09] 17:54:25,064 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://0.0.0.0:9990
> [transactions-repository-1-04x09] 17:54:25,064 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) started in 9746ms - Started 197 of 282 services (164 services are lazy, passive or on-demand)
> [transactions-repository-1-04x09] [GC (CMS Final Remark) 89552K(1013632K), 0.0310821 secs]
> {noformat}
> Note that I applied multiplexed logs from 3 nodes. I hope this will be helpful for diagnosing.
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * Kill some nodes
> * Start them again
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0
[JBoss JIRA] (ISPN-7487) Fix NPE in ClusterCacheStatus.java:147
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7487?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec updated ISPN-7487:
--------------------------------------
Fix Version/s: 9.0.0.CR2
9.0.0.Final
> Fix NPE in ClusterCacheStatus.java:147
> --------------------------------------
>
> Key: ISPN-7487
> URL: https://issues.jboss.org/browse/ISPN-7487
> Project: Infinispan
> Issue Type: Bug
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
> Assignee: Sebastian Łaskawiec
> Priority: Blocker
> Fix For: 9.0.0.CR2, 9.0.0.Final
>
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> kubetail -l environment=infrastructure
> Will tail 3 logs...
> transactions-repository-1-04x09
> transactions-repository-1-1f8dx
> transactions-repository-1-4z05w
> [transactions-repository-1-1f8dx] JAVA_OPTS already set in environment; overriding default settings with values: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-1f8dx] =========================================================================
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] JBoss Bootstrap Environment
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] JBOSS_HOME: /opt/jboss/infinispan-server
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] JAVA: /usr/lib/jvm/java/bin/java
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] JAVA_OPTS: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] =========================================================================
> [transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] 17:54:15,634 INFO [org.jboss.modules] (main) JBoss Modules version 1.5.2.Final
> [transactions-repository-1-1f8dx] 17:54:15,853 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
> [transactions-repository-1-1f8dx] 17:54:15,931 INFO [org.jboss.as] (MSC service thread 1-8) WFLYSRV0049: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) starting
> [transactions-repository-1-1f8dx] 17:54:17,232 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 7) WFLYCTL0028: Attribute 'default-stack' in the resource at address '/subsystem=datagrid-jgroups' is deprecated, and may be removed in future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
> [transactions-repository-1-1f8dx] [GC (CMS Initial Mark) 234913K(1013632K), 0.0473268 secs]
> [transactions-repository-1-1f8dx] 17:54:17,432 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
> [transactions-repository-1-1f8dx] 17:54:17,452 INFO [org.xnio] (MSC service thread 1-2) XNIO version 3.4.0.Final
> [transactions-repository-1-1f8dx] 17:54:17,464 INFO [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.4.0.Final
> [transactions-repository-1-1f8dx] 17:54:17,516 INFO [org.jboss.remoting] (MSC service thread 1-3) JBoss Remoting version 4.0.21.Final
> [transactions-repository-1-1f8dx] 17:54:17,553 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 19) Activating Infinispan subsystem.
> [transactions-repository-1-1f8dx] [GC (GCLocker Initiated GC) 279722K->29726K(1013632K), 0.0377206 secs]
> [transactions-repository-1-1f8dx] 17:54:17,611 INFO [org.infinispan.server.jgroups] (ServerService Thread Pool -- 21) DGJGRP0001: Activating JGroups subsystem.
> [transactions-repository-1-1f8dx] 17:54:17,630 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 29) WFLYTX0013: Node identifier property is set to the default value. Please make sure it is unique.
> [transactions-repository-1-1f8dx] 17:54:17,646 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 26) WFLYNAM0001: Activating Naming Subsystem
> [transactions-repository-1-1f8dx] 17:54:17,646 INFO [org.jboss.as.security] (ServerService Thread Pool -- 28) WFLYSEC0002: Activating Security Subsystem
> [transactions-repository-1-1f8dx] 17:54:17,652 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 22) WFLYIO001: Worker 'default' has auto-configured to 16 core threads with 128 task threads based on your 8 available processors
> [transactions-repository-1-1f8dx] 17:54:17,653 INFO [org.jboss.as.security] (MSC service thread 1-8) WFLYSEC0001: Current PicketBox version=4.9.6.Final
> [transactions-repository-1-1f8dx] 17:54:17,673 INFO [org.jboss.as.connector] (MSC service thread 1-4) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.3.4.Final)
> [transactions-repository-1-1f8dx] 17:54:17,701 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 18) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
> [transactions-repository-1-1f8dx] 17:54:17,703 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-6) WFLYJCA0018: Started Driver service with driver-name = h2
> [transactions-repository-1-1f8dx] 17:54:17,744 INFO [org.jboss.as.naming] (MSC service thread 1-7) WFLYNAM0003: Starting Naming Service
> [transactions-repository-1-1f8dx] 17:54:18,268 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-5) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]
> [transactions-repository-1-1f8dx] 17:54:18,425 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] 17:54:18,429 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-1f8dx] 17:54:18,779 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-5) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/jboss/infinispan-server/standalone/deployments
> [transactions-repository-1-04x09] JAVA_OPTS already set in environment; overriding default settings with values: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-04x09] =========================================================================
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] JBoss Bootstrap Environment
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] JBOSS_HOME: /opt/jboss/infinispan-server
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] JAVA: /usr/lib/jvm/java/bin/java
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] JAVA_OPTS: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] =========================================================================
> [transactions-repository-1-04x09]
> [transactions-repository-1-04x09] 17:54:15,628 INFO [org.jboss.modules] (main) JBoss Modules version 1.5.2.Final
> [transactions-repository-1-04x09] 17:54:15,849 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
> [transactions-repository-1-04x09] 17:54:15,960 INFO [org.jboss.as] (MSC service thread 1-5) WFLYSRV0049: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) starting
> [transactions-repository-1-04x09] [GC (CMS Initial Mark) 218119K(1013632K), 0.0624692 secs]
> [transactions-repository-1-04x09] 17:54:17,475 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 13) WFLYCTL0028: Attribute 'default-stack' in the resource at address '/subsystem=datagrid-jgroups' is deprecated, and may be removed in future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
> [transactions-repository-1-04x09] 17:54:17,647 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
> [transactions-repository-1-04x09] 17:54:17,664 INFO [org.xnio] (MSC service thread 1-3) XNIO version 3.4.0.Final
> [transactions-repository-1-04x09] 17:54:17,683 INFO [org.xnio.nio] (MSC service thread 1-3) XNIO NIO Implementation Version 3.4.0.Final
> [transactions-repository-1-04x09] 17:54:17,763 INFO [org.jboss.remoting] (MSC service thread 1-4) JBoss Remoting version 4.0.21.Final
> [transactions-repository-1-04x09] [GC (Allocation Failure) 279616K->29652K(1013632K), 0.0690895 secs]
> [transactions-repository-1-04x09] 17:54:17,872 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 19) Activating Infinispan subsystem.
> [transactions-repository-1-04x09] 17:54:17,875 INFO [org.infinispan.server.jgroups] (ServerService Thread Pool -- 21) DGJGRP0001: Activating JGroups subsystem.
> [transactions-repository-1-04x09] [GC (CMS Final Remark) 68259K(1013632K), 0.0485831 secs]
> [transactions-repository-1-04x09] 17:54:17,933 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 22) WFLYIO001: Worker 'default' has auto-configured to 16 core threads with 128 task threads based on your 8 available processors
> [transactions-repository-1-04x09] 17:54:17,960 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 29) WFLYTX0013: Node identifier property is set to the default value. Please make sure it is unique.
> [transactions-repository-1-04x09] 17:54:17,972 INFO [org.jboss.as.security] (ServerService Thread Pool -- 28) WFLYSEC0002: Activating Security Subsystem
> [transactions-repository-1-04x09] 17:54:17,972 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 26) WFLYNAM0001: Activating Naming Subsystem
> [transactions-repository-1-04x09] 17:54:17,984 INFO [org.jboss.as.security] (MSC service thread 1-4) WFLYSEC0001: Current PicketBox version=4.9.6.Final
> [transactions-repository-1-04x09] 17:54:18,004 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 18) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
> [transactions-repository-1-04x09] 17:54:18,027 INFO [org.jboss.as.naming] (MSC service thread 1-5) WFLYNAM0003: Starting Naming Service
> [transactions-repository-1-04x09] 17:54:18,053 INFO [org.jboss.as.connector] (MSC service thread 1-8) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.3.4.Final)
> [transactions-repository-1-04x09] 17:54:18,065 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-6) WFLYJCA0018: Started Driver service with driver-name = h2
> [transactions-repository-1-04x09] 17:54:18,543 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-04x09] 17:54:18,561 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-04x09] 17:54:18,584 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-6) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]
> [transactions-repository-1-04x09] 17:54:18,795 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-2) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/jboss/infinispan-server/standalone/deployments
> [transactions-repository-1-4z05w] JAVA_OPTS already set in environment; overriding default settings with values: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-4z05w] =========================================================================
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] JBoss Bootstrap Environment
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] JBOSS_HOME: /opt/jboss/infinispan-server
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] JAVA: /usr/lib/jvm/java/bin/java
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] JAVA_OPTS: -server -Xms1G -Xmx1G -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -XX:+PrintGC
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] =========================================================================
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] 17:54:15,714 INFO [org.jboss.modules] (main) JBoss Modules version 1.5.2.Final
> [transactions-repository-1-4z05w] 17:54:15,950 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
> [transactions-repository-1-4z05w] 17:54:16,016 INFO [org.jboss.as] (MSC service thread 1-6) WFLYSRV0049: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) starting
> [transactions-repository-1-4z05w] 17:54:17,364 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 5) WFLYCTL0028: Attribute 'default-stack' in the resource at address '/subsystem=datagrid-jgroups' is deprecated, and may be removed in future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
> [transactions-repository-1-4z05w] [GC (CMS Initial Mark) 234897K(1013632K), 0.0587177 secs]
> [transactions-repository-1-4z05w] 17:54:17,599 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
> [transactions-repository-1-4z05w] 17:54:17,620 INFO [org.xnio] (MSC service thread 1-5) XNIO version 3.4.0.Final
> [transactions-repository-1-4z05w] 17:54:17,626 INFO [org.xnio.nio] (MSC service thread 1-5) XNIO NIO Implementation Version 3.4.0.Final
> [transactions-repository-1-4z05w] 17:54:17,674 INFO [org.jboss.remoting] (MSC service thread 1-5) JBoss Remoting version 4.0.21.Final
> [transactions-repository-1-4z05w] [GC (Allocation Failure) 279616K->29736K(1013632K), 0.0733498 secs]
> [transactions-repository-1-4z05w] 17:54:17,799 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 19) Activating Infinispan subsystem.
> [transactions-repository-1-4z05w] 17:54:17,802 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 29) WFLYTX0013: Node identifier property is set to the default value. Please make sure it is unique.
> [transactions-repository-1-4z05w] 17:54:17,829 INFO [org.jboss.as.security] (ServerService Thread Pool -- 28) WFLYSEC0002: Activating Security Subsystem
> [transactions-repository-1-4z05w] 17:54:17,831 INFO [org.infinispan.server.jgroups] (ServerService Thread Pool -- 21) DGJGRP0001: Activating JGroups subsystem.
> [transactions-repository-1-4z05w] 17:54:17,835 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 22) WFLYIO001: Worker 'default' has auto-configured to 16 core threads with 128 task threads based on your 8 available processors
> [transactions-repository-1-4z05w] 17:54:17,870 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 26) WFLYNAM0001: Activating Naming Subsystem
> [transactions-repository-1-4z05w] 17:54:17,881 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 18) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
> [transactions-repository-1-4z05w] 17:54:17,887 INFO [org.jboss.as.connector] (MSC service thread 1-2) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.3.4.Final)
> [transactions-repository-1-4z05w] 17:54:17,895 INFO [org.jboss.as.security] (MSC service thread 1-4) WFLYSEC0001: Current PicketBox version=4.9.6.Final
> [transactions-repository-1-4z05w] 17:54:17,900 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-8) WFLYJCA0018: Started Driver service with driver-name = h2
> [transactions-repository-1-4z05w] 17:54:18,037 INFO [org.jboss.as.naming] (MSC service thread 1-6) WFLYNAM0003: Starting Naming Service
> [transactions-repository-1-4z05w] 17:54:18,842 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-5) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]
> [transactions-repository-1-4z05w] 17:54:18,920 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-4z05w] 17:54:18,934 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 19) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-4z05w] 17:54:19,094 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-6) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/jboss/infinispan-server/standalone/deployments
> [transactions-repository-1-4z05w] 17:54:20,553 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-6) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-4z05w] 17:54:20,555 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-6) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-4z05w] [GC (CMS Final Remark) 171687K(1013632K), 0.0585468 secs]
> [transactions-repository-1-4z05w] 17:54:20,716 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000078: Starting JGroups channel clustered
> [transactions-repository-1-4z05w] 17:54:20,732 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000094: Received new cluster view for channel clustered: [transactions-repository-1-4z05w|2] (3) [transactions-repository-1-4z05w, transactions-repository-1-04x09, transactions-repository-1-1f8dx]
> [transactions-repository-1-4z05w] 17:54:20,742 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000079: Channel clustered local address is transactions-repository-1-4z05w, physical addresses are [172.17.0.3:7600]
> [transactions-repository-1-4z05w] 17:54:20,754 INFO [org.infinispan.globalstate.impl.GlobalStateManagerImpl] (MSC service thread 1-6) ISPN000390: Persisted state, version=9.0.0-SNAPSHOT timestamp=2017-02-18T17:54:20.748Z
> [transactions-repository-1-4z05w] 17:54:20,755 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-6) ISPN000128: Infinispan version: Infinispan 'Ruppaner' 9.0.0-SNAPSHOT
> [transactions-repository-1-4z05w] 17:54:20,933 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-6) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-4z05w] 17:54:20,933 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-6) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-4z05w] 17:54:20,936 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-5) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-4z05w] 17:54:20,939 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-4) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-4z05w] 17:54:20,943 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-4) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-4z05w] 17:54:20,954 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-4) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-4z05w] 17:54:20,950 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-5) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-4z05w] 17:54:20,956 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-5) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] [GC (CMS Final Remark) 172593K(1013632K), 0.0447491 secs]
> [transactions-repository-1-04x09] 17:54:20,995 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-3) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:20,996 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-3) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:21,109 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000078: Starting JGroups channel clustered
> [transactions-repository-1-04x09] 17:54:21,129 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000094: Received new cluster view for channel clustered: [transactions-repository-1-4z05w|2] (3) [transactions-repository-1-4z05w, transactions-repository-1-04x09, transactions-repository-1-1f8dx]
> [transactions-repository-1-04x09] 17:54:21,138 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000079: Channel clustered local address is transactions-repository-1-04x09, physical addresses are [172.17.0.4:7600]
> [transactions-repository-1-1f8dx] 17:54:21,158 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-8) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-1f8dx] 17:54:21,158 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-8) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:21,160 INFO [org.infinispan.globalstate.impl.GlobalStateManagerImpl] (MSC service thread 1-3) ISPN000390: Persisted state, version=9.0.0-SNAPSHOT timestamp=2017-02-18T17:54:21.156Z
> [transactions-repository-1-04x09] 17:54:21,160 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-3) ISPN000128: Infinispan version: Infinispan 'Ruppaner' 9.0.0-SNAPSHOT
> [transactions-repository-1-1f8dx] 17:54:21,254 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000078: Starting JGroups channel clustered
> [transactions-repository-1-1f8dx] 17:54:21,266 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000094: Received new cluster view for channel clustered: [transactions-repository-1-4z05w|2] (3) [transactions-repository-1-4z05w, transactions-repository-1-04x09, transactions-repository-1-1f8dx]
> [transactions-repository-1-1f8dx] 17:54:21,269 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000079: Channel clustered local address is transactions-repository-1-1f8dx, physical addresses are [172.17.0.6:7600]
> [transactions-repository-1-1f8dx] 17:54:21,282 INFO [org.infinispan.globalstate.impl.GlobalStateManagerImpl] (MSC service thread 1-8) ISPN000390: Persisted state, version=9.0.0-SNAPSHOT timestamp=2017-02-18T17:54:21.279Z
> [transactions-repository-1-1f8dx] 17:54:21,283 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-8) ISPN000128: Infinispan version: Infinispan 'Ruppaner' 9.0.0-SNAPSHOT
> [transactions-repository-1-04x09] 17:54:21,297 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-3) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:21,297 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-3) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-04x09] 17:54:21,300 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-6) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-04x09] 17:54:21,303 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-2) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-04x09] 17:54:21,305 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-6) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-04x09] 17:54:21,310 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-6) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-04x09] 17:54:21,310 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-2) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-04x09] 17:54:21,311 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-2) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] 17:54:21,443 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-8) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-1f8dx] 17:54:21,443 WARN [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder] (MSC service thread 1-8) ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup
> [transactions-repository-1-1f8dx] 17:54:21,449 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-8) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-1f8dx] 17:54:21,450 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-8) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-1f8dx] 17:54:21,452 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-4) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] 17:54:21,462 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-4) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-1f8dx] 17:54:21,462 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-8) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [transactions-repository-1-1f8dx] 17:54:21,468 WARN [org.infinispan.configuration.cache.IndexingConfigurationBuilder] (MSC service thread 1-4) ISPN000403: No indexable classes were defined for this indexed cache; switching to autodetection (support for autodetection will be removed in Infinispan 9.0).
> [transactions-repository-1-04x09] [GC (CMS Initial Mark) 208341K(1013632K), 0.0101475 secs]
> [transactions-repository-1-4z05w] 17:54:23,974 ERROR [org.infinispan.topology.ClusterCacheStatus] (transport-thread--p4-t9) ISPN000228: Failed to recover cache transactions state after the current node became the coordinator: java.lang.NullPointerException
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterCacheStatus.updateAvailabilityMode(ClusterCacheStatus.java:147)
> [transactions-repository-1-4z05w] at org.infinispan.partitionhandling.impl.PreferConsistencyStrategy.updateMembersAndRebalance(PreferConsistencyStrategy.java:259)
> [transactions-repository-1-4z05w] at org.infinispan.partitionhandling.impl.PreferConsistencyStrategy.onPartitionMerge(PreferConsistencyStrategy.java:210)
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterCacheStatus.doMergePartitions(ClusterCacheStatus.java:554)
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$recoverClusterStatus$3(ClusterTopologyManagerImpl.java:499)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174)
> [transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [transactions-repository-1-4z05w] at java.lang.Thread.run(Thread.java:745)
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] 17:54:23,974 ERROR [org.infinispan.topology.ClusterCacheStatus] (transport-thread--p4-t7) ISPN000228: Failed to recover cache sessions state after the current node became the coordinator: java.lang.NullPointerException
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterCacheStatus.updateAvailabilityMode(ClusterCacheStatus.java:147)
> [transactions-repository-1-4z05w] at org.infinispan.partitionhandling.impl.PreferConsistencyStrategy.updateMembersAndRebalance(PreferConsistencyStrategy.java:259)
> [transactions-repository-1-4z05w] at org.infinispan.partitionhandling.impl.PreferConsistencyStrategy.onPartitionMerge(PreferConsistencyStrategy.java:210)
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterCacheStatus.doMergePartitions(ClusterCacheStatus.java:554)
> [transactions-repository-1-4z05w] at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$recoverClusterStatus$3(ClusterTopologyManagerImpl.java:499)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33)
> [transactions-repository-1-4z05w] at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174)
> [transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [transactions-repository-1-4z05w] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [transactions-repository-1-4z05w] at java.lang.Thread.run(Thread.java:745)
> [transactions-repository-1-4z05w]
> [transactions-repository-1-4z05w] 17:54:24,005 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-8) DGISPN0001: Started transactions cache from clustered container
> [transactions-repository-1-4z05w] 17:54:24,005 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-1) DGISPN0001: Started default cache from clustered container
> [transactions-repository-1-04x09] 17:54:24,005 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) DGISPN0001: Started default cache from clustered container
> [transactions-repository-1-04x09] 17:54:24,006 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0001: Started memcachedCache cache from clustered container
> [transactions-repository-1-04x09] 17:54:24,006 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-1) DGISPN0001: Started transactions cache from clustered container
> [transactions-repository-1-4z05w] 17:54:24,008 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) DGISPN0001: Started memcachedCache cache from clustered container
> [transactions-repository-1-4z05w] 17:54:24,010 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-6) DGISPN0001: Started sessions cache from clustered container
> [transactions-repository-1-04x09] 17:54:24,011 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-3) DGISPN0001: Started sessions cache from clustered container
> [transactions-repository-1-1f8dx] 17:54:24,012 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-5) DGISPN0001: Started sessions cache from clustered container
> [transactions-repository-1-1f8dx] 17:54:24,012 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-6) DGISPN0001: Started memcachedCache cache from clustered container
> [transactions-repository-1-1f8dx] 17:54:24,012 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-2) DGISPN0001: Started default cache from clustered container
> [transactions-repository-1-1f8dx] 17:54:24,013 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-7) DGISPN0001: Started transactions cache from clustered container
> [transactions-repository-1-4z05w] 17:54:24,036 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) DGENDPT10000: MemcachedServer starting
> [transactions-repository-1-4z05w] 17:54:24,039 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10000: HotRodServer starting
> [transactions-repository-1-4z05w] 17:54:24,039 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10001: HotRodServer listening on 172.17.0.3:11222
> [transactions-repository-1-4z05w] 17:54:24,040 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10000: WebSocketServer starting
> [transactions-repository-1-4z05w] 17:54:24,040 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10001: WebSocketServer listening on 172.17.0.3:8181
> [transactions-repository-1-1f8dx] 17:54:24,046 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) DGENDPT10000: WebSocketServer starting
> [transactions-repository-1-1f8dx] 17:54:24,045 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10000: MemcachedServer starting
> [transactions-repository-1-1f8dx] 17:54:24,046 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10000: HotRodServer starting
> [transactions-repository-1-1f8dx] 17:54:24,047 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10001: HotRodServer listening on 172.17.0.6:11222
> [transactions-repository-1-1f8dx] 17:54:24,048 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10001: MemcachedServer listening on 172.17.0.6:11211
> [transactions-repository-1-04x09] 17:54:24,051 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10000: MemcachedServer starting
> [transactions-repository-1-04x09] 17:54:24,051 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10000: WebSocketServer starting
> [transactions-repository-1-04x09] 17:54:24,052 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10000: HotRodServer starting
> [transactions-repository-1-04x09] 17:54:24,055 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10001: HotRodServer listening on 172.17.0.4:11222
> [transactions-repository-1-04x09] 17:54:24,055 INFO [org.infinispan.server.endpoint] (MSC service thread 1-3) DGENDPT10001: MemcachedServer listening on 172.17.0.4:11211
> [transactions-repository-1-04x09] 17:54:24,056 INFO [org.infinispan.server.endpoint] (MSC service thread 1-4) DGENDPT10001: WebSocketServer listening on 172.17.0.4:8181
> [transactions-repository-1-4z05w] 17:54:24,063 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) DGENDPT10001: MemcachedServer listening on 172.17.0.3:11211
> [transactions-repository-1-1f8dx] 17:54:24,059 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) DGENDPT10001: WebSocketServer listening on 172.17.0.6:8181
> [transactions-repository-1-4z05w] 17:54:24,070 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10000: REST starting
> [transactions-repository-1-1f8dx] 17:54:24,079 INFO [org.infinispan.server.endpoint] (MSC service thread 1-6) DGENDPT10000: REST starting
> [transactions-repository-1-04x09] 17:54:24,071 INFO [org.infinispan.server.endpoint] (MSC service thread 1-8) DGENDPT10000: REST starting
> [transactions-repository-1-4z05w] 17:54:24,193 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t20) ISPN000310: Starting cluster-wide rebalance for cache ___protobuf_metadata, topology CacheTopology{id=2, rebalanceId=2, currentCH=ReplicatedConsistentHash{ns = 256, owners = (1)[transactions-repository-1-4z05w: 256]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 126, transactions-repository-1-1f8dx: 130]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-1f8dx], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, ced31139-f1a1-4ee8-96af-f37c5386ecc0]}
> [transactions-repository-1-4z05w] 17:54:24,197 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t20) [Context=___protobuf_metadata][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,246 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t24) [Context=___protobuf_metadata][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,255 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t22) ISPN000310: Starting cluster-wide rebalance for cache ___script_cache, topology CacheTopology{id=2, rebalanceId=2, currentCH=ReplicatedConsistentHash{ns = 256, owners = (1)[transactions-repository-1-4z05w: 256]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 126, transactions-repository-1-1f8dx: 130]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-1f8dx], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, ced31139-f1a1-4ee8-96af-f37c5386ecc0]}
> [transactions-repository-1-4z05w] 17:54:24,256 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t22) [Context=___script_cache][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,272 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t2) [Context=___script_cache][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,317 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t2) ISPN000310: Starting cluster-wide rebalance for cache ___hotRodTopologyCache, topology CacheTopology{id=2, rebalanceId=2, currentCH=ReplicatedConsistentHash{ns = 256, owners = (1)[transactions-repository-1-4z05w: 256]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 134, transactions-repository-1-04x09: 122]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-04x09], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, 5d246b27-021c-4425-b3c4-14017f378635]}
> [transactions-repository-1-4z05w] 17:54:24,318 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t2) [Context=___hotRodTopologyCache][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,321 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t11) [Context=___hotRodTopologyCache][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,364 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t8) [Context=___protobuf_metadata][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,366 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) [Context=___script_cache][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,382 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) [Context=___hotRodTopologyCache][Context=transactions-repository-1-04x09]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,385 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t8) ISPN000336: Finished cluster-wide rebalance for cache ___protobuf_metadata, topology id = 2
> [transactions-repository-1-4z05w] 17:54:24,387 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t8) ISPN000310: Starting cluster-wide rebalance for cache ___protobuf_metadata, topology CacheTopology{id=4, rebalanceId=3, currentCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 126, transactions-repository-1-1f8dx: 130]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (3)[transactions-repository-1-4z05w: 90, transactions-repository-1-1f8dx: 86, transactions-repository-1-04x09: 80]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-1f8dx, transactions-repository-1-04x09], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, ced31139-f1a1-4ee8-96af-f37c5386ecc0, 5d246b27-021c-4425-b3c4-14017f378635]}
> [transactions-repository-1-4z05w] 17:54:24,388 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t8) [Context=___protobuf_metadata][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,394 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) ISPN000336: Finished cluster-wide rebalance for cache ___hotRodTopologyCache, topology id = 2
> [transactions-repository-1-4z05w] 17:54:24,395 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) ISPN000336: Finished cluster-wide rebalance for cache ___script_cache, topology id = 2
> [transactions-repository-1-4z05w] 17:54:24,397 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t19) [Context=___protobuf_metadata][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,397 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t7) [Context=___protobuf_metadata][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,398 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) ISPN000310: Starting cluster-wide rebalance for cache ___script_cache, topology CacheTopology{id=4, rebalanceId=3, currentCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 126, transactions-repository-1-1f8dx: 130]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (3)[transactions-repository-1-4z05w: 90, transactions-repository-1-1f8dx: 86, transactions-repository-1-04x09: 80]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-1f8dx, transactions-repository-1-04x09], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, ced31139-f1a1-4ee8-96af-f37c5386ecc0, 5d246b27-021c-4425-b3c4-14017f378635]}
> [transactions-repository-1-4z05w] 17:54:24,398 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t10) [Context=___script_cache][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,401 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) ISPN000310: Starting cluster-wide rebalance for cache ___hotRodTopologyCache, topology CacheTopology{id=4, rebalanceId=3, currentCH=ReplicatedConsistentHash{ns = 256, owners = (2)[transactions-repository-1-4z05w: 134, transactions-repository-1-04x09: 122]}, pendingCH=ReplicatedConsistentHash{ns = 256, owners = (3)[transactions-repository-1-4z05w: 90, transactions-repository-1-04x09: 80, transactions-repository-1-1f8dx: 86]}, unionCH=null, actualMembers=[transactions-repository-1-4z05w, transactions-repository-1-04x09, transactions-repository-1-1f8dx], persistentUUIDs=[45bed168-6b22-4cc9-b164-6a8b3fe6367f, 5d246b27-021c-4425-b3c4-14017f378635, ced31139-f1a1-4ee8-96af-f37c5386ecc0]}
> [transactions-repository-1-4z05w] 17:54:24,405 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t6) [Context=___hotRodTopologyCache][Context=transactions-repository-1-4z05w]ISPN100002: Started local rebalance
> [transactions-repository-1-4z05w] 17:54:24,407 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t20) [Context=___hotRodTopologyCache][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,409 INFO [org.infinispan.CLUSTER] (transport-thread--p4-t3) [Context=___script_cache][Context=transactions-repository-1-4z05w]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,420 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t11) [Context=___script_cache][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,425 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t11) [Context=___hotRodTopologyCache][Context=transactions-repository-1-04x09]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,476 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t17) [Context=___hotRodTopologyCache][Context=transactions-repository-1-1f8dx]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,480 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t17) ISPN000336: Finished cluster-wide rebalance for cache ___hotRodTopologyCache, topology id = 4
> [transactions-repository-1-4z05w] 17:54:24,536 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t21) [Context=___protobuf_metadata][Context=transactions-repository-1-04x09]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,537 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t21) ISPN000336: Finished cluster-wide rebalance for cache ___protobuf_metadata, topology id = 4
> [transactions-repository-1-4z05w] 17:54:24,544 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t24) [Context=___script_cache][Context=transactions-repository-1-04x09]ISPN100003: Finished local rebalance
> [transactions-repository-1-4z05w] 17:54:24,546 INFO [org.infinispan.CLUSTER] (remote-thread--p2-t24) ISPN000336: Finished cluster-wide rebalance for cache ___script_cache, topology id = 4
> [transactions-repository-1-4z05w] 17:54:24,567 INFO [org.infinispan.rest.NettyRestServer] (MSC service thread 1-1) ISPN012003: REST server starting, listening on 172.17.0.3:8080
> [transactions-repository-1-4z05w] 17:54:24,568 INFO [org.infinispan.server.endpoint] (MSC service thread 1-1) DGENDPT10002: REST mapped to /rest
> [transactions-repository-1-1f8dx] 17:54:24,576 INFO [org.infinispan.rest.NettyRestServer] (MSC service thread 1-6) ISPN012003: REST server starting, listening on 172.17.0.6:8080
> [transactions-repository-1-1f8dx] 17:54:24,577 INFO [org.infinispan.server.endpoint] (MSC service thread 1-6) DGENDPT10002: REST mapped to /rest
> [transactions-repository-1-04x09] 17:54:24,639 INFO [org.infinispan.rest.NettyRestServer] (MSC service thread 1-8) ISPN012003: REST server starting, listening on 172.17.0.4:8080
> [transactions-repository-1-04x09] 17:54:24,639 INFO [org.infinispan.server.endpoint] (MSC service thread 1-8) DGENDPT10002: REST mapped to /rest
> [transactions-repository-1-1f8dx] [GC (Allocation Failure) 309342K->63160K(1013632K), 0.1823599 secs]
> [transactions-repository-1-4z05w] [GC (Allocation Failure) 309352K->65174K(1013632K), 0.1616480 secs]
> [transactions-repository-1-04x09] [GC (Allocation Failure) 309268K->61649K(1013632K), 0.1507677 secs]
> [transactions-repository-1-1f8dx] 17:54:25,021 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management
> [transactions-repository-1-1f8dx] 17:54:25,022 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://0.0.0.0:9990
> [transactions-repository-1-1f8dx] 17:54:25,022 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) started in 9718ms - Started 197 of 282 services (164 services are lazy, passive or on-demand)
> [transactions-repository-1-4z05w] 17:54:25,021 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management
> [transactions-repository-1-4z05w] 17:54:25,024 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://0.0.0.0:9990
> [transactions-repository-1-4z05w] 17:54:25,027 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) started in 9643ms - Started 197 of 282 services (164 services are lazy, passive or on-demand)
> [transactions-repository-1-04x09] 17:54:25,063 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management
> [transactions-repository-1-04x09] 17:54:25,064 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://0.0.0.0:9990
> [transactions-repository-1-04x09] 17:54:25,064 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Infinispan Server 9.0.0-SNAPSHOT (WildFly Core 2.2.0.Final) started in 9746ms - Started 197 of 282 services (164 services are lazy, passive or on-demand)
> [transactions-repository-1-04x09] [GC (CMS Final Remark) 89552K(1013632K), 0.0310821 secs]
> {noformat}
> Note that I applied multiplexed logs from 3 nodes. I hope this will be helpful for diagnosing.
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * Kill some nodes
> * Start them again
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0
[JBoss JIRA] (ISPN-7525) ClusterCacheStatus additional fix
by Sebastian Łaskawiec (JIRA)
Sebastian Łaskawiec created ISPN-7525:
-----------------------------------------
Summary: ClusterCacheStatus additional fix
Key: ISPN-7525
URL: https://issues.jboss.org/browse/ISPN-7525
Project: Infinispan
Issue Type: Enhancement
Reporter: Sebastian Łaskawiec
Assignee: Sebastian Łaskawiec
Priority: Blocker
Fix For: 9.0.0.CR3
As [~dan.berindei] mentioned in the [PR|https://github.com/infinispan/infinispan/pull/4871#discussion_r102774407] we should look at additional scenario (when coordinator has newer topology) and also fix the test. It should fail when the fix is not there.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0
[JBoss JIRA] (ISPN-7522) LocalMode[No]PassivationTest fails randomly
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-7522?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-7522:
-------------------------------------
This is actually a bug in the CacheLoader bulk operations iterator most likely. Have to look further.
> LocalMode[No]PassivationTest fails randomly
> -------------------------------------------
>
> Key: ISPN-7522
> URL: https://issues.jboss.org/browse/ISPN-7522
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Eviction, Loaders and Stores
> Reporter: William Burns
> Assignee: William Burns
>
> LocalModePassivationTest and LocalModeNoPassivationTest can fail randomly saying entries aren't present. All entries should be present since there is a configured store, this is a pretty big loss of data.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
- 1
- 0