[JBoss JIRA] (ISPN-5386) Tx succeeds on coord, while being rollbacked on other participants due to Tx pruning
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5386?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5386:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1212795|https://bugzilla.redhat.com/show_bug.cgi?id=1212795] from VERIFIED to CLOSED
> Tx succeeds on coord, while being rollbacked on other participants due to Tx pruning
> ------------------------------------------------------------------------------------
>
> Key: ISPN-5386
> URL: https://issues.jboss.org/browse/ISPN-5386
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Reporter: Matej Čimbora
> Assignee: Dan Berindei
> Fix For: 7.2.0.Final
>
>
> All participants of transaction share the same topology. TX gets successfully prepared & commited on coordinator.
> {code}
> 03:49:27,759 DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,edg-perf08-48196) New view accepted: [edg-perf08-48196|18] (5) [edg-perf08-48196, edg-perf01-23632, edg-perf02-34805, edg-perf03-16232, edg-perf04-41106]
> 03:49:41,051 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (transport-thread-9) Installing new cache topology CacheTopology{id=53, rebalanceId=19, currentCH=DefaultConsistentHash{ns = 512, owners = (5)[edg-perf08-48196: 103+101, edg-perf01-23632: 102+103, edg-perf02-34805: 102+103, edg-perf03-16232: 102+103, edg-perf04-41106: 103+102]}, pendingCH=null, unionCH=null, actualMembers=[edg-perf08-48196, edg-perf01-23632, edg-perf02-34805, edg-perf03-16232, edg-perf04-41106]} on cache testCache
> ...
> 03:51:34,005 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (DefaultStressor-1) edg-perf08-48196 invoking PrepareCommand { ... gtx=GlobalTransaction:<edg-perf08-48196>:13330:local, cacheName='testCache', topologyId=53} to recipient list [edg-perf03-16232, edg-perf08-48196, edg-perf02-34805, edg-perf04-41106, edg-perf01-23632]
> 03:51:36,329 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (DefaultStressor-1) Responses: [sender=edg-perf03-16232,received=true, suspected=false] [sender=edg-perf02-34805, received=true, suspected=false] [sender=edg-perf04-41106, received=true, suspected=false] [sender=edg-perf01-23632, received=true, suspected=false]
> 03:51:36,342 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (DefaultStressor-1) edg-perf08-48196 invoking CommitCommand {gtx=GlobalTransaction:<edg-perf08-48196>:13330:local, cacheName='testCache', topologyId=53} to recipient list [edg-perf03-16232, edg-perf08-48196, edg-perf02-34805, edg-perf04-41106, edg-perf01-23632] with options RpcOptions{timeout=60000, unit=MILLISECONDS, fifoOrder=false, totalOrder=false, responseFilter=null, responseMode=SYNCHRONOUS_IGNORE_LEAVERS, skipReplicationQueue=false}
> 03:51:36,703 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (DefaultStressor-1) Responses: [sender=edg-perf03-16232, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] [sender=edg-perf02-34805, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] [sender=edg-perf04-41106, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] [sender=edg-perf01-23632, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false]
> {code}
> The problem is, that other participating nodes rollback it, as TX with higher id was completed before. Successfull response is returned for both prepare & commit commands.
> {code}
> 03:49:58,190 TRACE [org.infinispan.transaction.TransactionTable] (remote-thread-499) Marking transaction GlobalTransaction:<edg-perf08-48196>:13337:local as completed
> ...
> 03:51:34,122 TRACE [org.infinispan.transaction.TransactionTable] (remote-thread-593) Created and registered remote transaction RemoteTransaction{ ... lookedUpEntries={}, lockedKeys=null, backupKeyLocks=null, lookedUpEntriesTopology=2147483647, isMarkedForRollback=false, tx=GlobalTransaction:<edg-perf08-48196>:13330:remote, state=null}
> 03:51:34,073 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread-593) Calling perform() on PrepareCommand { ... gtx=GlobalTransaction:<edg-perf08-48196>:13330:remote, cacheName='testCache', topologyId=53}
> 03:51:34,342 TRACE [org.infinispan.interceptors.TxInterceptor] (remote-thread-593) Rolling back remote transaction GlobalTransaction:<edg-perf08-48196>:13330:remote because either already completed (true) or originator no longer in the cluster (false).
> 03:51:34,639 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread-593) About to send back response null for command PrepareCommand { ... gtx=GlobalTransaction:<edg-perf08-48196>:13330:remote, cacheName='testCache', topologyId=53}
> 03:51:36,355 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread-589) Calling perform() on CommitCommand {gtx=GlobalTransaction:<edg-perf08-48196>:13330:remote, cacheName='testCache', topologyId=53}
> 03:51:36,355 TRACE [org.infinispan.commands.tx.AbstractTransactionBoundaryCommand] (remote-thread-589) Did not find a RemoteTransaction for GlobalTransaction:<edg-perf08-48196>:13330:remote
> 03:51:36,355 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread-589) About to send back response SuccessfulResponse{responseValue=null} for command CommitCommand {gtx=GlobalTransaction:<edg-perf08-48196>:13330:remote, cacheName='testCache', topologyId=53}
> {code}
> Exception response should be returned instead to avoid incorrect assumptions about presence of updated entry in the cache.
> [~dan.berindei] spotted lastPrunedTxId modifications are not logged, let's make sure they are.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 10 months
[JBoss JIRA] (ISPN-5261) Transaction is recommited by joining node during merge
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5261?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5261:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1198267|https://bugzilla.redhat.com/show_bug.cgi?id=1198267] from VERIFIED to CLOSED
> Transaction is recommited by joining node during merge
> ------------------------------------------------------
>
> Key: ISPN-5261
> URL: https://issues.jboss.org/browse/ISPN-5261
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.1.1.Final
> Reporter: Matej Čimbora
> Assignee: Dan Berindei
> Fix For: 7.2.1.Final
>
>
> Transaction gets recommited after a node joins a cluster during merge and requests retransmission of previous messages, overwriting updates which happened till that point.
> 4 nodes - edg-perf01-04
> Scenario:
> 1. edg-perf01 begins a new transaction & updates value of K
> {code}
> 06:50:50,421 TRACE [org.infinispan.transaction.xa.TransactionXaAdapter] (DefaultStressor-7) start called on tx GlobalTransaction:<edg-perf01-19349>:3516:local
> {code}
> The latest view on edg-perf01 is
> {code}
> 06:50:35,103 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,edg-perf01-19349) ISPN000093: Received new, MERGED cluster view: MergeView::[edg-perf01-19349|5] (3) [edg-perf01-19349, edg-perf04-20753, edg-perf02-19191], 1 subgroups: [edg-perf01-19349|4] (2) [edg-perf01-19349, edg-perf02-19191]
> {code}
> 2. Prepare command is sent with nodes edg-perf02 & edg-perf04 responding successfully
> {code}
> 06:50:51,186 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (DefaultStressor-7) edg-perf01-19349 invoking PrepareCommand ... topologyId=12} to recipient list [edg-perf02-19191, edg-perf04-20753, edg-perf01-19349] ...
> {code}
> check message id & seqno
> {code}
> (06:50:51,195 TRACE [org.jgroups.protocols.TCP] (DefaultStressor-7) edg-perf01-19349: sending msg to null, src=edg-perf01-19349, headers are RequestCorrelator: id=200, type=REQ, id=17134, rsp_expected=true, exclusion_list=[edg-perf01-19349], NAKACK2: [MSG, seqno=1020], 20903: slaveIndex=0, TCP: [cluster_name=default])
> {code}
> 3. Transaction gets commited & value of K gets updated multiple times on edg-perf01 from this point on
> 4. edg-perf03 receives a new view containing all nodes
> {code}
> 06:50:53,963 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,edg-perf03-14171) ISPN000093: Received new, MERGED cluster view: MergeView::[edg-perf01-19349|6] (4) [edg-perf01-19349, edg-perf03-14171, edg-perf04-20753, edg-perf02-19191], 3 subgroups: [edg-perf03-14171|4] (2) [edg-perf03-14171, edg-perf04-20753], [edg-perf01-19349|5] (3) [edg-perf01-19349, edg-perf04-20753, edg-perf02-19191], [edg-perf01-19349|4] (2) [edg-perf01-19349, edg-perf02-19191]
> {code}
> 5. edg-perf03 apparently requests retransmission of previous messages (containing prepare from step#2) & edg-perf01 sends the response
> Request
> {code}
> 06:51:35,241 TRACE [org.jgroups.protocols.pbcast.NAKACK2] (Timer-3,edg-perf03-14171) edg-perf03-14171: sending XMIT_REQ ((79): {1015-1093}) to edg-perf01-19349
> {code}
> Response
> {code}
> 06:51:35,299 TRACE [org.jgroups.protocols.TCP] (INT-14,edg-perf03-14171) edg-perf03-14171: received [dst: edg-perf03-14171, src: edg-perf01-19349 (4 headers), size=975 bytes, flags=OOB|DONT_BUNDLE|NO_TOTAL_ORDER|INTERNAL], headers are RequestCorrelator: id=200, type=REQ, id=17134, rsp_expected=true, exclusion_list=[edg-perf01-19349], NAKACK2: [XMIT_RSP, seqno=1020], 20903: slaveIndex=0, TCP: [cluster_name=default]
> {code}
> 6. edg-perf03 prepares the old transaction & eventually commits it again
> {code}
> 06:51:35,304 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (INT-14,edg-perf03-14171) Attempting to execute command: PrepareCommand.....gtx=GlobalTransaction:<edg-perf01-19349>:3516:local, cacheName='testCache', topologyId=12} [sender=edg-perf01-19349]
> 06:51:35,304 TRACE [org.infinispan.statetransfer.StateTransferLockImpl] (INT-14,edg-perf03-14171) Checking if transaction data was received for topology 12, current topology is 14
> 06:51:35,305 TRACE [org.infinispan.transaction.TransactionTable] (remote-thread-40) Created and registered remote transaction ... tx=GlobalTransaction:<edg-perf01-19349>:3516:remote, ...
> 06:51:35,308 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (remote-thread-40) edg-perf03-14171 invoking PrepareCommand ... gtx=GlobalTransaction:<edg-perf01-19349>:3516:remote, cacheName='testCache', topologyId=15} to recipient list [edg-perf04-20753, edg-perf02-19191] ...
> {code}
> Successful commit responses
> {code}
> 06:51:36,750 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (remote-thread-44) Response(s) to CommitCommand {gtx=GlobalTransaction:<edg-perf01-19349>:3516:remote, cacheName='testCache', topologyId=15} is {edg-perf02-19191=SuccessfulResponse{responseValue=null} , edg-perf04-20753=SuccessfulResponse{responseValue=null} }
> {code}
> 7. When get(K) is invoked on edg-perf01, stale value is returned (the one from step #6, ignoring updates which have happened since step #3)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 10 months
[JBoss JIRA] (ISPN-5180) Hot Rod type converter in Compatibility should not marshall a byte[] again
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5180?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5180:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1196277|https://bugzilla.redhat.com/show_bug.cgi?id=1196277] from VERIFIED to CLOSED
> Hot Rod type converter in Compatibility should not marshall a byte[] again
> --------------------------------------------------------------------------
>
> Key: ISPN-5180
> URL: https://issues.jboss.org/browse/ISPN-5180
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Protocols
> Affects Versions: 7.0.3.Final, 7.1.0.CR2
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Priority: Critical
> Labels: compatibility
> Fix For: 7.1.0.Final
>
>
> When compatibility is enabled, storing a java serialized object with REST, and then retrieving it with Hot Rod results in byte[] being returned instead of the deserilialized object:
> {code}java.lang.AssertionError:
> Expected :org.infinispan.it.compatibility.EmbeddedRestHotRodTest$Person@22cd6f
> Actual :[B@2ff9eb2a
> at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
> at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364)
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80)
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:88)
> at org.infinispan.it.compatibility.EmbeddedRestHotRodTest.testCustomObjectRestPutHotRodEmbeddedGet(EmbeddedRestHotRodTest.java:217)
> {code}
> This is because when the entry is retrieved by Hot Rod and it's going to marshall it for sending it back to the client, it should check if the entry is already a byte[], in which case it should not touch it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 10 months
[JBoss JIRA] (ISPN-4503) Commands with topology id 0 are not properly ignored on joiners
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4503?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4503:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1220328|https://bugzilla.redhat.com/show_bug.cgi?id=1220328] from VERIFIED to CLOSED
> Commands with topology id 0 are not properly ignored on joiners
> ---------------------------------------------------------------
>
> Key: ISPN-4503
> URL: https://issues.jboss.org/browse/ISPN-4503
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 7.0.0.Beta1
>
>
> InboundInvocationHandlerImpl is supposed to ignore commands sent with a topology id smaller than the first topology id in which the local node was a member. But there is a loophole when the command was sent with topology id 0.
> This is visible in StateTransferFunctionalTest, where the writing thread keeps the cpu busy and can delay the 2nd node joining for a long time (especially when run on a single core with {{taskset -c 0}}). For some reason, the PrepareCommands are sent only to the local node, while the TxCompletionNotificationCommands are sent to the entire cluster ({{null}}). When the 2nd node manages to join, it receives a lot of TxCompletionNotificationCommands and processing them delays the processing of the rebalance commands. Since the writes eventually block waiting for the new topology to be installed on the joiner, the delayed rebalance commands cause the write to time out.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 10 months
[JBoss JIRA] (ISPN-4504) Topology id is not properly set on ClusteredGetCommands
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4504?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4504:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1220328|https://bugzilla.redhat.com/show_bug.cgi?id=1220328] from VERIFIED to CLOSED
> Topology id is not properly set on ClusteredGetCommands
> -------------------------------------------------------
>
> Key: ISPN-4504
> URL: https://issues.jboss.org/browse/ISPN-4504
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 7.0.0.Beta1
>
>
> Because the topology id is not properly set on ClusteredGetCommands, they don't wait for the sender's topology to be installed on the target.
> I have tried to fix this while implementing a fix for ISPN-4503. My solution caused 2 failures in RemoteGetDuringStateTransferTest, scenarios 5 and 7::
> | sc | currentTopologyId | currentTopologyId + 1 (rebalance) | currentTopologyId + 2 (finish) |
> | 5 | 0:remoteGet | 2:sendReply | 0:receiveReply, 1:sendReply |
> | 7 | | 0:remoteGet, 2: sendReply | 0:receiveReply, 1:sendReply |
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 10 months
[JBoss JIRA] (ISPN-5259) Disabling rebalancing causes errors when starting the second server in a cluster
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5259?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5259:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1200437|https://bugzilla.redhat.com/show_bug.cgi?id=1200437] from VERIFIED to CLOSED
> Disabling rebalancing causes errors when starting the second server in a cluster
> --------------------------------------------------------------------------------
>
> Key: ISPN-5259
> URL: https://issues.jboss.org/browse/ISPN-5259
> Project: Infinispan
> Issue Type: Bug
> Components: Server
> Affects Versions: 7.2.0.Alpha1, 7.1.1.Final
> Reporter: Alan Field
> Assignee: Pedro Ruivo
> Fix For: 7.2.0.Beta1, 7.2.0.Final
>
>
> Perform the following steps with Infinispan 7.2 server:
> # Start server using command: bin/clustered.sh -Djboss.node.name=node0 -Djboss.socket.binding.port-offset=0
> # Set JMX attribute jboss.infinispan/CacheManager/"clustered"/LocalTopologyManager/rebalancingEnabled to false
> # Start second serer using command: bin/clustered.sh -Djboss.node.name=node1 -Djboss.socket.binding.port-offset=10
> Result: The first server starts successfully and shows this log message when the JMX attribute is modified:
> {noformat}
> 09:38:05,620 INFO [org.infinispan.CLUSTER] (RMI TCP Connection(3)-127.0.0.1) ISPN000309: Rebalancing suspended
> {noformat}
> The second server starts, but then shows the following error messages:
> {noformat}
> =========================================================================
> JBoss Bootstrap Environment
> JBOSS_HOME: /home/afield/Development/projects/infinispan-public/server/integration/build/target/infinispan-server-7.2.0-SNAPSHOT
> JAVA: /usr/lib/jvm/java/bin/java
> JAVA_OPTS: -server -Xms1303m -Xmx1303m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.server.default.config=clustered.xml -Dsun.nio.ch.bugLevel=''
> =========================================================================
> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
> 09:38:24,626 INFO [org.jboss.modules] (main) JBoss Modules version 1.3.3.Final
> 09:38:24,895 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.2.Final
> 09:38:24,967 INFO [org.jboss.as] (MSC service thread 1-8) JBAS015899: JBoss Infinispan Server 7.2.0-SNAPSHOT (WildFly 8.1.0.Final) starting
> 09:38:26,332 INFO [org.jboss.as.server] (Controller Boot Thread) JBAS015888: Creating http management service using socket-binding (management-http)
> 09:38:26,355 INFO [org.xnio] (MSC service thread 1-1) XNIO version 3.2.2.Final
> 09:38:26,371 INFO [org.xnio.nio] (MSC service thread 1-1) XNIO NIO Implementation Version 3.2.2.Final
> 09:38:26,424 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 21) JBAS010280: Activating Infinispan subsystem.
> 09:38:26,448 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 22) WFLYIO001: Worker 'default' has auto-configured to 8 core threads with 64 task threads based on your 4 available processors
> 09:38:26,464 INFO [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 25) JBAS010260: Activating JGroups subsystem.
> 09:38:26,476 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 28) JBAS011800: Activating Naming Subsystem
> 09:38:26,479 INFO [org.jboss.remoting] (MSC service thread 1-1) JBoss Remoting version 4.0.3.Final
> 09:38:26,493 INFO [org.jboss.as.security] (ServerService Thread Pool -- 30) JBAS013171: Activating Security Subsystem
> 09:38:26,497 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 32) JBAS010153: Node identifier property is set to the default value. Please make sure it is unique.
> 09:38:26,505 INFO [org.jboss.as.security] (MSC service thread 1-3) JBAS013170: Current PicketBox version=4.0.21.Beta1
> 09:38:26,533 INFO [org.jboss.as.connector.logging] (MSC service thread 1-8) JBAS010408: Starting JCA Subsystem (IronJacamar 1.1.5.Final)
> 09:38:26,577 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) JBAS017502: Undertow 1.0.15.Final starting
> 09:38:26,579 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 33) JBAS017502: Undertow 1.0.15.Final starting
> 09:38:26,629 INFO [org.jboss.as.naming] (MSC service thread 1-6) JBAS011802: Starting Naming Service
> 09:38:26,889 INFO [org.wildfly.extension.undertow] (MSC service thread 1-6) JBAS017525: Started server default-server.
> 09:38:26,901 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) JBAS017531: Host default-host starting
> 09:38:27,015 INFO [org.wildfly.extension.undertow] (MSC service thread 1-5) JBAS017519: Undertow HTTP listener default listening on /127.0.0.1:8180
> 09:38:27,032 WARN [org.jgroups.stack.Configurator] (MSC service thread 1-3) JGRP000014: TP.singleton_name has been deprecated: Use fork channels instead
> 09:38:27,266 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-8) JBAS015012: Started FileSystemDeploymentService for directory /home/afield/Development/projects/infinispan-public/server/integration/build/target/infinispan-server-7.2.0-SNAPSHOT/standalone/deployments
> 09:38:27,632 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) JDGS010000: REST starting
> 09:38:28,049 INFO [org.hibernate.validator.internal.util.Version] (MSC service thread 1-7) HV000001: Hibernate Validator 5.1.0.Final
> 09:38:28,075 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000078: Starting JGroups channel clustered
> 09:38:28,111 WARN [org.jgroups.protocols.UDP] (MSC service thread 1-8) JGRP000015: the send buffer of socket DatagramSocket was set to 1MB, but the OS only allocated 212.99KB. This might lead to performance problems. Please set your max send buffer in the OS correctly (e.g. net.core.wmem_max on Linux)
> 09:38:28,114 WARN [org.jgroups.protocols.UDP] (MSC service thread 1-8) JGRP000015: the receive buffer of socket DatagramSocket was set to 20MB, but the OS only allocated 212.99KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)
> 09:38:28,117 WARN [org.jgroups.protocols.UDP] (MSC service thread 1-8) JGRP000015: the send buffer of socket MulticastSocket was set to 1MB, but the OS only allocated 212.99KB. This might lead to performance problems. Please set your max send buffer in the OS correctly (e.g. net.core.wmem_max on Linux)
> 09:38:28,119 WARN [org.jgroups.protocols.UDP] (MSC service thread 1-8) JGRP000015: the receive buffer of socket MulticastSocket was set to 25MB, but the OS only allocated 212.99KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)
> 09:38:28,127 INFO [org.jboss.resteasy.plugins.validation.AbstractValidatorContextResolver] (MSC service thread 1-7) Unable to find CDI supporting ValidatorFactory. Using default ValidatorFactory
> 09:38:28,231 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) JBAS017534: Registered web context: /
> 09:38:28,232 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) JDGS010002: REST mapped to /rest
> 09:38:28,285 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000094: Received new cluster view for channel clustered: [node0/clustered|1] (2) [node0/clustered, node1/clustered]
> 09:38:28,395 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000079: Channel clustered local address is node1/clustered, physical addresses are [127.0.0.1:55300]
> 09:38:28,486 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-8) ISPN000128: Infinispan version: Infinispan 'Hoptimus Prime' 7.2.0-SNAPSHOT
> 09:38:29,017 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-8) JBAS010281: Started ___protobuf_metadata cache from clustered container
> 09:38:29,046 INFO [org.infinispan.eviction.impl.EvictionManagerImpl] (MSC service thread 1-6) ISPN000025: wakeUpInterval is <= 0, not starting expired purge thread
> 09:38:29,038 INFO [org.infinispan.eviction.impl.EvictionManagerImpl] (MSC service thread 1-3) ISPN000025: wakeUpInterval is <= 0, not starting expired purge thread
> 09:38:29,067 INFO [org.infinispan.eviction.impl.EvictionManagerImpl] (MSC service thread 1-2) ISPN000025: wakeUpInterval is <= 0, not starting expired purge thread
> 09:38:29,073 INFO [org.infinispan.eviction.impl.EvictionManagerImpl] (MSC service thread 1-8) ISPN000025: wakeUpInterval is <= 0, not starting expired purge thread
> 09:39:29,145 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-3) MSC000001: Failed to start service jboss.infinispan.clustered.namedCache: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.namedCache: Failed to start service
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_31]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_31]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_31]
> Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:171)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:218)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:814)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:591)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:546)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:423)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:437)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:116)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:113)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:60)
> at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:121)
> at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:79)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> ... 3 more
> Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache namedCache on node1/clustered
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:219)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_31]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_31]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_31]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0_31]
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> ... 23 more
> 09:39:29,153 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-6) MSC000001: Failed to start service jboss.infinispan.clustered.transactionalCache: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.transactionalCache: Failed to start service
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_31]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_31]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_31]
> Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:171)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:218)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:814)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:591)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:546)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:423)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:437)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:116)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:113)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:60)
> at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:121)
> at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:79)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> ... 3 more
> Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache transactionalCache on node1/clustered
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:219)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_31]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_31]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_31]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0_31]
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> ... 23 more
> 09:39:29,151 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-8) MSC000001: Failed to start service jboss.infinispan.clustered.memcachedCache: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.memcachedCache: Failed to start service
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_31]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_31]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_31]
> Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:171)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:218)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:814)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:591)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:546)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:423)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:437)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:116)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:113)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:60)
> at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:121)
> at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:79)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> ... 3 more
> Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache memcachedCache on node1/clustered
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:219)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_31]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_31]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_31]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0_31]
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> ... 23 more
> 09:39:29,150 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-2) MSC000001: Failed to start service jboss.infinispan.clustered.default: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.default: Failed to start service
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_31]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_31]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_31]
> Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:171)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:218)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:814)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:591)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:546)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:423)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:437)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:116)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:113)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:60)
> at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:121)
> at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:79)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> ... 3 more
> Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache default on node1/clustered
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:219)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_31]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_31]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_31]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0_31]
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> ... 23 more
> 09:39:29,166 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([
> ("subsystem" => "infinispan"),
> ("cache-container" => "clustered"),
> ("distributed-cache" => "default")
> ]) - failure description: {"JBAS014671: Failed services" => {"jboss.infinispan.clustered.default" => "org.jboss.msc.service.StartException in service jboss.infinispan.clustered.default: Failed to start service
> Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache default on node1/clustered"}}
> 09:39:29,169 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([
> ("subsystem" => "infinispan"),
> ("cache-container" => "clustered"),
> ("distributed-cache" => "memcachedCache")
> ]) - failure description: {"JBAS014671: Failed services" => {"jboss.infinispan.clustered.memcachedCache" => "org.jboss.msc.service.StartException in service jboss.infinispan.clustered.memcachedCache: Failed to start service
> Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache memcachedCache on node1/clustered"}}
> 09:39:29,172 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([
> ("subsystem" => "infinispan"),
> ("cache-container" => "clustered"),
> ("distributed-cache" => "namedCache")
> ]) - failure description: {"JBAS014671: Failed services" => {"jboss.infinispan.clustered.namedCache" => "org.jboss.msc.service.StartException in service jboss.infinispan.clustered.namedCache: Failed to start service
> Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache namedCache on node1/clustered"}}
> 09:39:29,174 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([
> ("subsystem" => "infinispan"),
> ("cache-container" => "clustered"),
> ("distributed-cache" => "transactionalCache")
> ]) - failure description: {"JBAS014671: Failed services" => {"jboss.infinispan.clustered.transactionalCache" => "org.jboss.msc.service.StartException in service jboss.infinispan.clustered.transactionalCache: Failed to start service
> Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache transactionalCache on node1/clustered"}}
> 09:39:29,211 INFO [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
> JBAS014777: Services which failed to start: service jboss.infinispan.clustered.transactionalCache: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.transactionalCache: Failed to start service
> service jboss.infinispan.clustered.default: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.default: Failed to start service
> service jboss.infinispan.clustered.namedCache: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.namedCache: Failed to start service
> service jboss.infinispan.clustered.memcachedCache: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.memcachedCache: Failed to start service
> 09:39:29,342 INFO [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:10090/management
> 09:39:29,343 INFO [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://127.0.0.1:10090
> 09:39:29,343 ERROR [org.jboss.as] (Controller Boot Thread) JBAS015875: JBoss Infinispan Server 7.2.0-SNAPSHOT (WildFly 8.1.0.Final) started (with errors) in 65043ms - Started 147 of 175 services (7 services failed or missing dependencies, 64 services are lazy, passive or on-demand)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 10 months
[JBoss JIRA] (ISPN-3819) Test ClusteredCacheWithLongIndexNameTest.testAdditionOfNewNode fails randomly on RHEL
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3819?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-3819:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1040421|https://bugzilla.redhat.com/show_bug.cgi?id=1040421] from VERIFIED to CLOSED
> Test ClusteredCacheWithLongIndexNameTest.testAdditionOfNewNode fails randomly on RHEL
> -------------------------------------------------------------------------------------
>
> Key: ISPN-3819
> URL: https://issues.jboss.org/browse/ISPN-3819
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 6.0.0.Final
> Reporter: Vitalii Chepeliuk
> Assignee: Gustavo Fernandes
> Labels: testsuite_stability
> Fix For: 7.0.0.Beta1
>
>
> Description of failure please look in following jobs with different JDKs
> {noformat}
> org.infinispan.commons.CacheException: Could not prepare.
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:46)
> at org.infinispan.transaction.tm.DummyTransaction.notifyBeforeCompletion(DummyTransaction.java:210)
> at org.infinispan.transaction.tm.DummyTransaction.runPrepare(DummyTransaction.java:224)
> at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:64)
> at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:80)
> at org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1532)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1489)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:960)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:952)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:403)
> at org.infinispan.registry.impl.ClusterRegistryImpl.put(ClusterRegistryImpl.java:69)
> at org.infinispan.query.backend.ReadIntensiveClusterRegistryWrapper.put(ReadIntensiveClusterRegistryWrapper.java:66)
> at org.infinispan.query.backend.QueryInterceptor.enableClassesIncrementally(QueryInterceptor.java:278)
> at org.infinispan.query.backend.QueryInterceptor.updateKnownTypesIfNeeded(QueryInterceptor.java:313)
> at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:518)
> at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:161)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:46)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:166)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:95)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:134)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:111)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:74)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:39)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:95)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.access$000(InboundInvocationHandlerImpl.java:50)
> at org.infinispan.remoting.InboundInvocationHandlerImpl$2.run(InboundInvocationHandlerImpl.java:181)
> at org.infinispan.remoting.InboundInvocationHandlerImpl$2.run(InboundInvocationHandlerImpl.java:181)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: javax.transaction.xa.XAException
> at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:141)
> at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:104)
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:44)
> ... 49 more
> Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from ClusteredCacheWithLongIndexNameTest-NodeA-60170, see cause for remote stack trace
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:41)
> at org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:66)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:561)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:281)
> at org.infinispan.interceptors.distribution.TxDistributionInterceptor.prepareOnAffectedNodes(TxDistributionInterceptor.java:213)
> at org.infinispan.interceptors.distribution.TxDistributionInterceptor.visitPrepareCommand(TxDistributionInterceptor.java:197)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:109)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:78)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:87)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:36)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:124)
> at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:111)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:97)
> at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitPrepareCommand(TransactionSynchronizerInterceptor.java:42)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:217)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:147)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPrepareCommand(StateTransferInterceptor.java:75)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:97)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:111)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:74)
> at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:97)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:121)
> ... 51 more
> Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key org.infinispan.registry.ScopedKey@21631eb7 and requestor GlobalTransaction:<ClusteredCacheWithLongIndexNameTest-NodeB-51377>:8589:remote. Lock is held by GlobalTransaction:<ClusteredCacheWithLongIndexNameTest-NodeA-60170>:8588:local, while request came from ClusteredCacheWithLongIndexNameTest-NodeB-51377
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:169)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:98)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.lockAndRecord(OptimisticLockingInterceptor.java:211)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitSingleKeyCommand(OptimisticLockingInterceptor.java:206)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:199)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.acquireLocksVisitingCommands(OptimisticLockingInterceptor.java:270)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:75)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:36)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:124)
> at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:111)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:97)
> at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitPrepareCommand(TransactionSynchronizerInterceptor.java:39)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:217)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:147)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPrepareCommand(StateTransferInterceptor.java:75)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:97)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:111)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:74)
> at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:97)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:58)
> at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:97)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.commands.tx.PrepareCommand.perform(PrepareCommand.java:114)
> ... 6 more
> [testng-ClusteredCacheWithLongIndexNameTest] Test testAdditionOfNewNode(org.infinispan.query.blackbox.ClusteredCacheWithLongIndexNameTest) failed.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 10 months