[JBoss JIRA] (ISPN-5261) Transaction is recommited by joining node during merge
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5261?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5261:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1198267|https://bugzilla.redhat.com/show_bug.cgi?id=1198267] from POST to MODIFIED
> Transaction is recommited by joining node during merge
> ------------------------------------------------------
>
> Key: ISPN-5261
> URL: https://issues.jboss.org/browse/ISPN-5261
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.1.1.Final
> Reporter: Matej Čimbora
> Assignee: Dan Berindei
> Fix For: 7.2.1.Final
>
>
> Transaction gets recommited after a node joins a cluster during merge and requests retransmission of previous messages, overwriting updates which happened till that point.
> 4 nodes - edg-perf01-04
> Scenario:
> 1. edg-perf01 begins a new transaction & updates value of K
> {code}
> 06:50:50,421 TRACE [org.infinispan.transaction.xa.TransactionXaAdapter] (DefaultStressor-7) start called on tx GlobalTransaction:<edg-perf01-19349>:3516:local
> {code}
> The latest view on edg-perf01 is
> {code}
> 06:50:35,103 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,edg-perf01-19349) ISPN000093: Received new, MERGED cluster view: MergeView::[edg-perf01-19349|5] (3) [edg-perf01-19349, edg-perf04-20753, edg-perf02-19191], 1 subgroups: [edg-perf01-19349|4] (2) [edg-perf01-19349, edg-perf02-19191]
> {code}
> 2. Prepare command is sent with nodes edg-perf02 & edg-perf04 responding successfully
> {code}
> 06:50:51,186 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (DefaultStressor-7) edg-perf01-19349 invoking PrepareCommand ... topologyId=12} to recipient list [edg-perf02-19191, edg-perf04-20753, edg-perf01-19349] ...
> {code}
> check message id & seqno
> {code}
> (06:50:51,195 TRACE [org.jgroups.protocols.TCP] (DefaultStressor-7) edg-perf01-19349: sending msg to null, src=edg-perf01-19349, headers are RequestCorrelator: id=200, type=REQ, id=17134, rsp_expected=true, exclusion_list=[edg-perf01-19349], NAKACK2: [MSG, seqno=1020], 20903: slaveIndex=0, TCP: [cluster_name=default])
> {code}
> 3. Transaction gets commited & value of K gets updated multiple times on edg-perf01 from this point on
> 4. edg-perf03 receives a new view containing all nodes
> {code}
> 06:50:53,963 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,edg-perf03-14171) ISPN000093: Received new, MERGED cluster view: MergeView::[edg-perf01-19349|6] (4) [edg-perf01-19349, edg-perf03-14171, edg-perf04-20753, edg-perf02-19191], 3 subgroups: [edg-perf03-14171|4] (2) [edg-perf03-14171, edg-perf04-20753], [edg-perf01-19349|5] (3) [edg-perf01-19349, edg-perf04-20753, edg-perf02-19191], [edg-perf01-19349|4] (2) [edg-perf01-19349, edg-perf02-19191]
> {code}
> 5. edg-perf03 apparently requests retransmission of previous messages (containing prepare from step#2) & edg-perf01 sends the response
> Request
> {code}
> 06:51:35,241 TRACE [org.jgroups.protocols.pbcast.NAKACK2] (Timer-3,edg-perf03-14171) edg-perf03-14171: sending XMIT_REQ ((79): {1015-1093}) to edg-perf01-19349
> {code}
> Response
> {code}
> 06:51:35,299 TRACE [org.jgroups.protocols.TCP] (INT-14,edg-perf03-14171) edg-perf03-14171: received [dst: edg-perf03-14171, src: edg-perf01-19349 (4 headers), size=975 bytes, flags=OOB|DONT_BUNDLE|NO_TOTAL_ORDER|INTERNAL], headers are RequestCorrelator: id=200, type=REQ, id=17134, rsp_expected=true, exclusion_list=[edg-perf01-19349], NAKACK2: [XMIT_RSP, seqno=1020], 20903: slaveIndex=0, TCP: [cluster_name=default]
> {code}
> 6. edg-perf03 prepares the old transaction & eventually commits it again
> {code}
> 06:51:35,304 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (INT-14,edg-perf03-14171) Attempting to execute command: PrepareCommand.....gtx=GlobalTransaction:<edg-perf01-19349>:3516:local, cacheName='testCache', topologyId=12} [sender=edg-perf01-19349]
> 06:51:35,304 TRACE [org.infinispan.statetransfer.StateTransferLockImpl] (INT-14,edg-perf03-14171) Checking if transaction data was received for topology 12, current topology is 14
> 06:51:35,305 TRACE [org.infinispan.transaction.TransactionTable] (remote-thread-40) Created and registered remote transaction ... tx=GlobalTransaction:<edg-perf01-19349>:3516:remote, ...
> 06:51:35,308 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (remote-thread-40) edg-perf03-14171 invoking PrepareCommand ... gtx=GlobalTransaction:<edg-perf01-19349>:3516:remote, cacheName='testCache', topologyId=15} to recipient list [edg-perf04-20753, edg-perf02-19191] ...
> {code}
> Successful commit responses
> {code}
> 06:51:36,750 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (remote-thread-44) Response(s) to CommitCommand {gtx=GlobalTransaction:<edg-perf01-19349>:3516:remote, cacheName='testCache', topologyId=15} is {edg-perf02-19191=SuccessfulResponse{responseValue=null} , edg-perf04-20753=SuccessfulResponse{responseValue=null} }
> {code}
> 7. When get(K) is invoked on edg-perf01, stale value is returned (the one from step #6, ignoring updates which have happened since step #3)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5274) Inconsistent data after transaction rollback (with success on originator)
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5274?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5274:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1199490|https://bugzilla.redhat.com/show_bug.cgi?id=1199490] from POST to MODIFIED
> Inconsistent data after transaction rollback (with success on originator)
> -------------------------------------------------------------------------
>
> Key: ISPN-5274
> URL: https://issues.jboss.org/browse/ISPN-5274
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.1.1.Final
> Reporter: Matej Čimbora
> Assignee: Dan Berindei
> Fix For: 7.2.0.Final
>
>
> Scenario
> Nodes edg-perf[10-13], partition handling on
> 1.Transaction is started on edg-perf10
> {code}
> 10:24:48,228 TRACE [org.infinispan.transaction.xa.TransactionXaAdapter] (DefaultStressor-6) start called on tx GlobalTransaction:<edg-perf10-20667>:38910:local
> {code}
> 2. Value of key_000000000000065B is updated within the transaction
> {code}
> 10:24:48,405 TRACE [org.radargun.service.InfinispanOperations$Cache] (DefaultStressor-6) PUT cache=testCache key=key_000000000000065B value=[6 #15: 296, 501, 1109, 1119, 1459, 1544, 1999, 2083, 2130, 2257, 2298, 2784, 2941, 3009, 3147, ]
> {code}
> 3. Transaction is successfully prepared on edg-perf11 & edg-perf12
> {code}
> 10:24:48,559 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (DefaultStressor-6) Responses: [sender=edg-perf11-61837, received=true, suspected=false]
> [sender=edg-perf12-7305, received=true, suspected=false]
> {code}
> 4. Transaction commit is issued
> {code}
> 10:24:48,562 TRACE [org.infinispan.transaction.TransactionCoordinator] (DefaultStressor-6) Committing transaction GlobalTransaction:<edg-perf10-20667>:38910:local
> {code}
> 5. Other participating nodes (edg-perf11 & edg-perf12) are suspected...
> {code}
> 10:24:52,705 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (DefaultStressor-6) Target node edg-perf11-61837 left during remote call, ignoring
> 10:24:52,716 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (DefaultStressor-6) Target node edg-perf12-7305 left during remote call, ignoring
> {code}
> ... as they received a new view (without edg-perf10) meanwhile.
> {code}
> 10:24:48,547 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,edg-perf12-7305) ISPN000093: Received new, MERGED cluster view: MergeView::[edg-perf12-7305|20] (3) [edg-perf12-7305, edg-perf11-61837, edg-perf13-25187], 2 subgroups: [edg-perf13-25187|8] (1) [edg-perf13-25187], [edg-perf13-25187|19] (1) [edg-perf13-25187]
> {code}
> Still, the transaction is commited on edg-perf10 & updated entry is stored locally
> {code}
> 10:24:52,894 TRACE [org.infinispan.statetransfer.CommitManager] (DefaultStressor-6) Trying to commit. Key=key_000000000000065B. Operation Flag=null, L1 invalidation=false
> 10:24:52,896 TRACE [org.infinispan.statetransfer.CommitManager] (DefaultStressor-6) Committing key=key_000000000000065B. It is a L1 invalidation or a normal put and no tracking is enabled!
> 10:24:52,908 TRACE [org.infinispan.container.DefaultDataContainer] (DefaultStressor-6) Creating new ICE for writing. Existing=ImmortalCacheEntry{key=key_000000000000065B, value=[6 #14: 296, 501, 1109, 1119, 1459, 1544, 1999, 2083, 2130, 2257, 2298, 2784, 2941, 3009, ]}, metadata=EmbeddedMetadata{version=null}, new value=[6 #15: 296, 501, 1109, 1119, 1459, 1544, 1999, 2083, 2130, 2257, 2298, 2784, 2941, 3009, 3147, ]
> {code}
> 6. Other nodes rollback the transaction
> {code}
> 10:24:50,376 DEBUG [org.infinispan.transaction.TransactionTable] (transport-thread-10) Rolling back transaction GlobalTransaction:<edg-perf10-20667>:38910:remote because originator edg-perf10-20667 left the cluster
> {code}
> 7. edg-perf10 receives a new view, containing nodes edg-perf[10,11,13]. Incoming state transfer overwrites the updated value
> {code}
> 10:25:09,614 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,edg-perf10-20667) ISPN000093: Received new, MERGED cluster view: MergeView::[edg-perf10-20667|22] (3) [edg-perf10-20667, edg-perf11-61837, edg-perf13-25187], 4 subgroups: [edg-perf10-20667|15] (2) [edg-perf10-20667, edg-perf11-61837], [edg-perf10-20667|19] (3) [edg-perf10-20667, edg-perf12-7305, edg-perf11-61837], [edg-perf10-20667|18] (1) [edg-perf10-20667], [edg-perf10-20667|21] (2) [edg-perf10-20667, edg-perf13-25187]
> {code}
> 8. get operation returns outdated value
> {code}
> 10:26:21,020 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (DefaultStressor-6) Response(s) to ClusteredGetCommand{key=key_000000000000065B, flags=null} is {edg-perf12-7305=SuccessfulResponse{responseValue=ImmortalCacheValue {value=[6 #14: 296, 501, 1109, 1119, 1459, 1544, 1999, 2083, 2130, 2257, 2298, 2784, 2941, 3009, ]}} }
> {code}
> From client perspective, this behavior is not transparent. Provided the transaction ended up successfully, presence of the updated entry can be assumed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-4546) Possible stale lock when the primary owner leaves during rebalance
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4546?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4546:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1163727|https://bugzilla.redhat.com/show_bug.cgi?id=1163727] from POST to MODIFIED
> Possible stale lock when the primary owner leaves during rebalance
> ------------------------------------------------------------------
>
> Key: ISPN-4546
> URL: https://issues.jboss.org/browse/ISPN-4546
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.0.0.Alpha5, 7.1.1.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 7.2.0.Final
>
>
> Topology T: coordinator = A, owners(k) = [C, D], pending_owners(k) = null
> B sends prepareCommand(tx1, put(k, v)) to C, D
> D adds backup locks and replies
> C acquires lock, ready to send reply to B
> A starts installing topology T+1: owners(k) = [C, D], pending_owners(k) = [C, E]
> A, C and E install topology T+1, B and D do not
> E requests and receives tx data from C, including tx1
> C leaves
> B sees a SuspectException, sends rollbackCommand(tx1) to C, D
> D removes tx1
> C has left, but is ignored
> B reports to the user that the tx has been rolled back
> B and D install topology T+1 (optional)
> A starts installing topology T+2: owners(k) = [D], pending_owners(k) = [E]
> A, B, D, E all install topology T+2
> E requests and receives state from D, but it does not remove tx1
> A starts installing topology T+3: owners(k) = [E], pending_owners(k) = null
> E now has a stale backup lock on k
> It seems very hard to reproduce in production: C would have to leave soon enough so that B and D haven't received the T+1 topology yet, but late enough for it to send its transaction data to E.
> A possible solution would be to catch any SuspectException during prepare/commit/rollback (without ignoring leavers), wait for a new topology, and replicate the command again on the new owners. Obviously, this wouldn't work with asynchronous prepare/commit/rollback.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-4504) Topology id is not properly set on ClusteredGetCommands
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4504?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4504:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1220328|https://bugzilla.redhat.com/show_bug.cgi?id=1220328] from POST to MODIFIED
> Topology id is not properly set on ClusteredGetCommands
> -------------------------------------------------------
>
> Key: ISPN-4504
> URL: https://issues.jboss.org/browse/ISPN-4504
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 7.0.0.Beta1
>
>
> Because the topology id is not properly set on ClusteredGetCommands, they don't wait for the sender's topology to be installed on the target.
> I have tried to fix this while implementing a fix for ISPN-4503. My solution caused 2 failures in RemoteGetDuringStateTransferTest, scenarios 5 and 7::
> | sc | currentTopologyId | currentTopologyId + 1 (rebalance) | currentTopologyId + 2 (finish) |
> | 5 | 0:remoteGet | 2:sendReply | 0:receiveReply, 1:sendReply |
> | 7 | | 0:remoteGet, 2: sendReply | 0:receiveReply, 1:sendReply |
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-4503) Commands with topology id 0 are not properly ignored on joiners
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4503?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4503:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1220328|https://bugzilla.redhat.com/show_bug.cgi?id=1220328] from POST to MODIFIED
> Commands with topology id 0 are not properly ignored on joiners
> ---------------------------------------------------------------
>
> Key: ISPN-4503
> URL: https://issues.jboss.org/browse/ISPN-4503
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 7.0.0.Beta1
>
>
> InboundInvocationHandlerImpl is supposed to ignore commands sent with a topology id smaller than the first topology id in which the local node was a member. But there is a loophole when the command was sent with topology id 0.
> This is visible in StateTransferFunctionalTest, where the writing thread keeps the cpu busy and can delay the 2nd node joining for a long time (especially when run on a single core with {{taskset -c 0}}). For some reason, the PrepareCommands are sent only to the local node, while the TxCompletionNotificationCommands are sent to the entire cluster ({{null}}). When the 2nd node manages to join, it receives a lot of TxCompletionNotificationCommands and processing them delays the processing of the rebalance commands. Since the writes eventually block waiting for the new topology to be installed on the joiner, the delayed rebalance commands cause the write to time out.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5106) Deadlock on GlobalComponentRegistry when starting a cluster
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5106?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5106:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1217380|https://bugzilla.redhat.com/show_bug.cgi?id=1217380] from POST to MODIFIED
> Deadlock on GlobalComponentRegistry when starting a cluster
> -----------------------------------------------------------
>
> Key: ISPN-5106
> URL: https://issues.jboss.org/browse/ISPN-5106
> Project: Infinispan
> Issue Type: Bug
> Components: Server
> Reporter: Jakub Markos
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 7.2.0.Final
>
> Attachments: dumps_and_logs.zip
>
>
> We have a test which starts 4 server nodes, and sometimes they fail to complete the startup. This happens with the current snapshot.
> It appears there's a deadlock on intrinsic locks on GlobalComponentRegistry, since the CacheTopologyControlCommand.POLICY_GET_STATUS is sent with the lock acquired but this lock is also needed for injecting dependencies when the command is processed on the remote node.
> Here are the relevant parts from the dumps, node02:
> {code}
> "remote-thread--p3-t1" daemon prio=10 tid=0x00007f7a00002800 nid=0x487f waiting for monitor entry [0x00007f796bbfa000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:262)
> - waiting to lock <0x000000060365b6b8> (a org.infinispan.factories.GlobalComponentRegistry)
> at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:227)
> at org.infinispan.factories.AbstractComponentRegistry.wireDependencies(AbstractComponentRegistry.java:132)
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$2.run(GlobalInboundInvocationHandler.java:156)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Locked ownable synchronizers:
> - <0x0000000615af46d0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "MSC service thread 1-16" prio=10 tid=0x00007f79ec071800 nid=0x4839 waiting on condition [0x00007f7a40239000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000614d47e60> (a java.util.concurrent.FutureTask)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:422)
> at java.util.concurrent.FutureTask.get(FutureTask.java:199)
> at org.infinispan.topology.ClusterTopologyManagerImpl.executeOnClusterSync(ClusterTopologyManagerImpl.java:432)
> at org.infinispan.topology.ClusterTopologyManagerImpl.executeOnClusterSync(ClusterTopologyManagerImpl.java:385)
> at org.infinispan.topology.ClusterTopologyManagerImpl.confirmMembersAvailable(ClusterTopologyManagerImpl.java:368)
> at org.infinispan.topology.ClusterTopologyManagerImpl.updateCacheMembers(ClusterTopologyManagerImpl.java:359)
> at org.infinispan.topology.ClusterTopologyManagerImpl.handleClusterView(ClusterTopologyManagerImpl.java:281)
> - locked <0x000000060420d4a8> (a java.lang.Object)
> at org.infinispan.topology.ClusterTopologyManagerImpl.start(ClusterTopologyManagerImpl.java:103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> - locked <0x000000060365b6b8> (a org.infinispan.factories.GlobalComponentRegistry)
> at org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:221)
> - locked <0x000000060365b6b8> (a org.infinispan.factories.GlobalComponentRegistry)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:580)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:546)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:423)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:437)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:116)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:113)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:60)
> at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:121)
> at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:79)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Locked ownable synchronizers:
> - <0x0000000653444750> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> {code}
> and node03
> {code}
> "remote-thread--p3-t1" daemon prio=10 tid=0x00007f016c079000 nid=0x1a43 waiting for monitor entry [0x00007f0114396000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:262)
> - waiting to lock <0x0000000609c2bf50> (a org.infinispan.factories.GlobalComponentRegistry)
> at org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:227)
> at org.infinispan.factories.AbstractComponentRegistry.wireDependencies(AbstractComponentRegistry.java:132)
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$2.run(GlobalInboundInvocationHandler.java:156)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Locked ownable synchronizers:
> - <0x0000000615a05750> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "MSC service thread 1-16" prio=10 tid=0x00007f015c071800 nid=0x19ff waiting on condition [0x00007f01b0558000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000615025bb0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at org.jgroups.util.CondVar.waitFor(CondVar.java:64)
> at org.jgroups.blocks.Request.waitForResults(Request.java:195)
> at org.jgroups.blocks.Request.responsesComplete(Request.java:181)
> at org.jgroups.blocks.Request.execute(Request.java:89)
> at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:409)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:374)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:188)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:562)
> at org.infinispan.topology.ClusterTopologyManagerImpl.start(ClusterTopologyManagerImpl.java:112)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> - locked <0x0000000609c2bf50> (a org.infinispan.factories.GlobalComponentRegistry)
> at org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:221)
> - locked <0x0000000609c2bf50> (a org.infinispan.factories.GlobalComponentRegistry)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:580)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:546)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:423)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:437)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89)
> at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:116)
> at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:113)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:60)
> at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:121)
> at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:79)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Locked ownable synchronizers:
> - <0x00000006534e9628> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5458) Broken modules due to circular dependencies
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-5458?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-5458:
------------------------------------
Priority: Critical (was: Major)
> Broken modules due to circular dependencies
> -------------------------------------------
>
> Key: ISPN-5458
> URL: https://issues.jboss.org/browse/ISPN-5458
> Project: Infinispan
> Issue Type: Bug
> Components: Build process
> Reporter: Gustavo Fernandes
> Priority: Critical
>
> There's still a circular dependency hell between hibernate search 5.2. (depends on infinispan 7.2.0.Final) and Infinispan (depends on hibernate Search 5.2.0.Final), causing failures on master:
> {code}
> Caused by: java.lang.Exception: {"JBAS014671: Failed services" => {"jboss.persistenceunit.\"master.war#pu-master\"" => "org.jboss.msc.service.StartException in service jboss.persistenceunit.\"master.war#pu-master\": org.jboss.modules.ModuleLoadError: org.infinispan:ispn-7.2
> Caused by: org.jboss.modules.ModuleLoadError: org.infinispan:ispn-7.2"}}
> {code}
> There was an alias in place to mitigate that until hibernate search has infinispan dependency, but apparently it was removed on ISPN-5445
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5458) Broken modules due to circular dependencies
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-5458?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-5458:
------------------------------------
Status: Open (was: New)
> Broken modules due to circular dependencies
> -------------------------------------------
>
> Key: ISPN-5458
> URL: https://issues.jboss.org/browse/ISPN-5458
> Project: Infinispan
> Issue Type: Bug
> Components: Build process
> Reporter: Gustavo Fernandes
>
> There's still a circular dependency hell between hibernate search 5.2. (depends on infinispan 7.2.0.Final) and Infinispan (depends on hibernate Search 5.2.0.Final), causing failures on master:
> {code}
> Caused by: java.lang.Exception: {"JBAS014671: Failed services" => {"jboss.persistenceunit.\"master.war#pu-master\"" => "org.jboss.msc.service.StartException in service jboss.persistenceunit.\"master.war#pu-master\": org.jboss.modules.ModuleLoadError: org.infinispan:ispn-7.2
> Caused by: org.jboss.modules.ModuleLoadError: org.infinispan:ispn-7.2"}}
> {code}
> There was an alias in place to mitigate that until hibernate search has infinispan dependency, but apparently it was removed on ISPN-5445
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5458) Broken modules due to circular dependencies
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-5458?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-5458:
------------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/3458
> Broken modules due to circular dependencies
> -------------------------------------------
>
> Key: ISPN-5458
> URL: https://issues.jboss.org/browse/ISPN-5458
> Project: Infinispan
> Issue Type: Bug
> Components: Build process
> Reporter: Gustavo Fernandes
>
> There's still a circular dependency hell between hibernate search 5.2. (depends on infinispan 7.2.0.Final) and Infinispan (depends on hibernate Search 5.2.0.Final), causing failures on master:
> {code}
> Caused by: java.lang.Exception: {"JBAS014671: Failed services" => {"jboss.persistenceunit.\"master.war#pu-master\"" => "org.jboss.msc.service.StartException in service jboss.persistenceunit.\"master.war#pu-master\": org.jboss.modules.ModuleLoadError: org.infinispan:ispn-7.2
> Caused by: org.jboss.modules.ModuleLoadError: org.infinispan:ispn-7.2"}}
> {code}
> There was an alias in place to mitigate that until hibernate search has infinispan dependency, but apparently it was removed on ISPN-5445
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months