[JBoss JIRA] (ISPN-5460) Prepare commands sent before the target became a member should be rejected
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5460?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5460:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1221164|https://bugzilla.redhat.com/show_bug.cgi?id=1221164] from MODIFIED to ON_QA
> Prepare commands sent before the target became a member should be rejected
> --------------------------------------------------------------------------
>
> Key: ISPN-5460
> URL: https://issues.jboss.org/browse/ISPN-5460
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Labels: testsuite_stability
> Fix For: 7.2.2.Final, 8.0.0.Alpha1
>
>
> Since ISPN-4198 was fixed, joiners ignore commands that were sent in a topology in which they were not members. But the joiner will still send a {{null}} response, which is valid, so the tx originator will not retry the command.
> The owner may have already sent the list of transactions to the joiner before the prepare, so this sequence of events can happen:
> 1. A new topology is installed, which includes the joiner (B) in the write consistent hash. B requests the transactions from A, but doesn't receive anything because the affected keys of {{GlobalTransaction:<NodeA-40680>:99974:local}} haven't been updated yet.
> {noformat}
> 19:14:11,027 TRACE (remote-thread-NodeB-p35288-t2:) [StateConsumerImpl] Received new topology for cache resultCache2, isRebalance = false, isMember = false, topology = CacheTopology{id=0, rebalanceId=0, currentCH=ReplicatedConsistentHash{ns = 1, owners = (1)[NodeA-40680: 1]}, pendingCH=null, unionCH=null, actualMembers=[NodeA-40680]}
> 19:14:11,039 TRACE (transport-thread-NodeA-p35281-t6:) [StateTransferInterceptor] handleTxCommand for command PrepareCommand {modifications=[PutKeyValueCommand{key=rules, value=1, flags=null, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}], onePhaseCommit=false, gtx=GlobalTransaction:<NodeA-40680>:99974:local, cacheName='resultCache2', topologyId=-1}, origin null
> 19:14:11,044 TRACE (transport-thread-NodeA-p35281-t1:) [StateConsumerImpl] Received new topology for cache resultCache2, isRebalance = true, isMember = true, topology = CacheTopology{id=1, rebalanceId=1, currentCH=ReplicatedConsistentHash{ns = 1, owners = (1)[NodeA-40680: 1]}, pendingCH=ReplicatedConsistentHash{ns = 1, owners = (2)[NodeA-40680: 1, NodeB-64486: 0]}, unionCH=ReplicatedConsistentHash{ns = 1, owners = (2)[NodeA-40680: 1, NodeB-64486: 0]}, actualMembers=[NodeA-40680, NodeB-64486]}
> 19:14:11,045 TRACE (remote-thread-NodeA-p35282-t4:) [StateProviderImpl] Received request for transactions from node NodeB-64486 for segments [0] of cache resultCache2 with topology id 1
> 19:14:11,045 TRACE (remote-thread-NodeA-p35282-t4:) [StateProviderImpl] Skipping transaction NodeB-64486 because the state requestor doesn't own any key
> 19:14:11,045 TRACE (remote-thread-NodeA-p35282-t4:) [CommandAwareRpcDispatcher] About to send back response SuccessfulResponse{responseValue=[]} for command StateRequestCommand{cache=resultCache2, origin=NodeB-64486, type=GET_TRANSACTIONS, topologyId=1, segments=[0]}
> 19:14:11,045 DEBUG (transport-thread-NodeB-p35287-t3:) [StateConsumerImpl] Applying 0 transactions for cache resultCache2 transferred from node NodeA-40680
> {noformat}
> 2. Node A adds the affected key and sends the prepare command to B with topology 0. B ignores the command.
> {noformat}
> 19:14:11,039 TRACE (transport-thread-NodeA-p35281-t6:) [AbstractCacheTransaction] Registering locked key: rules
> 19:14:11,047 TRACE (transport-thread-NodeA-p35281-t6:) [JGroupsTransport] dests=null, command=PrepareCommand {modifications=[PutKeyValueCommand{key=rules, value=1, flags=null, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}], onePhaseCommit=false, gtx=GlobalTransaction:<NodeA-40680>:99974:local, cacheName='resultCache2', topologyId=0}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=15000
> 19:14:11,048 TRACE (OOB-4,NodeB-64486:) [GlobalInboundInvocationHandler] Attempting to execute CacheRpcCommand: PrepareCommand {modifications=[PutKeyValueCommand{key=rules, value=1, flags=null, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}], onePhaseCommit=false, gtx=GlobalTransaction:<NodeA-40680>:99974:local, cacheName='resultCache2', topologyId=0} [sender=NodeA-40680]
> 19:14:11,048 TRACE (remote-thread-NodeB-p35288-t6:) [NonTotalOrderPerCacheInboundInvocationHandler] Ignoring command sent before the local node was a member (command topology id is 0)
> 19:14:11,048 TRACE (remote-thread-NodeB-p35288-t6:) [CommandAwareRpcDispatcher] About to send back response null for command PrepareCommand {modifications=[PutKeyValueCommand{key=rules, value=1, flags=null, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}], onePhaseCommit=false, gtx=GlobalTransaction:<NodeA-40680>:99974:remote, cacheName='resultCache2', topologyId=0}
> 19:14:11,058 TRACE (transport-thread-NodeA-p35281-t6:) [RpcManagerImpl] Response(s) to PrepareCommand {modifications=[PutKeyValueCommand{key=rules, value=1, flags=null, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}], onePhaseCommit=false, gtx=GlobalTransaction:<NodeA-40680>:99974:local, cacheName='resultCache2', topologyId=0} is {}
> {noformat}
> 3. A tries to commit the tx, but B throws an exception because it can't find the remote tx.
> {noformat}
> 19:14:11,066 TRACE (transport-thread-NodeA-p35281-t6:) [LocalTransaction] Adding remote locks on [NodeA-40680, NodeB-64486]. Remote locks are null
> 19:14:11,066 TRACE (transport-thread-NodeA-p35281-t6:) [TransactionCoordinator] Committing transaction GlobalTransaction:<NodeA-40680>:99974:local
> 19:14:11,066 TRACE (transport-thread-NodeA-p35281-t6:) [RpcManagerImpl] NodeA-40680 invoking CommitCommand {gtx=GlobalTransaction:<NodeA-40680>:99974:local, cacheName='resultCache2', topologyId=2} to recipient list null with options RpcOptions{timeout=15000, unit=MILLISECONDS, deliverOrder=NONE, responseFilter=null, responseMode=SYNCHRONOUS_IGNORE_LEAVERS, skipReplicationQueue=false}
> 19:14:11,066 TRACE (OOB-5,NodeB-64486:) [GlobalInboundInvocationHandler] Attempting to execute CacheRpcCommand: CommitCommand {gtx=GlobalTransaction:<NodeA-40680>:99974:local, cacheName='resultCache2', topologyId=2} [sender=NodeA-40680]
> 19:14:11,066 TRACE (remote-thread-NodeB-p35288-t6:) [NonTotalOrderPerCacheInboundInvocationHandler] Calling perform() on CommitCommand {gtx=GlobalTransaction:<NodeA-40680>:99974:remote, cacheName='resultCache2', topologyId=2}
> 19:14:11,066 TRACE (remote-thread-NodeB-p35288-t6:) [AbstractTransactionBoundaryCommand] Did not find a RemoteTransaction for GlobalTransaction:<NodeA-40680>:99974:remote
> 19:14:11,067 WARN (remote-thread-NodeB-p35288-t6:) [NonTotalOrderPerCacheInboundInvocationHandler] ISPN000071: Caught exception when handling command CommitCommand {gtx=GlobalTransaction:<NodeA-40680>:99974:remote, cacheName='resultCache2', topologyId=2}
> java.lang.IllegalStateException: Remote transaction not found: GlobalTransaction:<NodeA-40680>:99974:remote
> at org.infinispan.commands.tx.CommitCommand.invalidRemoteTxReturnValue(CommitCommand.java:54)
> at org.infinispan.commands.tx.AbstractTransactionBoundaryCommand.perform(AbstractTransactionBoundaryCommand.java:89)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:84)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:31)
> {noformat}
> This is causing random failures in the map/reduce tests (e.g. DistributedSharedCacheTwoNodesMapReduceTest), because map/reduce doesn't wait for all the nodes to join before inserting in the cache.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5221) Java Hotrod client, nearcache broken after a org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5221?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5221:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1206590|https://bugzilla.redhat.com/show_bug.cgi?id=1206590] from MODIFIED to ON_QA
> Java Hotrod client, nearcache broken after a org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-5221
> URL: https://issues.jboss.org/browse/ISPN-5221
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 7.1.0.Final
> Environment: Infinispan 7.1.0Final
> Reporter: Enrico Olivelli
> Assignee: Galder Zamarreño
> Fix For: 7.2.0.Beta1, 7.2.0.Final
>
>
> Using the LAZY nearcache (new in 7.1.0Final) when a java.net.SocketTimeoutException occurs against one of the given HotRod server the RemoteCache becames not usable any more
> This happens very frequently in our DEV environment, but I cannot reproduce it inside a test case
> This is the stacktrace
> {code}
> 15/02/13 14:51:39 ERROR event.ClientListenerNotifier: ISPN004043: Unrecoverable error reading event from server xxx.xxx.xxx/10.168.10.117:11222, exiting event reader thread
> org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:184)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readMagic(Codec20.java:282)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readEvent(Codec20.java:126)
> at org.infinispan.client.hotrod.event.ClientListenerNotifier$EventDispatcher.run(ClientListenerNotifier.java:236)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.SocketTimeoutException
> at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:211)
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:179)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5443) NPE in ReadCommittedEntry.getLifespan() during Cache.size()
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5443?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5443:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1219573|https://bugzilla.redhat.com/show_bug.cgi?id=1219573] from MODIFIED to ON_QA
> NPE in ReadCommittedEntry.getLifespan() during Cache.size()
> -----------------------------------------------------------
>
> Key: ISPN-5443
> URL: https://issues.jboss.org/browse/ISPN-5443
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.2.0.Final
> Reporter: Paul Ferraro
> Assignee: William Burns
> Priority: Blocker
> Fix For: 7.2.1.Final, 8.0.0.Alpha1
>
>
> Infinispan 7.2.0 causes one of our 2LC cache tests to fail consistently. An entity is removed from the 2LC followed by printing of cache statistics, all within the context of a transaction. Here is the exception.
> {noformat}
> ERROR [org.jboss.as.ejb3.invocation] (default task-23) WFLYEJB0034: EJB Invocation failed on component StatefulBean for method public abstract void org.jboss.as.test.clustering.cluster.ejb.xpc.bean.Stateful.deleteEmployee(int): javax.ejb.EJBException: java.lang.NullPointerException
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:187)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:277)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
> at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:634)
> at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
> at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)
> at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)
> at org.jboss.as.test.clustering.cluster.ejb.xpc.bean.Stateful$$$view3.deleteEmployee(Unknown Source)
> at org.jboss.as.test.clustering.cluster.ejb.xpc.servlet.StatefulServlet.doGet(StatefulServlet.java:155)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)
> at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:274)
> at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:253)
> at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.infinispan.container.entries.ReadCommittedEntry.getLifespan(ReadCommittedEntry.java:99)
> at org.infinispan.container.InternalEntryFactoryImpl.create(InternalEntryFactoryImpl.java:53)
> at org.infinispan.iteration.impl.TransactionAwareCloseableIterator.getNextFromIterator(TransactionAwareCloseableIterator.java:119)
> at org.infinispan.iteration.impl.TransactionAwareCloseableIterator.<init>(TransactionAwareCloseableIterator.java:54)
> at org.infinispan.iteration.impl.TransactionAwareCloseableIterable.iterator(TransactionAwareCloseableIterable.java:44)
> at org.infinispan.iteration.impl.TransactionAwareCloseableIterable.iterator(TransactionAwareCloseableIterable.java:19)
> at org.infinispan.commands.read.SizeCommand.perform(SizeCommand.java:47)
> at org.infinispan.commands.read.SizeCommand.perform(SizeCommand.java:25)
> at org.infinispan.interceptors.CallInterceptor.handleDefault(CallInterceptor.java:124)
> at org.infinispan.commands.AbstractVisitor.visitSizeCommand(AbstractVisitor.java:81)
> at org.infinispan.commands.read.SizeCommand.acceptVisitor(SizeCommand.java:35)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> at org.infinispan.commands.AbstractVisitor.visitSizeCommand(AbstractVisitor.java:81)
> at org.infinispan.commands.read.SizeCommand.acceptVisitor(SizeCommand.java:35)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> at org.infinispan.commands.AbstractVisitor.visitSizeCommand(AbstractVisitor.java:81)
> at org.infinispan.commands.read.SizeCommand.acceptVisitor(SizeCommand.java:35)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> at org.infinispan.commands.AbstractVisitor.visitSizeCommand(AbstractVisitor.java:81)
> at org.infinispan.commands.read.SizeCommand.acceptVisitor(SizeCommand.java:35)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> at org.infinispan.commands.AbstractVisitor.visitSizeCommand(AbstractVisitor.java:81)
> at org.infinispan.commands.read.SizeCommand.acceptVisitor(SizeCommand.java:35)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:346)
> at org.infinispan.interceptors.TxInterceptor.visitSizeCommand(TxInterceptor.java:293)
> at org.infinispan.commands.read.SizeCommand.acceptVisitor(SizeCommand.java:35)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> at org.infinispan.commands.AbstractVisitor.visitSizeCommand(AbstractVisitor.java:81)
> at org.infinispan.commands.read.SizeCommand.acceptVisitor(SizeCommand.java:35)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitSizeCommand(AbstractVisitor.java:81)
> at org.infinispan.commands.read.SizeCommand.acceptVisitor(SizeCommand.java:35)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.cache.impl.CacheImpl.size(CacheImpl.java:374)
> at org.infinispan.cache.impl.CacheImpl.size(CacheImpl.java:369)
> at org.infinispan.cache.impl.AbstractDelegatingCache.size(AbstractDelegatingCache.java:267)
> at org.hibernate.cache.infinispan.impl.BaseRegion.getElementCountInMemory(BaseRegion.java:100)
> at org.hibernate.stat.internal.ConcurrentSecondLevelCacheStatisticsImpl.getElementCountInMemory(ConcurrentSecondLevelCacheStatisticsImpl.java:64)
> at org.hibernate.stat.internal.ConcurrentSecondLevelCacheStatisticsImpl.toString(ConcurrentSecondLevelCacheStatisticsImpl.java:93)
> at org.jboss.as.test.clustering.cluster.ejb.xpc.bean.StatefulBean.logStats(StatefulBean.java:172)
> at org.jboss.as.test.clustering.cluster.ejb.xpc.bean.StatefulBean.deleteEmployee(StatefulBean.java:106)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
> at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:82)
> at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93)
> at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.jpa.interceptor.SFSBInvocationInterceptor.processInvocation(SFSBInvocationInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.processInvocation(StatefulSessionSynchronizationInterceptor.java:127)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
> at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
> at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.stateful.StatefulComponentInstanceInterceptor.processInvocation(StatefulComponentInstanceInterceptor.java:65)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275)
> ... 63 more
> {noformat}
> Now I know that hibernate-infinispan should not be using a cluster-wide size() operation here (see https://hibernate.atlassian.net/browse/HHH-9776), but there is still an urgent issue here that need to be addressed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5274) Inconsistent data after transaction rollback (with success on originator)
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5274?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5274:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1199490|https://bugzilla.redhat.com/show_bug.cgi?id=1199490] from MODIFIED to ON_QA
> Inconsistent data after transaction rollback (with success on originator)
> -------------------------------------------------------------------------
>
> Key: ISPN-5274
> URL: https://issues.jboss.org/browse/ISPN-5274
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.1.1.Final
> Reporter: Matej Čimbora
> Assignee: Dan Berindei
> Fix For: 7.2.0.Final
>
>
> Scenario
> Nodes edg-perf[10-13], partition handling on
> 1.Transaction is started on edg-perf10
> {code}
> 10:24:48,228 TRACE [org.infinispan.transaction.xa.TransactionXaAdapter] (DefaultStressor-6) start called on tx GlobalTransaction:<edg-perf10-20667>:38910:local
> {code}
> 2. Value of key_000000000000065B is updated within the transaction
> {code}
> 10:24:48,405 TRACE [org.radargun.service.InfinispanOperations$Cache] (DefaultStressor-6) PUT cache=testCache key=key_000000000000065B value=[6 #15: 296, 501, 1109, 1119, 1459, 1544, 1999, 2083, 2130, 2257, 2298, 2784, 2941, 3009, 3147, ]
> {code}
> 3. Transaction is successfully prepared on edg-perf11 & edg-perf12
> {code}
> 10:24:48,559 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (DefaultStressor-6) Responses: [sender=edg-perf11-61837, received=true, suspected=false]
> [sender=edg-perf12-7305, received=true, suspected=false]
> {code}
> 4. Transaction commit is issued
> {code}
> 10:24:48,562 TRACE [org.infinispan.transaction.TransactionCoordinator] (DefaultStressor-6) Committing transaction GlobalTransaction:<edg-perf10-20667>:38910:local
> {code}
> 5. Other participating nodes (edg-perf11 & edg-perf12) are suspected...
> {code}
> 10:24:52,705 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (DefaultStressor-6) Target node edg-perf11-61837 left during remote call, ignoring
> 10:24:52,716 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (DefaultStressor-6) Target node edg-perf12-7305 left during remote call, ignoring
> {code}
> ... as they received a new view (without edg-perf10) meanwhile.
> {code}
> 10:24:48,547 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,edg-perf12-7305) ISPN000093: Received new, MERGED cluster view: MergeView::[edg-perf12-7305|20] (3) [edg-perf12-7305, edg-perf11-61837, edg-perf13-25187], 2 subgroups: [edg-perf13-25187|8] (1) [edg-perf13-25187], [edg-perf13-25187|19] (1) [edg-perf13-25187]
> {code}
> Still, the transaction is commited on edg-perf10 & updated entry is stored locally
> {code}
> 10:24:52,894 TRACE [org.infinispan.statetransfer.CommitManager] (DefaultStressor-6) Trying to commit. Key=key_000000000000065B. Operation Flag=null, L1 invalidation=false
> 10:24:52,896 TRACE [org.infinispan.statetransfer.CommitManager] (DefaultStressor-6) Committing key=key_000000000000065B. It is a L1 invalidation or a normal put and no tracking is enabled!
> 10:24:52,908 TRACE [org.infinispan.container.DefaultDataContainer] (DefaultStressor-6) Creating new ICE for writing. Existing=ImmortalCacheEntry{key=key_000000000000065B, value=[6 #14: 296, 501, 1109, 1119, 1459, 1544, 1999, 2083, 2130, 2257, 2298, 2784, 2941, 3009, ]}, metadata=EmbeddedMetadata{version=null}, new value=[6 #15: 296, 501, 1109, 1119, 1459, 1544, 1999, 2083, 2130, 2257, 2298, 2784, 2941, 3009, 3147, ]
> {code}
> 6. Other nodes rollback the transaction
> {code}
> 10:24:50,376 DEBUG [org.infinispan.transaction.TransactionTable] (transport-thread-10) Rolling back transaction GlobalTransaction:<edg-perf10-20667>:38910:remote because originator edg-perf10-20667 left the cluster
> {code}
> 7. edg-perf10 receives a new view, containing nodes edg-perf[10,11,13]. Incoming state transfer overwrites the updated value
> {code}
> 10:25:09,614 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,edg-perf10-20667) ISPN000093: Received new, MERGED cluster view: MergeView::[edg-perf10-20667|22] (3) [edg-perf10-20667, edg-perf11-61837, edg-perf13-25187], 4 subgroups: [edg-perf10-20667|15] (2) [edg-perf10-20667, edg-perf11-61837], [edg-perf10-20667|19] (3) [edg-perf10-20667, edg-perf12-7305, edg-perf11-61837], [edg-perf10-20667|18] (1) [edg-perf10-20667], [edg-perf10-20667|21] (2) [edg-perf10-20667, edg-perf13-25187]
> {code}
> 8. get operation returns outdated value
> {code}
> 10:26:21,020 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (DefaultStressor-6) Response(s) to ClusteredGetCommand{key=key_000000000000065B, flags=null} is {edg-perf12-7305=SuccessfulResponse{responseValue=ImmortalCacheValue {value=[6 #14: 296, 501, 1109, 1119, 1459, 1544, 1999, 2083, 2130, 2257, 2298, 2784, 2941, 3009, ]}} }
> {code}
> From client perspective, this behavior is not transparent. Provided the transaction ended up successfully, presence of the updated entry can be assumed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-4503) Commands with topology id 0 are not properly ignored on joiners
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4503?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4503:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1220328|https://bugzilla.redhat.com/show_bug.cgi?id=1220328] from MODIFIED to ON_QA
> Commands with topology id 0 are not properly ignored on joiners
> ---------------------------------------------------------------
>
> Key: ISPN-4503
> URL: https://issues.jboss.org/browse/ISPN-4503
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 7.0.0.Beta1
>
>
> InboundInvocationHandlerImpl is supposed to ignore commands sent with a topology id smaller than the first topology id in which the local node was a member. But there is a loophole when the command was sent with topology id 0.
> This is visible in StateTransferFunctionalTest, where the writing thread keeps the cpu busy and can delay the 2nd node joining for a long time (especially when run on a single core with {{taskset -c 0}}). For some reason, the PrepareCommands are sent only to the local node, while the TxCompletionNotificationCommands are sent to the entire cluster ({{null}}). When the 2nd node manages to join, it receives a lot of TxCompletionNotificationCommands and processing them delays the processing of the rebalance commands. Since the writes eventually block waiting for the new topology to be installed on the joiner, the delayed rebalance commands cause the write to time out.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-4504) Topology id is not properly set on ClusteredGetCommands
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4504?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4504:
-----------------------------------------------
Sebastian Łaskawiec <slaskawi(a)redhat.com> changed the Status of [bug 1220328|https://bugzilla.redhat.com/show_bug.cgi?id=1220328] from MODIFIED to ON_QA
> Topology id is not properly set on ClusteredGetCommands
> -------------------------------------------------------
>
> Key: ISPN-4504
> URL: https://issues.jboss.org/browse/ISPN-4504
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 7.0.0.Beta1
>
>
> Because the topology id is not properly set on ClusteredGetCommands, they don't wait for the sender's topology to be installed on the target.
> I have tried to fix this while implementing a fix for ISPN-4503. My solution caused 2 failures in RemoteGetDuringStateTransferTest, scenarios 5 and 7::
> | sc | currentTopologyId | currentTopologyId + 1 (rebalance) | currentTopologyId + 2 (finish) |
> | 5 | 0:remoteGet | 2:sendReply | 0:receiveReply, 1:sendReply |
> | 7 | | 0:remoteGet, 2: sendReply | 0:receiveReply, 1:sendReply |
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months