[jboss-jira] [JBoss JIRA] (WFLY-4805) Distributable app causes TransferQueueBundler Invalid argument and Topology errors

[Romain Pelisse (JIRA)]

     [ https://issues.jboss.org/browse/WFLY-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Romain Pelisse reassigned WFLY-4805:
------------------------------------

    Assignee: Romain Pelisse  (was: Paul Ferraro)


> Distributable app causes TransferQueueBundler Invalid argument and Topology errors
> ----------------------------------------------------------------------------------
>
>                 Key: WFLY-4805
>                 URL: https://issues.jboss.org/browse/WFLY-4805
>             Project: WildFly
>          Issue Type: Bug
>          Components: Clustering
>    Affects Versions: 10.0.0.Alpha4
>         Environment: Oracle JDK 1.8, Open JDK 1.8, RHEL 7
>            Reporter: Michal Karm Babacek
>            Assignee: Romain Pelisse
>         Attachments: clusterbench.war, configs.zip, logs.zip
>
>
> Simple shutdown failover tests are failing due to apparent error in / misconfiguration of the distributed cache.
> If one follows the aforementioned steps to reproduce, the following symptoms appear:
>  * Variations on {noformat}SEVERE [org.jgroups.protocols.UDP] (TransferQueueBundler,ee,rhel7gax86-64) JGRP000029: rhel7gax86-64: failed sending message to rhel7gax86-64 (59 bytes): java.io.IOException: Invalid argument, headers: UNICAST3: ACK, seqno=9410, ts=217, UDP: [cluster_name=ee]{noformat}
>  * Failures of this kind: {noformat}ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (transport-thread--p2-t12) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 11
>     at org.infinispan.statetransfer.StateTransferLockImpl.waitForTransactionData(StateTransferLockImpl.java:92)
>     at org.infinispan.interceptors.base.BaseStateTransferInterceptor.waitForTransactionData(BaseStateTransferInterceptor.java:96)
>     at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:285)
>     at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:254)
>     at org.infinispan.statetransfer.StateTransferInterceptor.visitRemoveCommand(StateTransferInterceptor.java:130)
>     at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
>     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
>     at org.infinispan.interceptors.CacheMgmtInterceptor.visitRemoveCommand(CacheMgmtInterceptor.java:209)
>     at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
>     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
>     at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
>     at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
>     at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:49)
>     at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
>     at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
>     at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1617)
>     at org.infinispan.cache.impl.CacheImpl.removeInternal(CacheImpl.java:579)
>     at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:572)
>     at org.infinispan.cache.impl.DecoratedCache.remove(DecoratedCache.java:442)
>     at org.infinispan.cache.impl.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:297)
>     at org.wildfly.clustering.server.registry.CacheRegistry.topologyChanged(CacheRegistry.java:152)
>     at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:497)
>     at org.infinispan.notifications.impl.AbstractListenerImpl$ListenerInvocationImpl$1.run(AbstractListenerImpl.java:286)
>     at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)
>     at org.infinispan.notifications.impl.AbstractListenerImpl$ListenerInvocationImpl.invoke(AbstractListenerImpl.java:309)
>     at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.doRealInvocation(CacheNotifierImpl.java:1180)
>     at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.invoke(CacheNotifierImpl.java:1139)
>     at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.invoke(CacheNotifierImpl.java:1105)
>     at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyTopologyChanged(CacheNotifierImpl.java:560)
>     at org.infinispan.statetransfer.StateTransferManagerImpl.doTopologyUpdate(StateTransferManagerImpl.java:201)
>     at org.infinispan.statetransfer.StateTransferManagerImpl.access$000(StateTransferManagerImpl.java:45)
>     at org.infinispan.statetransfer.StateTransferManagerImpl$1.updateConsistentHash(StateTransferManagerImpl.java:113)
>     at org.infinispan.topology.LocalTopologyManagerImpl.doHandleTopologyUpdate(LocalTopologyManagerImpl.java:285)
>     at org.infinispan.topology.LocalTopologyManagerImpl$1.run(LocalTopologyManagerImpl.java:218)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at org.infinispan.executors.SemaphoreCompletionService$QueueingTask.runInternal(SemaphoreCompletionService.java:166)
>     at org.infinispan.executors.SemaphoreCompletionService$QueueingTask.run(SemaphoreCompletionService.java:144)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745){noformat}
>  * Warnings such as: {noformat}WARN  [org.infinispan.transaction.impl.TransactionTable] (TxCleanupService,dist,rhel7gax86-64) ISPN000326: Remote transaction GlobalTransaction:<rhel7gax86-64>:9373:remote timed out. Rolling back after 70810 ms{noformat}
> Logs from one of such play & test scenarios are attached: [^logs.zip], [^configs.zip].
> Any ideas which configuration directive or application setting might be at the bottom of this?
> Needless to say any such test passes without any error with EAP 6.4 and 6.3 both with shutdown and undeploy failover scenarios.
> Thx for comments.



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)