[JBoss JIRA] (ISPN-5498) ClientClusterFailoverEventsTest.testEventReplayWithAndWithoutStateAfterFailover random failures
by Galder Zamarreño (Jira)
[ https://issues.jboss.org/browse/ISPN-5498?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5498:
-----------------------------------
Fix Version/s: 9.4.5.Final
(was: 9.4.4.Final)
> ClientClusterFailoverEventsTest.testEventReplayWithAndWithoutStateAfterFailover random failures
> -----------------------------------------------------------------------------------------------
>
> Key: ISPN-5498
> URL: https://issues.jboss.org/browse/ISPN-5498
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 9.4.5.Final
>
>
> It appears the cluster event notifier wants to send some events to a node that no longer exists, and the exception is propagated all the way to the HR client:
> {noformat}
> 12:04:03,088 ERROR (HotRodServerWorker-78-1:) [InvocationContextInterceptor] ISPN000136: Execution errorjava.lang.IllegalArgumentException: Target node ClientClusterFailoverEventsTest-NodeA-33510 is not a cluster member, members are [ClientClusterFailoverEventsTest-NodeB-14182]
> at org.infinispan.distexec.DefaultExecutorService.submit(DefaultExecutorService.java:414)
> at org.infinispan.distexec.DefaultExecutorService.submit(DefaultExecutorService.java:403)
> at org.infinispan.distexec.DistributedExecutionCompletionService.submit(DistributedExecutionCompletionService.java:178)
> at org.infinispan.notifications.cachelistener.cluster.impl.BatchingClusterEventManagerImpl$UnicastEventContext.sendToTargets(BatchingClusterEventManagerImpl.java:97)
> at org.infinispan.notifications.cachelistener.cluster.impl.BatchingClusterEventManagerImpl.sendEvents(BatchingClusterEventManagerImpl.java:50)
> at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryCreated(CacheNotifierImpl.java:292)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.notifyCommitEntry(ClusteringDependentLogic.java:138)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$DistributionLogic.commitSingleEntry(ClusteringDependentLogic.java:493)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:108)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:367)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:545)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:344)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:418)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:449)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:195)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:88)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:55)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:324)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:256)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:115)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1617)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1097)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1089)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:522)
> at org.infinispan.server.hotrod.CacheDecodeContext.put(CacheDecodeContext.scala:216)
> at org.infinispan.server.hotrod.HotRodDecoder.org$infinispan$server$hotrod$HotRodDecoder$$decodeValue(HotRodDecoder.scala:132)
> at org.infinispan.server.hotrod.HotRodDecoder$$anonfun$decode$1.apply$mcV$sp(HotRodDecoder.scala:50)
> at org.infinispan.server.hotrod.HotRodDecoder.wrapSecurity(HotRodDecoder.scala:208)
> at org.infinispan.server.hotrod.HotRodDecoder.decode(HotRodDecoder.scala:45)
> org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for message id[922] returned server error (status=0x85): java.lang.IllegalArgumentException: Target node ClientClusterFailoverEventsTest-NodeA-33510 is not a cluster member, members are [ClientClusterFailoverEventsTest-NodeB-14182]
> at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:336)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:126)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:112)
> at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56)
> at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:57)
> at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:31)
> at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:20)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:52)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:247)
> at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79)
> at org.infinispan.client.hotrod.event.ClientClusterFailoverEventsTest.testEventReplayWithAndWithoutStateAfterFailover(ClientClusterFailoverEventsTest.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 4 months
[JBoss JIRA] (ISPN-5470) Remote-executor threads should not block to acquire locks
by Galder Zamarreño (Jira)
[ https://issues.jboss.org/browse/ISPN-5470?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5470:
-----------------------------------
Fix Version/s: 9.4.5.Final
(was: 9.4.4.Final)
> Remote-executor threads should not block to acquire locks
> ---------------------------------------------------------
>
> Key: ISPN-5470
> URL: https://issues.jboss.org/browse/ISPN-5470
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Priority: Major
> Fix For: 9.4.5.Final
>
>
> Currently, enabling the queue on the remote-executor thread pool can cause deadlocks, because a CommitCommand/1PCPrepareCommand could end up in the queue while a remote-executor thread is busy waiting to acquire the same lock that this commit would release.
> If trying to acquire a lock would free the thread until the key has been acquired, we could enable the queue on the remote-executor/OOB thread pools, and we would need a lot less threads for the same load.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 4 months
[JBoss JIRA] (ISPN-5427) Change getAll to return ordered map
by Galder Zamarreño (Jira)
[ https://issues.jboss.org/browse/ISPN-5427?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5427:
-----------------------------------
Fix Version/s: 9.4.5.Final
(was: 9.4.4.Final)
> Change getAll to return ordered map
> -----------------------------------
>
> Key: ISPN-5427
> URL: https://issues.jboss.org/browse/ISPN-5427
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core, Remote Protocols
> Reporter: William Burns
> Assignee: William Burns
> Priority: Major
> Fix For: 9.4.5.Final
>
>
> Currently our getAll returns a map of entries that exist in the cache in any order. We could enhance this to return a map where the entries are in the same iteration order of the Set (important when the user uses a LinkedHashSet). We could even do something like List<V> getAll(List<K>) to show better ordering as an option too.
> This has 2 immediate advantages.
> 1. Remote getAll no longer requires to serialize all the keys in the response since it knows which values map to which keys based on what it sent.
> 2. Query results need ordered List as well. See query/src/main/java/org/infinispan/query/impl/EntityLoader.java
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 4 months
[JBoss JIRA] (ISPN-5415) Expose protobuf entries to scripting
by Galder Zamarreño (Jira)
[ https://issues.jboss.org/browse/ISPN-5415?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5415:
-----------------------------------
Fix Version/s: 9.4.5.Final
(was: 9.4.4.Final)
> Expose protobuf entries to scripting
> ------------------------------------
>
> Key: ISPN-5415
> URL: https://issues.jboss.org/browse/ISPN-5415
> Project: Infinispan
> Issue Type: Feature Request
> Components: Remote Querying
> Affects Versions: 8.0.0.Final
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Priority: Major
> Fix For: 9.4.5.Final
>
>
> We need an alternative API for Protostream marshalling that is easy to consume from scripting languages. The messages need to be unmarshalled into a map-like object that can be accessed easily from scripting languages. No marshaller implementation code should be provided by users, also no annotations.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 4 months
[JBoss JIRA] (ISPN-5357) Hot Rod protocol 3.0
by Galder Zamarreño (Jira)
[ https://issues.jboss.org/browse/ISPN-5357?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5357:
-----------------------------------
Fix Version/s: 9.4.5.Final
(was: 9.4.4.Final)
> Hot Rod protocol 3.0
> --------------------
>
> Key: ISPN-5357
> URL: https://issues.jboss.org/browse/ISPN-5357
> Project: Infinispan
> Issue Type: Enhancement
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Priority: Major
> Fix For: 9.4.5.Final
>
>
> Infinispan should expose a single endpoint which should behave as follows:
> - by default offer an HTTP/1.1 RESTftul interface
> - through HTTP upgrade allow switching to better protocols
> - support an HTTP/2 RESTful interface
> - support Hot Rod 3.0, which would be a gRPC protocol with additional L2 and L3 intelligence for capable clients
> *UPDATE (6/10/2015)*: Hot Rod 3.0 could become a HTTP REST protocol taking advantage of existing infrastructure to deal with HTTP requests efficiently, and if it was based on HTTP 2.0, we could have binary data inside of it. Performance should be on par. Text mode could be available for debug or demos. This essentially would mean REST and Hot Rod would merge into one.
> A binary protocol rethink is needed to take better advantage of CPUs and buffering. This is based on the ideas of Martin Thompson's Simple Binary Encoding [blog post|http://mechanical-sympathy.blogspot.cz/2014/05/simple-binary-encoding.html]
> In essence, we need to move Hot Rod protocol towards having mostly fixed size fields, hence getting rid of `vInt` and `vLong` formats which although safe some bytes, they hinder long stream reading patterns.
> An optional part here would be whether to consider using SBE directly, but this JIRA should be mostly oriented at making sure we have a protocol that can be read fast and efficiently.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 4 months
[JBoss JIRA] (ISPN-5241) Cache topology updates should use the NO_FC flag
by Galder Zamarreño (Jira)
[ https://issues.jboss.org/browse/ISPN-5241?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5241:
-----------------------------------
Fix Version/s: 9.4.5.Final
(was: 9.4.4.Final)
> Cache topology updates should use the NO_FC flag
> ------------------------------------------------
>
> Key: ISPN-5241
> URL: https://issues.jboss.org/browse/ISPN-5241
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.1.0.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 9.4.5.Final
>
>
> Topology updates are sent while holding the ClusterCacheStatus lock, so they should never block. However, when MFC is present, the topology update can block waiting for enough credits. As most CacheTopologyControlCommands need to acquire the ClusterCacheStatus lock, this can easily lead to a full remote-executor pool (and OOB pool) and the appearance of a deadlock.
> What's more, if one node is not responsive, it can block all the other nodes from receiving further topology updates. Topology updates should be as prompt as possible, so we should use the NO_FC flag to ensure that each node receives topology updates as soon as possible.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 4 months
[JBoss JIRA] (ISPN-5163) A write operation with the SKIP_LOCKING flag can roll back the transaction
by Galder Zamarreño (Jira)
[ https://issues.jboss.org/browse/ISPN-5163?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5163:
-----------------------------------
Fix Version/s: 9.4.5.Final
(was: 9.4.4.Final)
> A write operation with the SKIP_LOCKING flag can roll back the transaction
> --------------------------------------------------------------------------
>
> Key: ISPN-5163
> URL: https://issues.jboss.org/browse/ISPN-5163
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.0.3.Final, 7.1.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Major
> Fix For: 9.4.5.Final
>
>
> When a write operation has the SKIP_LOCKING flag, it does not send a {{LockControlCommand}} to the primary owner, but it can send a {{ClusteredGetCommand}} with {{acquireRemoteLocks=true}} instead. The {{ClusteredGetCommmand}} will then execute a {{LockControlCommand}} with the origin not set properly, and {{TxInterceptor}} will roll back the transaction because the originator ({{null}}) appears to have left the cluster.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 4 months
[JBoss JIRA] (ISPN-6041) Remote Listeners: Client event reader thread reports EOF as error
by Galder Zamarreño (Jira)
[ https://issues.jboss.org/browse/ISPN-6041?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-6041:
-----------------------------------
Fix Version/s: 9.4.5.Final
(was: 9.4.4.Final)
> Remote Listeners: Client event reader thread reports EOF as error
> -----------------------------------------------------------------
>
> Key: ISPN-6041
> URL: https://issues.jboss.org/browse/ISPN-6041
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Protocols
> Affects Versions: 8.1.0.Final
> Reporter: Dan Berindei
> Priority: Minor
> Fix For: 9.4.5.Final
>
>
> {noformat}
> 14:02:14,904 ERROR (Client-Listener-87aa07aee56d43e1) [ClientListenerNotifier] ISPN004043: Unrecoverable error reading event from server /127.0.0.1:15530, exiting event reader thread
> org.infinispan.client.hotrod.exceptions.TransportException: End of stream reached!
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:198)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readMagic(Codec20.java:305)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readEvent(Codec20.java:147)
> at org.infinispan.client.hotrod.event.ClientListenerNotifier$EventDispatcher.run(ClientListenerNotifier.java:252)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> {noformat}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 4 months