[JBoss JIRA] (ISPN-5518) Introduce a separate thread pool for async cache operations
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5518?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5518:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1226923|https://bugzilla.redhat.com/show_bug.cgi?id=1226923] from MODIFIED to ON_QA
> Introduce a separate thread pool for async cache operations
> -----------------------------------------------------------
>
> Key: ISPN-5518
> URL: https://issues.jboss.org/browse/ISPN-5518
> Project: Infinispan
> Issue Type: Feature Request
> Components: Configuration, Core
> Affects Versions: 7.2.2.Final, 8.0.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 8.0.0.Alpha2
>
>
> At the moment, it is very easy for an application to start a huge number of putAsync operations and fill the transport executor's thread pool, delaying internal work such as state transfer. Increasing the size of the transport executor's thread pool won't work, because that would in turn fill the remote commands and JGroups' OOB thread pools, with the same effect.
> If the cache async operations used a different thread pool, it would be possible to configure more {{remoteCommandsThreadPool}} threads than {{asyncOperationsThreadPool}} threads, avoiding this problem.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5531) java.lang.UnsupportedOperationException during remove (using RemoteCacheManager)
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5531?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5531:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1229791|https://bugzilla.redhat.com/show_bug.cgi?id=1229791] from MODIFIED to ON_QA
> java.lang.UnsupportedOperationException during remove (using RemoteCacheManager)
> --------------------------------------------------------------------------------
>
> Key: ISPN-5531
> URL: https://issues.jboss.org/browse/ISPN-5531
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.2.2.Final
> Reporter: Enrico Olivelli
> Assignee: William Burns
> Fix For: 8.0.0.Alpha2, 7.2.3.Final
>
>
> during a "remove" from a java HotRod client we got this error.
> This error happens only when there are more than one hotrod server in the cluster.
> We are using hotrod server in library mode (embedded)
> {code}
> 15/06/09 08:05:34 ERROR interceptors.InvocationContextInterceptor: ISPN000136: Execution error
> org.infinispan.commons.CacheListenerException: ISPN000280: Caught exception [java.lang.UnsupportedOperationException] while invoking method [public void org.infinispan.n
> otifications.cachelistener.cluster.RemoteClusterListener.handleClusterEvents(org.infinispan.notifications.cachelistener.event.CacheEntryEvent) throws java.lang.Exception] on listener instance: org.infinispan.not
> ifications.cachelistener.cluster.RemoteClusterListener@28d94aee
> at org.infinispan.notifications.impl.AbstractListenerImpl$ListenerInvocationImpl$1.run(AbstractListenerImpl.java:291)
> at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)
> at org.infinispan.notifications.impl.AbstractListenerImpl$ListenerInvocationImpl.invoke(AbstractListenerImpl.java:309)
> at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.doRealInvocation(CacheNotifierImpl.java:1180)
> at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.invokeNoChecks(CacheNotifierImpl.java:1175)
> at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.invoke(CacheNotifierImpl.java:1152)
> at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryRemoved(CacheNotifierImpl.java:333)
> at org.infinispan.commands.write.RemoveCommand.notify(RemoveCommand.java:97)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.notifyCommitEntry(ClusteringDependentLogic.java:130)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$DistributionLogic.commitSingleEntry(ClusteringDependentLogic.java:493)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:108)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:371)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:549)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:348)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:422)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:453)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitRemoveCommand(EntryWrappingInterceptor.java:252)
> at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.distribution.L1LastChanceInterceptor.visitDataWriteCommand(L1LastChanceInterceptor.java:79)
> at org.infinispan.interceptors.distribution.L1LastChanceInterceptor.visitRemoveCommand(L1LastChanceInterceptor.java:75)
> at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:88)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitRemoveCommand(AbstractLockingInterceptor.java:65)
> at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:49)
> at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:324)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:256)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitRemoveCommand(StateTransferInterceptor.java:130)
> at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitRemoveCommand(CacheMgmtInterceptor.java:209)
> at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.compat.BaseTypeConverterInterceptor.visitRemoveCommand(BaseTypeConverterInterceptor.java:228
> at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:49)
> at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:58)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1617)
> at org.infinispan.cache.impl.CacheImpl.removeInternal(CacheImpl.java:579)
> at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:572)
> at org.infinispan.cache.impl.DecoratedCache.remove(DecoratedCache.java:442)
> at org.infinispan.server.hotrod.CacheDecodeContext.remove(CacheDecodeContext.scala:235)
> at org.infinispan.server.hotrod.HotRodDecoder.handleModification(HotRodDecoder.scala:216)
> at org.infinispan.server.hotrod.HotRodDecoder.org$infinispan$server$hotrod$HotRodDecoder$$decodeKey(HotRodDecoder.scala:102)
> at org.infinispan.server.hotrod.HotRodDecoder$$anonfun$decode$1.apply$mcV$sp(HotRodDecoder.scala:48)
> at org.infinispan.server.hotrod.HotRodDecoder$$anon$1.run(HotRodDecoder.scala:206)
> at org.infinispan.server.hotrod.HotRodDecoder$$anon$1.run(HotRodDecoder.scala:205)
> at org.infinispan.security.Security.doAs(Security.java:143)
> at org.infinispan.server.hotrod.HotRodDecoder.wrapSecurity(HotRodDecoder.scala:205)
> at org.infinispan.server.hotrod.HotRodDecoder.decode(HotRodDecoder.scala:45)
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:370)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:168)
> at org.infinispan.server.hotrod.HotRodDecoder.org$infinispan$server$core$transport$StatsChannelHandler$$super$channelRead(HotRodDecoder.scala:31)
> at org.infinispan.server.core.transport.StatsChannelHandler$class.channelRead(StatsChannelHandler.scala:32)
> at org.infinispan.server.hotrod.HotRodDecoder.channelRead(HotRodDecoder.scala:31)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.UnsupportedOperationException
> at java.util.AbstractCollection.add(AbstractCollection.java:262)
> at java.util.AbstractCollection.addAll(AbstractCollection.java:344)
> at org.infinispan.notifications.cachelistener.cluster.impl.BatchingClusterEventManagerImpl$UnicastEventContext.addTargets(BatchingClusterEventManagerImpl.java:82)
> at org.infinispan.notifications.cachelistener.cluster.impl.BatchingClusterEventManagerImpl.addEvents(BatchingClusterEventManagerImpl.java:43)
> at org.infinispan.notifications.cachelistener.cluster.RemoteClusterListener.handleClusterEvents(RemoteClusterListener.java:108)
> at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.infinispan.notifications.impl.AbstractListenerImpl$ListenerInvocationImpl$1.run(AbstractListenerImpl.java:286)
> ... 76 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5315) Tight loop on ClientListenerNotifier
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5315?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5315:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1231215|https://bugzilla.redhat.com/show_bug.cgi?id=1231215] from MODIFIED to ON_QA
> Tight loop on ClientListenerNotifier
> ------------------------------------
>
> Key: ISPN-5315
> URL: https://issues.jboss.org/browse/ISPN-5315
> Project: Infinispan
> Issue Type: Bug
> Components: Listeners
> Affects Versions: 7.2.0.Beta1, 7.1.1.Final
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Priority: Minor
> Fix For: 7.2.0.Beta2, 7.2.0.Final
>
> Attachments: stackTrace
>
>
> Sometimes the {{ClientListenerNotifier}} keeps printing extremely high number of warning and quickly fill GBs of disk space:
> {code}
> 11:28:16,805 WARN [ClientListenerNotifier] (Client-Listener-aacb322ffc60405e) ISPN004039: Unable to complete reading event from server null
> java.nio.channels.IllegalBlockingModeException
> at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:201)
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:179)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readMagic(Codec20.java:282)
> at org.infinispan.client.hotrod^C.impl.protocol.Codec20.readEvent(Codec20.java:126)
> at org.infinispan.client.hotrod.event.ClientListenerNotifier$EventDispatcher.run(ClientListenerNotifier.java:237)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> This can be reproduced when running the server test suite, more specifically the test {{ExampleConfigsIT}}. Right after the server is started the above warning is logged in a loop for the duration of the tests.
> I've started to observe this after commit (1), which changed the behaviour to retry on SocketTimeouts.
> Before the aforementioned commit, log were printing:
> {code}
> 10:49:56,361 ERROR [ClientListenerNotifier] (Client-Listener-b53963d5ab2b4234) ISPN004043: Unrecoverable error reading event from server null, exiting event reader thread
> org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:184)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readMagic(Codec20.java:282)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readEvent(Codec20.java:126)
> at org.infinispan.client.hotrod.event.ClientListenerNotifier$EventDispatcher.run(ClientListenerNotifier.java:236)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.SocketTimeoutException
> at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:179)
> ... 8 more
> {code}
> And thus the listener thread was exiting,
> (1) https://github.com/infinispan/infinispan/commit/1a03a0a84307ae8b95c06b398...
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5524) Race condition in SemaphoreCompletionService.executeFront()
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5524?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5524:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1229181|https://bugzilla.redhat.com/show_bug.cgi?id=1229181] from MODIFIED to ON_QA
> Race condition in SemaphoreCompletionService.executeFront()
> -----------------------------------------------------------
>
> Key: ISPN-5524
> URL: https://issues.jboss.org/browse/ISPN-5524
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.2.2.Final, 8.0.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 8.0.0.Alpha2, 7.2.3.Final, 8.0.0.Final
>
>
> If two threads call {{executeFront()}} at the same time, it's possible that none of them executes any task, although there are permits available, and one of the threads just added a task in the queue. This scenario causes random initial state transfer timeouts in the testsuite:
> # T1: continueTaskInBackground(): queue = empty, permits = 0
> # T1: backgroundTaskFinished()
> # T1: = semaphore.release(): queue = empty, permits = 1
> # T1: = executeFront()
> # T1: == semaphore.acquire() -> true: queue = empty, permits = 0
> # T2: == queue.poll() -> null
> # T2: submit(task)
> # T2: = executeFront()
> # T2: == semaphore.acquire() -> false
> # T2: return without executing any task
> # T1: return without executing any task
> Failure example: http://ci.infinispan.org/viewLog.html?buildId=27153&tab=buildResultsDiv&b...
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5689) Preserve compatibility with Hibernate 5.0.0.Final
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-5689?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-5689:
------------------------------------
Component/s: Core
> Preserve compatibility with Hibernate 5.0.0.Final
> -------------------------------------------------
>
> Key: ISPN-5689
> URL: https://issues.jboss.org/browse/ISPN-5689
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 8.0.0.Beta2
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Fix For: 8.0.0.CR1
>
>
> {{org.infinispan.commands.write.AbstractDataWriteCommand}} introduced a final method {{getLockOwner()}} with the same name as a method used by a child class in Hibernate 2LC {{org.hibernate.cache.infinispan.util.BeginInvalidationCommand}}
> To maintain compatibility between Infinispan 8 and Hibernate 5 (already released), that method should be renamed
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5689) Preserve compatibility with Hibernate 5.0.0.Final
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-5689?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-5689:
------------------------------------
Affects Version/s: 8.0.0.Beta2
> Preserve compatibility with Hibernate 5.0.0.Final
> -------------------------------------------------
>
> Key: ISPN-5689
> URL: https://issues.jboss.org/browse/ISPN-5689
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 8.0.0.Beta2
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Fix For: 8.0.0.CR1
>
>
> {{org.infinispan.commands.write.AbstractDataWriteCommand}} introduced a final method {{getLockOwner()}} with the same name as a method used by a child class in Hibernate 2LC {{org.hibernate.cache.infinispan.util.BeginInvalidationCommand}}
> To maintain compatibility between Infinispan 8 and Hibernate 5 (already released), that method should be renamed
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5689) Preserve compatibility with Hibernate 5.0.0.Final
by Gustavo Fernandes (JIRA)
Gustavo Fernandes created ISPN-5689:
---------------------------------------
Summary: Preserve compatibility with Hibernate 5.0.0.Final
Key: ISPN-5689
URL: https://issues.jboss.org/browse/ISPN-5689
Project: Infinispan
Issue Type: Task
Reporter: Gustavo Fernandes
{{org.infinispan.commands.write.AbstractDataWriteCommand}} introduced a final method {{getLockOwner()}} with the same name as a method used by a child class in Hibernate 2LC {{org.hibernate.cache.infinispan.util.BeginInvalidationCommand}}
To maintain compatibility between Infinispan 8 and Hibernate 5 (already released), that method should be renamed
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months