[JBoss JIRA] (ISPN-3704) StateTransfer's PutKeyValueCommand may trigger remote get
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-3704?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-3704:
------------------------------
Fix Version/s: 6.1.0.Final
> StateTransfer's PutKeyValueCommand may trigger remote get
> ---------------------------------------------------------
>
> Key: ISPN-3704
> URL: https://issues.jboss.org/browse/ISPN-3704
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 6.0.0.CR1
> Reporter: Radim Vansa
> Assignee: Dan Berindei
> Priority: Critical
> Labels: 620
> Fix For: 6.1.0.Final
>
>
> In TX mode with write skew check on, ST executing puts may trigger a remote get.
> The condition in TxDistributionInterceptor.handleTxWriteCommand should probably be switched from
> {code}
> if (ctx.isOriginLocal() && !skipRemoteGet || command.isConditional() || shouldFetchRemoteValuesForWriteSkewCheck(ctx, command))
> remoteGetBeforeWrite(ctx, command, recipientGenerator);
> {code}
> to
> {code}
> if (!skipRemoteGet && (ctx.isOriginLocal() || command.isConditional() || shouldFetchRemoteValuesForWriteSkewCheck(ctx, command)))
> remoteGetBeforeWrite(ctx, command, recipientGenerator);
> {code}
> EDIT:
> I have also registered a situation where the Prepare/Commit command was executed remotely from within the ST because the topology has changed during the remote get. This should be avoided as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-3215) Improve message routing for PutMapCommand for non transactional caches
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-3215?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-3215:
------------------------------
Affects Version/s: 6.0.0.Final
> Improve message routing for PutMapCommand for non transactional caches
> ----------------------------------------------------------------------
>
> Key: ISPN-3215
> URL: https://issues.jboss.org/browse/ISPN-3215
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core API
> Affects Versions: 6.0.0.Final
> Reporter: Radim Vansa
> Assignee: Mircea Markus
>
> Current implementation:
> A wants to execute PutMapCommand with many keys - let's assume that in fact the keys span all nodes in the cluster.
> 1. A locks all local keys and sends via unicast a message to each primary owner of some of the keys in the map
> 2. A sends unicast message to each node, requesting the operation
> 3. Each node locks its keys and sends multicast message to ALL other nodes in the cluster
> This happens N - 1 times:
> 4. Each node receives the multicast message, (updates the non-primary segments) and sends reply back to the sender of mcast message.
> 5. The primary owners send confirmation back to A.
> Let's compute how many messages are here received - it's
> N - 1 // A's request
> (N - 1) * (N - 1) // multicast message received
> (N - 1) * (N - 1) // reply to the multicast message received
> N - 1 // response to A
> That's 2*N^2 - 2*N messages. In case nobody needs flow control replenishments, nothing is lost etc. That ^2 exponent does not look like the cluster is really scaling.
> Could the requestor orchestrate the whole operation? The idea is that all messages are sent only between requestor and other nodes, never between the other nodes. The requestor would lock the primary keys by one set of messages (waiting for reply), updating the non-primaries by another set of messages and then again unlocking all primaries by last message.
> The set of messages could be either unicast with selected keys only for the recipient, or multicast with whole map - rationalization which one is actually better is subject to performance test.
> This results in 6*N - 6 messages (or 5*N - 5 if the last message wouldn't require the reply). You can easily see when 5*(N - 1) is better than 2*N*(N - 1).
> Or is this too similar to transactions with multiple keys?
> I think that with current implementation, the putAll operation should be discouraged as it does not provide better performance than multiple put (and in terms of atomicity it's probably not much better either).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-2892) View installation loop when restarting cache on multiple nodes
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2892?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-2892:
-----------------------------------------------
Paul Ferraro <paul.ferraro(a)redhat.com> changed the Status of [bug 924806|https://bugzilla.redhat.com/show_bug.cgi?id=924806] from NEW to CLOSED
> View installation loop when restarting cache on multiple nodes
> --------------------------------------------------------------
>
> Key: ISPN-2892
> URL: https://issues.jboss.org/browse/ISPN-2892
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.1.7.Final
> Reporter: Dennis Reed
> Assignee: Dan Berindei
>
> Restarting a cache on multiple nodes at the same time can cause the following error:
> ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-19,node1/web) ISPN000172: Failed to prepare view CacheView{viewId=18, members=[node2/web]} for cache default-host/test, rolling back to view CacheView{viewId=17, members=[node1/web, node2/web]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: java.lang.IllegalStateException: default-host/test: Received cache view prepare request after the local node has already shut down
> After the initial error, the following error began repeating every second for a few minutes until BaseStateTransferManagerImpl.waitForJoinToComplete() timed out and the cache failed to start:
> ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-19,node1/web) ISPN000172: Failed to prepare view CacheView{viewId=21, members=[node2/web]} for cache default-host/test, rolling back to view CacheView{viewId=20, members=[]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: java.lang.IllegalStateException: Cannot prepare new view CacheView{viewId=21, members=[node2/web]} on cache default-host/test, we are currently preparing view CacheView{viewId=18, members=[node2/web]}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-2752) IllegalStateException on shutdown
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2752?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-2752:
------------------------------------------
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=903308, https://bugzilla.redhat.com/show_bug.cgi?id=924755 (was: https://bugzilla.redhat.com/show_bug.cgi?id=903308)
> IllegalStateException on shutdown
> ---------------------------------
>
> Key: ISPN-2752
> URL: https://issues.jboss.org/browse/ISPN-2752
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.0.CR2
> Reporter: Michal Linhard
> Assignee: Dan Berindei
> Priority: Minor
>
> This happens on graceful shutdown of 8 or 6 nodes of 5.2.0.CR2:
> {code}
> 07:17:45,892 ERROR [org.infinispan.topology.ClusterTopologyManagerImpl] ISPN000196: Failed to recover cluster state after the current node became the coordinator: java.util.concurrent.ExecutionException: org.infinispan.CacheException: Remote (node0005/default) failed unexpectedly
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) [rt.jar:1.6.0_33]
> at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_33]
> at org.infinispan.topology.ClusterTopologyManagerImpl.executeOnClusterSync(ClusterTopologyManagerImpl.java:567) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.topology.ClusterTopologyManagerImpl.recoverClusterStatus(ClusterTopologyManagerImpl.java:432) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.topology.ClusterTopologyManagerImpl.handleNewView(ClusterTopologyManagerImpl.java:231) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.topology.ClusterTopologyManagerImpl$ClusterViewListener.handleViewChange(ClusterTopologyManagerImpl.java:597) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_33]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_33]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_33]
> at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_33]
> at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation$1.run(AbstractListenerImpl.java:212) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_33]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_33]
> at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_33]
> Caused by: org.infinispan.CacheException: Remote (node0005/default) failed unexpectedly
> at org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:96) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:541) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.topology.ClusterTopologyManagerImpl$2.call(ClusterTopologyManagerImpl.java:549) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.topology.ClusterTopologyManagerImpl$2.call(ClusterTopologyManagerImpl.java:546) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [rt.jar:1.6.0_33]
> at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_33]
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_33]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_33]
> at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_33]
> at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.0.0.GA-redhat-2.jar:2.0.0.GA-redhat-2]
> Caused by: java.lang.IllegalStateException: transport was closed
> at org.jgroups.blocks.GroupRequest.transportClosed(GroupRequest.java:273) [jgroups-3.2.6.Final-redhat-1.jar:3.2.6.Final-redhat-1]
> at org.jgroups.blocks.RequestCorrelator.stop(RequestCorrelator.java:269) [jgroups-3.2.6.Final-redhat-1.jar:3.2.6.Final-redhat-1]
> at org.jgroups.blocks.MessageDispatcher.stop(MessageDispatcher.java:152) [jgroups-3.2.6.Final-redhat-1.jar:3.2.6.Final-redhat-1]
> at org.jgroups.blocks.MessageDispatcher.channelDisconnected(MessageDispatcher.java:455) [jgroups-3.2.6.Final-redhat-1.jar:3.2.6.Final-redhat-1]
> at org.jgroups.Channel.notifyChannelDisconnected(Channel.java:507) [jgroups-3.2.6.Final-redhat-1.jar:3.2.6.Final-redhat-1]
> at org.jgroups.JChannel.disconnect(JChannel.java:363) [jgroups-3.2.6.Final-redhat-1.jar:3.2.6.Final-redhat-1]
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.stop(JGroupsTransport.java:258) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_33]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_33]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_33]
> at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_33]
> at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:883) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.factories.AbstractComponentRegistry.internalStop(AbstractComponentRegistry.java:690) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.factories.AbstractComponentRegistry.stop(AbstractComponentRegistry.java:568) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.factories.GlobalComponentRegistry.stop(GlobalComponentRegistry.java:260) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:742) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.infinispan.manager.AbstractDelegatingEmbeddedCacheManager.stop(AbstractDelegatingEmbeddedCacheManager.java:179) [infinispan-core-5.2.0.CR2-redhat-1.jar:5.2.0.CR2-redhat-1]
> at org.jboss.as.clustering.infinispan.subsystem.EmbeddedCacheManagerService.stop(EmbeddedCacheManagerService.java:76) [jboss-datagrid-infinispan-6.1.0.ER9-redhat-1.jar:6.1.0.ER9-redhat-1]
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:1911) [jboss-msc-1.0.2.GA-redhat-2.jar:1.0.2.GA-redhat-2]
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:1874) [jboss-msc-1.0.2.GA-redhat-2.jar:1.0.2.GA-redhat-2]
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_33]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_33]
> at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_33]
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-3450) Jon plugin - failing RebuildIndex operation on [MassIndexer] component
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-3450?page=com.atlassian.jira.plugin.... ]
Adrian Nistor edited comment on ISPN-3450 at 12/12/13 9:37 AM:
---------------------------------------------------------------
This definitelly does not work because in CacheComponent.queryBean method there is a check that ensures the MBean name has a type=Cache attribute. This is true for all cache components but not for the mass indexing MBean which has type=Query. For this one the method returns null, leading to the observed NPE.
An easy fix would be to add a type parameter to the method in question.
was (Author: anistor):
This definitelly does not work because in CacheComponent.queryBean method there is a check that ensure the MBean name has a type=Cache attribute. This is true for all cache components but not for the mass indexing mbean which has type=Query. For this one the method returns null, leading to the observer NPE.
An easy fix would be to add a type parameter to the method in question.
> Jon plugin - failing RebuildIndex operation on [MassIndexer] component
> ----------------------------------------------------------------------
>
> Key: ISPN-3450
> URL: https://issues.jboss.org/browse/ISPN-3450
> Project: Infinispan
> Issue Type: Bug
> Reporter: Tomas Sykora
> Assignee: Adrian Nistor
>
> RebuildIndex operation on [MassIndexer] component is failing while issuing in JON with this error:
> java.lang.NullPointerException
> at org.infinispan.rhq.CacheComponent.invokeOperation(CacheComponent.java:157)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:634)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> I reused configuration from ISPN test suite + I was able to issue this operation successfully via jconsole tool. That's why I suspect that problems are somewhere in JON-ISPN interaction or plugin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-3450) Jon plugin - failing RebuildIndex operation on [MassIndexer] component
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-3450?page=com.atlassian.jira.plugin.... ]
Adrian Nistor reassigned ISPN-3450:
-----------------------------------
Assignee: Adrian Nistor (was: Mircea Markus)
> Jon plugin - failing RebuildIndex operation on [MassIndexer] component
> ----------------------------------------------------------------------
>
> Key: ISPN-3450
> URL: https://issues.jboss.org/browse/ISPN-3450
> Project: Infinispan
> Issue Type: Bug
> Reporter: Tomas Sykora
> Assignee: Adrian Nistor
>
> RebuildIndex operation on [MassIndexer] component is failing while issuing in JON with this error:
> java.lang.NullPointerException
> at org.infinispan.rhq.CacheComponent.invokeOperation(CacheComponent.java:157)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:634)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> I reused configuration from ISPN test suite + I was able to issue this operation successfully via jconsole tool. That's why I suspect that problems are somewhere in JON-ISPN interaction or plugin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-3450) Jon plugin - failing RebuildIndex operation on [MassIndexer] component
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-3450?page=com.atlassian.jira.plugin.... ]
Adrian Nistor commented on ISPN-3450:
-------------------------------------
This definitelly does not work because in CacheComponent.queryBean method there is a check that ensure the MBean name has a type=Cache attribute. This is true for all cache components but not for the mass indexing mbean which has type=Query. For this one the method returns null, leading to the observer NPE.
An easy fix would be to add a type parameter to the method in question.
> Jon plugin - failing RebuildIndex operation on [MassIndexer] component
> ----------------------------------------------------------------------
>
> Key: ISPN-3450
> URL: https://issues.jboss.org/browse/ISPN-3450
> Project: Infinispan
> Issue Type: Bug
> Reporter: Tomas Sykora
> Assignee: Mircea Markus
>
> RebuildIndex operation on [MassIndexer] component is failing while issuing in JON with this error:
> java.lang.NullPointerException
> at org.infinispan.rhq.CacheComponent.invokeOperation(CacheComponent.java:157)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:634)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> I reused configuration from ISPN test suite + I was able to issue this operation successfully via jconsole tool. That's why I suspect that problems are somewhere in JON-ISPN interaction or plugin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months