[JBoss JIRA] (ISPN-2420) Increment the topology id when a node leaves the cache
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-2420?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-2420:
-------------------------------
Fix Version/s: 5.2.0.CR2
(was: 6.0.0.Final)
> Increment the topology id when a node leaves the cache
> ------------------------------------------------------
>
> Key: ISPN-2420
> URL: https://issues.jboss.org/browse/ISPN-2420
> Project: Infinispan
> Issue Type: Feature Request
> Components: State transfer
> Affects Versions: 5.2.0.Beta2
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 5.2.0.CR2
>
>
> We currently increment the topology id only when we "rebalance" the consistent hash (i.e. we add new owners). This allows us to to do some optimizations after a leave, like not forwarding commands (because there are no new owners).
> Unfortunately, it is not very intuitive, because it doesn't match how JGroups works, so it can cause bugs like ISPN-2417.
> Additionally, it turns out there are many places where we care that a node left, so the code is more complex to handle this (e.g. TransactionTable.useStrictTopologyIdComparison()), or it is more slow for the common case when there is no leaver (e.g. LocalTransaction.getCommitNodes()).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2719) Tx complete notification and prepare crossing leads to failed transaction
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-2719?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-2719:
------------------------------------
Galder, are you sure the PrepareCommand is sent asynchronously? AFAIK the PrepareCommand is only sent asynchronously if the cache is 1-PC, but then there's no TxCompletionNotificationCommand.
> Tx complete notification and prepare crossing leads to failed transaction
> -------------------------------------------------------------------------
>
> Key: ISPN-2719
> URL: https://issues.jboss.org/browse/ISPN-2719
> Project: Infinispan
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 5.2.0.CR2
> Reporter: Galder Zamarreño
> Assignee: Mircea Markus
> Priority: Critical
> Fix For: 5.2.0.Final
>
> Attachments: testRemoteCacheListener-5.log
>
>
> {code}testRemoteCacheListener(org.infinispan.replication.SyncCacheListenerTest) Time elapsed: 0.059 sec <<< FAILURE!
> org.infinispan.CacheException: Could not commit implicit transaction
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1159)
> at org.infinispan.CacheImpl.removeInternal(CacheImpl.java:310)
> at org.infinispan.CacheImpl.remove(CacheImpl.java:304)
> at org.infinispan.CacheImpl.remove(CacheImpl.java:298)
> at org.infinispan.replication.SyncCacheListenerTest.testRemoteCacheListener(SyncCacheListenerTest.java:113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:715)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:907)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1237)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1156)
> ... 25 more{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2703) The "machine" attribute of server hinting does not work
by Martin Gencur (JIRA)
[ https://issues.jboss.org/browse/ISPN-2703?page=com.atlassian.jira.plugin.... ]
Martin Gencur commented on ISPN-2703:
-------------------------------------
Actually, the originally attached logs were taken from related BugZilla and were supposed to show the problem just for "site" attribute. This has been recently fixed and now only the "machine" attribute is not working.
> The "machine" attribute of server hinting does not work
> -------------------------------------------------------
>
> Key: ISPN-2703
> URL: https://issues.jboss.org/browse/ISPN-2703
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.0.Beta6
> Reporter: Martin Gencur
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.0.Final
>
> Attachments: server-hinting-logs.zip, trace_log_machine_attribute_failing.txt
>
>
> The rack and site attributes already work correctly. I'm creating this one issue just for the remaining "machine" attribute.
> Will attach trace logs
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2703) The "machine" attribute of server hinting does not work
by Martin Gencur (JIRA)
[ https://issues.jboss.org/browse/ISPN-2703?page=com.atlassian.jira.plugin.... ]
Martin Gencur edited comment on ISPN-2703 at 1/22/13 7:01 AM:
--------------------------------------------------------------
Dan, I'm attaching DEBUG logs from all three servers as well as the maven build output. The topology information is configured in JGroups subsystem of JDG and this is the configuration snippet: https://svn.devel.redhat.com/repos/jboss-qa/jdg/jdg-functional-tests/trun...
Maven puts together individual pieces of configuration and the final standalone-ha.xml configuration file is located at JDG_HOME/standalone/configuration/standalone-ha.xml
The previously attached log was provided by Tomas and I guess it's not correct.
We wouldn't consider it as a bug but this test was passing for JDG 6.0.1 (based on ISPN 5.1.7).
was (Author: mgencur):
Dan, I'm attaching DEBUG logs from all three servers as well as the maven build output. The topology information is configured in JGroups subsystem of JDG and this is the configuration snippet: https://svn.devel.redhat.com/repos/jboss-qa/jdg/jdg-functional-tests/trun...
Maven puts together individual pieces of configuration and the final standalone-ha.xml configuration file is located at ${JDG_HOME}/standalone/configuration/standalone-ha.xml
The previously attached log was provided by Tomas and I guess it's not correct.
We wouldn't consider it as a bug but this test was passing for JDG 6.0.1 (based on ISPN 5.1.7).
> The "machine" attribute of server hinting does not work
> -------------------------------------------------------
>
> Key: ISPN-2703
> URL: https://issues.jboss.org/browse/ISPN-2703
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.0.Beta6
> Reporter: Martin Gencur
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.0.Final
>
> Attachments: server-hinting-logs.zip, trace_log_machine_attribute_failing.txt
>
>
> The rack and site attributes already work correctly. I'm creating this one issue just for the remaining "machine" attribute.
> Will attach trace logs
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2703) The "machine" attribute of server hinting does not work
by Martin Gencur (JIRA)
[ https://issues.jboss.org/browse/ISPN-2703?page=com.atlassian.jira.plugin.... ]
Martin Gencur commented on ISPN-2703:
-------------------------------------
Dan, I'm attaching DEBUG logs from all three servers as well as the maven build output. The topology information is configured in JGroups subsystem of JDG and this is the configuration snippet: https://svn.devel.redhat.com/repos/jboss-qa/jdg/jdg-functional-tests/trun...
Maven puts together individual pieces of configuration and the final standalone-ha.xml configuration file is located at ${JDG_HOME}/standalone/configuration/standalone-ha.xml
The previously attached log was provided by Tomas and I guess it's not correct.
We wouldn't consider it as a bug but this test was passing for JDG 6.0.1 (based on ISPN 5.1.7).
> The "machine" attribute of server hinting does not work
> -------------------------------------------------------
>
> Key: ISPN-2703
> URL: https://issues.jboss.org/browse/ISPN-2703
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.0.Beta6
> Reporter: Martin Gencur
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.0.Final
>
> Attachments: server-hinting-logs.zip, trace_log_machine_attribute_failing.txt
>
>
> The rack and site attributes already work correctly. I'm creating this one issue just for the remaining "machine" attribute.
> Will attach trace logs
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2737) Thread naming anomaly when reporting lock timeout
by Michal Linhard (JIRA)
[ https://issues.jboss.org/browse/ISPN-2737?page=com.atlassian.jira.plugin.... ]
Michal Linhard commented on ISPN-2737:
--------------------------------------
this may be problem in jboss logging or netty as well...
> Thread naming anomaly when reporting lock timeout
> -------------------------------------------------
>
> Key: ISPN-2737
> URL: https://issues.jboss.org/browse/ISPN-2737
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 5.2.0.CR1
> Reporter: Michal Linhard
> Assignee: Mircea Markus
> Priority: Minor
>
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EDG6/view/EDG-REPOR...
> {code}
> 11:47:30,859 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (MemcachedServerWorker-277) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [3 seconds] on key [memcachedCache#key763328] for requestor [Thread[OOB-127,null,5,Thread Pools]]! Lock held by [Thread[OOB-150,null,5,Thread Pools]]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:217) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLockNoCheck(LockManagerImpl.java:200) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:114) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
> ....
> at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.Discovery.up(Discovery.java:359) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2640) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1287) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1850) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1823) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_38]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_38]
> at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
> {code}
> note the thread name "MemcachedServerWorker" in an operation coming from the JGroups stack...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2737) Thread naming anomaly when reporting lock timeout
by Michal Linhard (JIRA)
[ https://issues.jboss.org/browse/ISPN-2737?page=com.atlassian.jira.plugin.... ]
Michal Linhard updated ISPN-2737:
---------------------------------
Affects Version/s: 5.2.0.CR1
> Thread naming anomaly when reporting lock timeout
> -------------------------------------------------
>
> Key: ISPN-2737
> URL: https://issues.jboss.org/browse/ISPN-2737
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 5.2.0.CR1
> Reporter: Michal Linhard
> Assignee: Mircea Markus
> Priority: Minor
>
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EDG6/view/EDG-REPOR...
> {code}
> 11:47:30,859 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (MemcachedServerWorker-277) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [3 seconds] on key [memcachedCache#key763328] for requestor [Thread[OOB-127,null,5,Thread Pools]]! Lock held by [Thread[OOB-150,null,5,Thread Pools]]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:217) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLockNoCheck(LockManagerImpl.java:200) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:114) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
> ....
> at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.Discovery.up(Discovery.java:359) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2640) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1287) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1850) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1823) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_38]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_38]
> at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
> {code}
> note the thread name "MemcachedServerWorker" in an operation coming from the JGroups stack...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2737) Thread naming anomaly when reporting lock timeout
by Michal Linhard (JIRA)
Michal Linhard created ISPN-2737:
------------------------------------
Summary: Thread naming anomaly when reporting lock timeout
Key: ISPN-2737
URL: https://issues.jboss.org/browse/ISPN-2737
Project: Infinispan
Issue Type: Bug
Components: Locking and Concurrency
Reporter: Michal Linhard
Assignee: Mircea Markus
Priority: Minor
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EDG6/view/EDG-REPOR...
{code}
11:47:30,859 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (MemcachedServerWorker-277) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [3 seconds] on key [memcachedCache#key763328] for requestor [Thread[OOB-127,null,5,Thread Pools]]! Lock held by [Thread[OOB-150,null,5,Thread Pools]]
at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:217) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLockNoCheck(LockManagerImpl.java:200) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:114) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
....
at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
at org.jgroups.protocols.Discovery.up(Discovery.java:359) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2640) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
at org.jgroups.protocols.TP.passMessageUp(TP.java:1287) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1850) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1823) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_38]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_38]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
{code}
note the thread name "MemcachedServerWorker" in an operation coming from the JGroups stack...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2703) The "machine" attribute of server hinting does not work
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-2703?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-2703:
------------------------------------
[~mgencur], I looked at the test but I don't really understand how it works. Where is the topology information configured?
Are you sure that all 3 servers are up and running during the test? I have looked again at the attached log, and I didn't see any signs of a 3rd node starting. This is the only mention of {{node2}}:
{noformat}
[INFO] --- maven-resources-plugin:2.5:copy-resources (node2) @ server-hinting-site-tests ---
{noformat}
Could you run the test again and attach the logs from all the servers?
Note that we already have a test for the machine attribute and it works (TopologyAwareConsistentHashFactoryTest.testDifferentMachines), so I'm pretty sure this is just a setup issue.
> The "machine" attribute of server hinting does not work
> -------------------------------------------------------
>
> Key: ISPN-2703
> URL: https://issues.jboss.org/browse/ISPN-2703
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.0.Beta6
> Reporter: Martin Gencur
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.0.Final
>
> Attachments: trace_log_machine_attribute_failing.txt
>
>
> The rack and site attributes already work correctly. I'm creating this one issue just for the remaining "machine" attribute.
> Will attach trace logs
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months