[JBoss JIRA] (ISPN-5454) XSite: RetryMechanismTest random failures
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5454?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5454:
----------------------------------
Fix Version/s: 9.0.0.Beta2
(was: 9.0.0.Beta1)
> XSite: RetryMechanismTest random failures
> -----------------------------------------
>
> Key: ISPN-5454
> URL: https://issues.jboss.org/browse/ISPN-5454
> Project: Infinispan
> Issue Type: Bug
> Components: Cross-Site Replication
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 9.0.0.Beta2
>
>
> {{ClusteredCacheBackupReceiver.awaitRemoteTask()}} doesn't respect the state push command's timeout, at least when it's smaller than the sync replication timeout in the target cache. When that happens, the state provider will resend the state, and there will be 2 state push commands executing at the same time.
> RetryMechanismTest changes the state push timeout to 2 seconds, but the sync replication timeout stays at 15 seconds. This causes failures in {{testRetryLocally}} and {{testFailRetryLocally}}, if it takes more than 2 seconds to suspect the killed node.
> {noformat}
> 10:02:13,007 TRACE (asyncTransportThread-8,NodeN:) [RetryOnFailureXSiteCommand] Sending XSiteStatePushCommand{cacheName=___defaultcache, timeout=2000 (1 keys)} to [NYC (sync, timeout=2000)]
> 10:02:16,008 TRACE (asyncTransportThread-8,NodeN:) [RetryOnFailureXSiteCommand] Sending XSiteStatePushCommand{cacheName=___defaultcache, timeout=2000 (1 keys)} to [NYC (sync, timeout=2000)]
> 10:02:16,040 TRACE (asyncTransportThread-4,NodeP:) [RpcManagerImpl] replication exception:
> org.infinispan.remoting.transport.jgroups.SuspectException: Node NodeQ-56809 was suspected
> 10:02:16,040 TRACE (asyncTransportThread-0,NodeP:) [RpcManagerImpl] replication exception:
> org.infinispan.remoting.transport.jgroups.SuspectException: Node NodeQ-56809 was suspected
> 10:02:19,147 ERROR (testng-RetryMechanismTest:) [UnitTestTestNGListener] Test testFailRetryLocally(org.infinispan.xsite.statetransfer.failures.RetryMechanismTest) failed.
> java.lang.AssertionError: expected:<2> but was:<3>
> at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
> at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364)
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80)
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:245)
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:252)
> at org.infinispan.xsite.statetransfer.failures.RetryMechanismTest.testFailRetryLocally(RetryMechanismTest.java:227)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (ISPN-5163) A write operation with the SKIP_LOCKING flag can roll back the transaction
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5163?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5163:
----------------------------------
Fix Version/s: 9.0.0.Beta2
(was: 9.0.0.Beta1)
> A write operation with the SKIP_LOCKING flag can roll back the transaction
> --------------------------------------------------------------------------
>
> Key: ISPN-5163
> URL: https://issues.jboss.org/browse/ISPN-5163
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.0.3.Final, 7.1.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.0.0.Beta2
>
>
> When a write operation has the SKIP_LOCKING flag, it does not send a {{LockControlCommand}} to the primary owner, but it can send a {{ClusteredGetCommand}} with {{acquireRemoteLocks=true}} instead. The {{ClusteredGetCommmand}} will then execute a {{LockControlCommand}} with the origin not set properly, and {{TxInterceptor}} will roll back the transaction because the originator ({{null}}) appears to have left the cluster.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (ISPN-5476) Cross-site tests should run in parallel
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5476?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5476:
----------------------------------
Fix Version/s: 9.0.0.Beta2
(was: 9.0.0.Beta1)
> Cross-site tests should run in parallel
> ---------------------------------------
>
> Key: ISPN-5476
> URL: https://issues.jboss.org/browse/ISPN-5476
> Project: Infinispan
> Issue Type: Task
> Components: Core, Cross-Site Replication, Test Suite - Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Fix For: 9.0.0.Beta2
>
>
> Currently the cross-site tests have to run in a single thread, and that means they're much slower than the regular core tests.
> It also means they need to run with a separate Maven profile, and that (combined with their duration) makes it very unlikely for devs to run the xsite tests on their machines.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (ISPN-5427) Change getAll to return ordered map
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5427?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5427:
----------------------------------
Fix Version/s: 9.0.0.Beta2
(was: 9.0.0.Beta1)
> Change getAll to return ordered map
> -----------------------------------
>
> Key: ISPN-5427
> URL: https://issues.jboss.org/browse/ISPN-5427
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core, Remote Protocols
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 9.0.0.Beta2
>
>
> Currently our getAll returns a map of entries that exist in the cache in any order. We could enhance this to return a map where the entries are in the same iteration order of the Set (important when the user uses a LinkedHashSet). We could even do something like List<V> getAll(List<K>) to show better ordering as an option too.
> This has 2 immediate advantages.
> 1. Remote getAll no longer requires to serialize all the keys in the response since it knows which values map to which keys based on what it sent.
> 2. Query results need ordered List as well. See query/src/main/java/org/infinispan/query/impl/EntityLoader.java
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (ISPN-5470) Remote-executor threads should not block to acquire locks
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5470?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5470:
----------------------------------
Fix Version/s: 9.0.0.Beta2
(was: 9.0.0.Beta1)
> Remote-executor threads should not block to acquire locks
> ---------------------------------------------------------
>
> Key: ISPN-5470
> URL: https://issues.jboss.org/browse/ISPN-5470
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Fix For: 9.0.0.Beta2
>
>
> Currently, enabling the queue on the remote-executor thread pool can cause deadlocks, because a CommitCommand/1PCPrepareCommand could end up in the queue while a remote-executor thread is busy waiting to acquire the same lock that this commit would release.
> If trying to acquire a lock would free the thread until the key has been acquired, we could enable the queue on the remote-executor/OOB thread pools, and we would need a lot less threads for the same load.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (ISPN-5241) Cache topology updates should use the NO_FC flag
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5241?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5241:
----------------------------------
Fix Version/s: 9.0.0.Beta2
(was: 9.0.0.Beta1)
> Cache topology updates should use the NO_FC flag
> ------------------------------------------------
>
> Key: ISPN-5241
> URL: https://issues.jboss.org/browse/ISPN-5241
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.1.0.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 9.0.0.Beta2
>
>
> Topology updates are sent while holding the ClusterCacheStatus lock, so they should never block. However, when MFC is present, the topology update can block waiting for enough credits. As most CacheTopologyControlCommands need to acquire the ClusterCacheStatus lock, this can easily lead to a full remote-executor pool (and OOB pool) and the appearance of a deadlock.
> What's more, if one node is not responsive, it can block all the other nodes from receiving further topology updates. Topology updates should be as prompt as possible, so we should use the NO_FC flag to ensure that each node receives topology updates as soon as possible.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (ISPN-5506) Dist/ReplEmbeddedRestHotRodTest failing randomly with "java.net.SocketException: Socket closed"
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5506?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5506:
----------------------------------
Fix Version/s: 9.0.0.Beta2
(was: 9.0.0.Beta1)
> Dist/ReplEmbeddedRestHotRodTest failing randomly with "java.net.SocketException: Socket closed"
> -----------------------------------------------------------------------------------------------
>
> Key: ISPN-5506
> URL: https://issues.jboss.org/browse/ISPN-5506
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Protocols
> Affects Versions: 7.2.1.Final, 8.0.0.Alpha1
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Fix For: 9.0.0.Beta2
>
> Attachments: infinispan_5506.tgz
>
>
> {code}
> testEmbeddedPutRestHotRodGet(org.infinispan.it.compatibility.DistEmbeddedRestHotRodTest) Time elapsed: 0.002 sec <<< FAILURE!
> java.net.SocketException: Socket closed
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at java.net.Socket.connect(Socket.java:538)
> at java.net.Socket.<init>(Socket.java:434)
> at java.net.Socket.<init>(Socket.java:286)
> at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
> at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
> at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
> at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
> at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
> at org.infinispan.it.compatibility.ReplEmbeddedRestHotRodTest.testEmbeddedPutRestHotRodGet(ReplEmbeddedRestHotRodTest.java:80)
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (ISPN-5572) Exposed JMX MBeans should be separate components
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5572?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5572:
----------------------------------
Fix Version/s: 9.0.0.Beta2
(was: 9.0.0.Beta1)
> Exposed JMX MBeans should be separate components
> ------------------------------------------------
>
> Key: ISPN-5572
> URL: https://issues.jboss.org/browse/ISPN-5572
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 8.0.0.Alpha2, 7.2.3.Final
> Reporter: Dan Berindei
> Fix For: 9.0.0.Beta2
>
>
> We currently expose internal components as JMX MBeans, and that makes our JMX "API" very unstructured. The exposed MBeans should be separate components, and the only concern in their interfaces should be ease of use.
> One example of JMX getting in the way of refactoring is {{CacheMgmtInterceptor}}. The interceptor chain is dynamic, so it should be possible to insert the interceptor only when statistics are enabled. But because the {{statisticsEnabled}} attribute is on the interceptor itself, that becomes a lot trickier, and we had to introduce a separate configuration attribute that disables statistics permanently (ISPN-5542).
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months