[JBoss JIRA] (ISPN-2081) Transaction leak caused by reordering between prepare and commit
by Mircea Markus (JIRA)
Mircea Markus created ISPN-2081:
-----------------------------------
Summary: Transaction leak caused by reordering between prepare and commit
Key: ISPN-2081
URL: https://issues.jboss.org/browse/ISPN-2081
Project: Infinispan
Issue Type: Bug
Components: Transactions
Affects Versions: 5.1.5.FINAL
Reporter: Mircea Markus
Assignee: Mircea Markus
Fix For: 5.2.0.ALPHA1, 5.2.0.FINAL
Attachments: RollbackNotSentBeforePrepareTest.java
There's no ordering between the prepare and commit/rollback messages, as the later are sent OOB.
With this in mind, the following transaction leak might happen:
Tx1 send prepare on nodes {A,B}
1. the message reaches A and timeouts but hasn't yet been processed on B
2. The transaction originator reacts immediately to the timeout received from A without waiting the response from B and sends a rollback request
3. The rollback request is processed on A and B
4. The initial prepare is then processed on B
At this point we have an orphan transaction prepare on B.
Whilst this is not causing any inconsistencies, it keeps keys locked indefinitely and is a memory leak.
The solution would be to wait at 2 for all the prepare messages *before* sending the rollback.
Attached is a unit test to reproduce the issue.
Related mailing list thread: http://infinispan.markmail.org/search/#query:%20list%3Aorg.jboss.lists.in...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month
[JBoss JIRA] (ISPN-2187) Pre-Invocation flag PUT_FOR_EXTERNAL_READ throws exception
by Anna Manukyan (JIRA)
Anna Manukyan created ISPN-2187:
-----------------------------------
Summary: Pre-Invocation flag PUT_FOR_EXTERNAL_READ throws exception
Key: ISPN-2187
URL: https://issues.jboss.org/browse/ISPN-2187
Project: Infinispan
Issue Type: Bug
Components: Core API
Reporter: Anna Manukyan
Assignee: Manik Surtani
Hi,
While writing tests for Infinispan Flag.PUT_FOR_EXTERNAL_READ the following issue has been found.
In documentation it is said:
PUT_FOR_EXTERNAL_READ
Flags the invocation as a Cache.putForExternalRead(Object, Object) call, as opposed to a regular Map.put(Object, Object).
And the documentation for Cache.putForExternalRead(Object, Object) says:
void putForExternalRead(K key,
V value)
...................
Errors and exceptions are 'silent' - logged at a much lower level than normal, and this method does not throw exceptions
The issue is the following:
when trying to perform operation using PUT_FOR_EXTERNAL_READ flag, the exception is thrown, it is not 'silent'.
cache1.getAdvancedCache().withFlags(Flag.PUT_FOR_EXTERNAL_READ).put(key, value);
The test is the following:
public void testExceptionSuppression() throws Exception {
Cache cache1 = cache(0, "replSync");
Cache cache2 = cache(1, "replSync");
Transport mockTransport = mock(Transport.class);
RpcManagerImpl rpcManager = (RpcManagerImpl) TestingUtil.extractComponent(cache1, RpcManager.class);
Transport originalTransport = TestingUtil.extractComponent(cache1, Transport.class);
try {
Address mockAddress1 = mock(Address.class);
Address mockAddress2 = mock(Address.class);
List<Address> memberList = new ArrayList<Address>(2);
memberList.add(mockAddress1);
memberList.add(mockAddress2);
rpcManager.setTransport(mockTransport);
when(mockTransport.getMembers()).thenReturn(memberList);
when(mockTransport.getViewId()).thenReturn(originalTransport.getViewId());
when(mockTransport.invokeRemotely(anyAddresses(), (CacheRpcCommand) anyObject(), anyResponseMode(),
anyLong(), anyBoolean(), (ResponseFilter) anyObject()))
.thenThrow(new RuntimeException("Barf!"));
try {
cache1.put(key, value);
fail("Should have barfed");
}
catch (RuntimeException re) {
}
// clean up any indeterminate state left over
try {
cache1.remove(key);
fail("Should have barfed");
}
catch (RuntimeException re) {
}
assertNull("Should have cleaned up", cache1.get(key));
// should not barf
cache1.putForExternalRead(key, value);
/** ------------------- Testing the same feature with Flag.PUT_FOR_EXTERNAL_READ **/
try {
cache1.remove(key);
fail("Should have barfed");
}
catch (RuntimeException re) {
}
cache1.getAdvancedCache().withFlags(Flag.PUT_FOR_EXTERNAL_READ).put(key, value);
}
finally {
if (rpcManager != null) rpcManager.setTransport(originalTransport);
}
}
Best regards,
Anna.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month
[JBoss JIRA] (ISPN-2302) New JdbcCacheStoreConfigurationBuilder should support existing ConnectionFactoryConfig
by Thomas Fromm (JIRA)
Thomas Fromm created ISPN-2302:
----------------------------------
Summary: New JdbcCacheStoreConfigurationBuilder should support existing ConnectionFactoryConfig
Key: ISPN-2302
URL: https://issues.jboss.org/browse/ISPN-2302
Project: Infinispan
Issue Type: Feature Request
Components: Configuration
Affects Versions: 5.2.0.Alpha3
Reporter: Thomas Fromm
Assignee: Mircea Markus
I've the situation the I need to reuse existing ConnectionFactoryConfigs.
Maybe implement ConnectionFactoryConfigurationBuilder with method read(ConnectionFactoryConfig template). Then extend JdbcCacheStoreConfigurationBuilder and add connectionFactory()?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month
[JBoss JIRA] (ISPN-2335) MultiHotRodServersTest not works with ConfigurationBuilder API
by Thomas Fromm (JIRA)
Thomas Fromm created ISPN-2335:
----------------------------------
Summary: MultiHotRodServersTest not works with ConfigurationBuilder API
Key: ISPN-2335
URL: https://issues.jboss.org/browse/ISPN-2335
Project: Infinispan
Issue Type: Bug
Components: Test Suite
Affects Versions: 5.2.0.Alpha4
Reporter: Thomas Fromm
Assignee: Tristan Tarrant
Attachments: trace-infinispan.log
I tried to use ConfigurationBuilder API with addClusterEnabledCacheManager inside MultiHotRodServersTest.addHotRodServer.
Unfortunality it works on with the old Configuration object, with the Builder the test fails. I tried to find any relevant difference inside configuration, but wasn't successful :-(
So I use a LegacyConfigurationAdaptor to create old config.
Attachmed I've traces from the failing tests and from the working one. Maybe it helps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month
[JBoss JIRA] (ISPN-2408) Apply state results in ClasCastException due to context not being transactional
by Adrian Nistor (JIRA)
Adrian Nistor created ISPN-2408:
-----------------------------------
Summary: Apply state results in ClasCastException due to context not being transactional
Key: ISPN-2408
URL: https://issues.jboss.org/browse/ISPN-2408
Project: Infinispan
Issue Type: Bug
Components: State transfer
Affects Versions: 5.2.0.Beta2
Reporter: Adrian Nistor
Assignee: Adrian Nistor
Priority: Blocker
Fix For: 5.2.0.CR1
The putIfAbsent command executed for applying state received from neighbours during state transfer used to be done with SKIP_LOCKING flag in 5.2.Beta1.
Considering that putIfAbsent cannot provide sufficient guarantees without locking I removed the SKIP_LOCKING flag in 5.2.Beta2.
Unfortunately this means that the non-Tx remote context we use for apply state is no longer suitable. PessimisticLockingInterceptor expects a transactional context and this causes all puts to fail and log an exception. State transfer is basically not able to transfer any data for pessimistic caches.
A quick workaround is to put back SKIP_LOCKING from where it was removed. See line 298: https://github.com/infinispan/infinispan/blob/7ee1ce5a186928b740f4cf9f375...
But this workaround is not correct. SKIP_LOCKING may cause uncertain results with putIfAbsent.
{noformat}
2012-10-15 19:46:35,680 WARN [StateConsumerImpl] (OOB-1,ISPN,NodeC-26225) ISPN000016: Problem org.infinispan.context.impl.NonTxInvocationContext cannot be cast to org.infinispan.context.impl.TxInvocationContext encountered when applying state for key MagicKey{hashcode=153290881, address='NodeA-9763'}!
2012-10-15 19:46:35,680 TRACE [InvocationContextInterceptor] (OOB-1,ISPN,NodeC-26225) Invoked with command PutKeyValueCommand{key=MagicKey{hashcode=-764090028, address='NodeA-9763'}, value=MagicKey{hashcode=-764090028, address='NodeA-9763'}, flags=[CACHE_MODE_LOCAL, SKIP_SHARED_CACHE_STORE, SKIP_OWNERSHIP_CHECK, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=true, lifespanMillis=-1, maxIdleTimeMillis=-1} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@625cb0bb]
2012-10-15 19:46:35,680 ERROR [InvocationContextInterceptor] (OOB-1,ISPN,NodeC-26225) ISPN000136: Execution error
java.lang.ClassCastException: org.infinispan.context.impl.NonTxInvocationContext cannot be cast to org.infinispan.context.impl.TxInvocationContext
at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPutKeyValueCommand(PessimisticLockingInterceptor.java:114)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:63)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:212)
at org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:150)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:203)
at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:189)
at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:135)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:129)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:93)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:63)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:347)
at org.infinispan.statetransfer.StateConsumerImpl.doApplyState(StateConsumerImpl.java:306)
at org.infinispan.statetransfer.StateConsumerImpl.applyState(StateConsumerImpl.java:264)
at org.infinispan.statetransfer.StateResponseCommand.perform(StateResponseCommand.java:86)
at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:95)
at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:110)
at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:82)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:244)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:217)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:483)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:390)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:248)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:604)
at org.jgroups.JChannel.up(JChannel.java:670)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
at org.jgroups.protocols.FC.up(FC.java:479)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)
at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:736)
at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:414)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:601)
at org.jgroups.protocols.Discovery.up(Discovery.java:359)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1293)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1856)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1829)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month
[JBoss JIRA] (ISPN-2423) Handle the site unreachable case when sending unicasts to backup sites.
by Erik Salter (JIRA)
Erik Salter created ISPN-2423:
---------------------------------
Summary: Handle the site unreachable case when sending unicasts to backup sites.
Key: ISPN-2423
URL: https://issues.jboss.org/browse/ISPN-2423
Project: Infinispan
Issue Type: Bug
Components: Cross-Site Replication
Affects Versions: 5.2.0.Beta2
Reporter: Erik Salter
Assignee: Mircea Markus
When the BackupInterceptors send a unicast to an unreachable site, the underlying RELAY2 protocol can be configured to retry sending the message.
The current implementation iterates over sites and sequentially tries to send a message. If one of the sites is unreachable, it will block waiting for a response. For instance, if in sites {A,B,C} and A is down, we can block on waiting for the response from A. As this is in the critical path for transactions, locks, etc., this affects the overall throughput of the system.
Note that this happens for async and sync replication.
A potential solution is in JGroupsTransport.backupRemotely(), we create a Runnable which calls MessageDispatcher.sendWithFuture() and submit it to the internal asyncExecutor thread pool. For async replication, the responses can then be processed (and sites taken offline, etc) outside the tx critical path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months