[JBoss JIRA] (ISPN-6745) Locks are lost in pessimistic cache
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-6745?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-6745:
------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/4428
> Locks are lost in pessimistic cache
> -----------------------------------
>
> Key: ISPN-6745
> URL: https://issues.jboss.org/browse/ISPN-6745
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.2.3.Final
> Environment: JBoss DataGrid 6.5.0 (6.3.1.Final-redhat-1)
> 3 nodes in REPL_SYNC mode
> pessimistic locking
> read committed isolation
> Reporter: Eugene Scripnik
> Assignee: Pedro Ruivo
> Attachments: InfinispanNodeFailureTest.java
>
>
> When you perform multiple TX write operations in one transaction (put, replace, lock, etc) and one of the nodes goes down, there is a slight chance that some locks will be lost and acquired by another transaction before current transaction ends.
> So client ends up with two transactions holding the same lock on pessimistic cache at the same time. Both transactions commit at the end successfully.
> I spent some time debugging infinispan code and found that PessimisticLockingInterceptor#releaseLocksOnFailureBeforePrepare releases all locks when OutdatedTopologyException occurs on remote node. But then StateTransferInterceptor#handleTxWriteCommand retries last command. This behavior produces inconsistent state - all locks before last command are released and any other transaction can acquire them.
> I am attaching Test which reproduces this problem
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 6 months
[JBoss JIRA] (ISPN-6801) TransactionXaAdapter and SynchronizationAdapter are too big
by Dan Berindei (JIRA)
Dan Berindei created ISPN-6801:
----------------------------------
Summary: TransactionXaAdapter and SynchronizationAdapter are too big
Key: ISPN-6801
URL: https://issues.jboss.org/browse/ISPN-6801
Project: Infinispan
Issue Type: Bug
Components: Core
Affects Versions: 9.0.0.Alpha2
Reporter: Dan Berindei
Assignee: Dan Berindei
Priority: Minor
Fix For: 9.0.0.Alpha3
Most fields in {{TransactionXaAdapter}} and {{SynchronizationAdapter}} can be stored in the {{TransactionTable}}. That would save a bit of memory and CPU for each transaction.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 6 months
[JBoss JIRA] (ISPN-6800) Remove MultipleRpcCommand
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-6800?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-6800:
-------------------------------
Status: Open (was: New)
> Remove MultipleRpcCommand
> -------------------------
>
> Key: ISPN-6800
> URL: https://issues.jboss.org/browse/ISPN-6800
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 9.0.0.Alpha2
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.0.0.Alpha3
>
>
> Having {{MultipleRpcCommand}} makes some things more complex than they should be, e.g. {{CacheRpcCommandExternalizer}} needs a thread-local because of it.
> It was only used by the replication queue, which has been already removed in ISPN-6417. So it's safe to remove it completely and simplify {{CacheRpcCommandExternalizer}}.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 6 months
[JBoss JIRA] (ISPN-6800) Remove MultipleRpcCommand
by Dan Berindei (JIRA)
Dan Berindei created ISPN-6800:
----------------------------------
Summary: Remove MultipleRpcCommand
Key: ISPN-6800
URL: https://issues.jboss.org/browse/ISPN-6800
Project: Infinispan
Issue Type: Task
Components: Core
Affects Versions: 9.0.0.Alpha2
Reporter: Dan Berindei
Assignee: Dan Berindei
Fix For: 9.0.0.Alpha3
Having {{MultipleRpcCommand}} makes some things more complex than they should be, e.g. {{CacheRpcCommandExternalizer}} needs a thread-local because of it.
It was only used by the replication queue, which has been already removed in ISPN-6417. So it's safe to remove it completely and simplify {{CacheRpcCommandExternalizer}}.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 6 months
[JBoss JIRA] (ISPN-6799) OOB thread pool fills with threads trying to send remote get responses
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-6799?page=com.atlassian.jira.plugin.... ]
Dan Berindei reassigned ISPN-6799:
----------------------------------
Assignee: Dan Berindei
> OOB thread pool fills with threads trying to send remote get responses
> ----------------------------------------------------------------------
>
> Key: ISPN-6799
> URL: https://issues.jboss.org/browse/ISPN-6799
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.0.0.Alpha2, 8.2.2.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.0.0.Alpha3
>
>
> Note: This is a scenario that happens in the stress tests, with 4 nodes in dist mode, and 200+ threads per node doing only reads. I have not been able to reproduce it locally, even with a much lower OOB thread pool size and UFC.max_credits.
> We don't use the {{NO_FC}} flag, so threads sending both requests and responses can block in UFC/MFC. Remote gets are executed directly on the OOB thread, so when we run out of credits for one node, the OOB pool can quickly become full with threads waiting to send a remote get response to that node.
> While we can't send responses to that node, we won't send credits to it, either, as credits are only sent *after* the message has been processed by the application. That means OOB threads on all nodes will start blocking, trying to send remote get responses to us.
> This is made a worse by our staggering of remote gets. As remote get responses block, the stagger timeout kicks in and we send even more remote gets, making it even harder for the system to recover.
> UFC/MFC can send a {{CREDIT_REQUEST}} message to ask for more credits. The {{REPLENISH}} messages are handled on JGroups' internal thread pool, so they are not blocked. However, the CREDIT_REQUEST can be sent at most once every {{UFC.max_block_time}} ms, so they can't be relied on to provide enough credits. With the default settings, the throughput would be {{max_credits / max_block_time == 2mb / 0.5s == 4mb/s}}, which is really small compared to regular throughput.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 6 months