[JBoss JIRA] (JBTM-2394) Transaction Reaper "abuses" in-use connection, leading to RPC protocol corruption
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-2394?page=com.atlassian.jira.plugin.... ]
Tom Jenkinson resolved JBTM-2394.
---------------------------------
Resolution: Rejected
> Transaction Reaper "abuses" in-use connection, leading to RPC protocol corruption
> ---------------------------------------------------------------------------------
>
> Key: JBTM-2394
> URL: https://issues.jboss.org/browse/JBTM-2394
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Reporter: Christian von Kutzleben
> Assignee: Tom Jenkinson
>
> The scenario is as follows:
> An regular JBoss application thread uses the connection associated with an XAResource for it's work with the database backend. Start(xid) had been invoked, which means, the associated connection is "switched" to that xid. After finishing work, the regular JBoss application thread would eventually call end(xid, TMSUCCESS).
> The application executes a query that takes longer than the configured timeout.
> The work with the database is done via database specific RPC invocations, caused by a enterprise bean (which uses the JPA API and the JPA implementation eventually talks to the database via the database protocol).
> The (lower-level) connection is a TCP/IP connection, and
> the RPC protocol if of the form: "send data to server, then wait for server reply"
> At this point it should be clear, that the TCP/IP connection should not be shared by another thread whilst in the middle of an RPC invocation, because neither does the database server expect any data at this moment on this particular TCP/IP connection nor does it work, that then 2 threads wait on the same socket to receive a reply. (E.g. a ClosedByInterruptException is likely, there might be other error though, depending how wrong data is interpreted, e.g. BufferUnderflowException).
> Unfortunately, this is exactly the behavior of the JBoss "Transaction Reaper" thread, that uses the very same XAResource, that had been used by the regular JBoss application thread and is currently associated to a connection, "switched" to the current xid.
> By definition of the XA specification, any other XAResource could be used to terminate that transaction branch, and it would be fine, if the transaction reaper thread would use any XAResource (regular one, or one from our recovery module) to do that.
> This is a conceptual flaw of the transaction reaper implementation, and we can't implement a workaround (except extremely silly things like reaper thread recognition by comparing thread names ...)
> We could synchronize access to the TCP/IP connection, to not allow another thread communication, while another thread is active doing so, however, that kind of defeats the purpose of the reaper thread, as it would be blocked indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 1 month
[JBoss JIRA] (JBTM-2394) Transaction Reaper "abuses" in-use connection, leading to RPC protocol corruption
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-2394?page=com.atlassian.jira.plugin.... ]
Tom Jenkinson commented on JBTM-2394:
-------------------------------------
I think you should open a discussion about this because the JTA spec allows for the behavior you are observing (3.4.3) so We can probably assist you better as a discussion.
For example, @mmusgrov recently put together something to help JPA who did not want their synchronization called from a concurrent thread which might help you: JBTM-2342 and JBTM-2343 That being said, it is generally desirable that XAResources are rolled back immediately to avoid holding locks longer than necessary.
> Transaction Reaper "abuses" in-use connection, leading to RPC protocol corruption
> ---------------------------------------------------------------------------------
>
> Key: JBTM-2394
> URL: https://issues.jboss.org/browse/JBTM-2394
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Reporter: Christian von Kutzleben
> Assignee: Tom Jenkinson
>
> The scenario is as follows:
> An regular JBoss application thread uses the connection associated with an XAResource for it's work with the database backend. Start(xid) had been invoked, which means, the associated connection is "switched" to that xid. After finishing work, the regular JBoss application thread would eventually call end(xid, TMSUCCESS).
> The application executes a query that takes longer than the configured timeout.
> The work with the database is done via database specific RPC invocations, caused by a enterprise bean (which uses the JPA API and the JPA implementation eventually talks to the database via the database protocol).
> The (lower-level) connection is a TCP/IP connection, and
> the RPC protocol if of the form: "send data to server, then wait for server reply"
> At this point it should be clear, that the TCP/IP connection should not be shared by another thread whilst in the middle of an RPC invocation, because neither does the database server expect any data at this moment on this particular TCP/IP connection nor does it work, that then 2 threads wait on the same socket to receive a reply. (E.g. a ClosedByInterruptException is likely, there might be other error though, depending how wrong data is interpreted, e.g. BufferUnderflowException).
> Unfortunately, this is exactly the behavior of the JBoss "Transaction Reaper" thread, that uses the very same XAResource, that had been used by the regular JBoss application thread and is currently associated to a connection, "switched" to the current xid.
> By definition of the XA specification, any other XAResource could be used to terminate that transaction branch, and it would be fine, if the transaction reaper thread would use any XAResource (regular one, or one from our recovery module) to do that.
> This is a conceptual flaw of the transaction reaper implementation, and we can't implement a workaround (except extremely silly things like reaper thread recognition by comparing thread names ...)
> We could synchronize access to the TCP/IP connection, to not allow another thread communication, while another thread is active doing so, however, that kind of defeats the purpose of the reaper thread, as it would be blocked indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 1 month
[JBoss JIRA] (JBTM-2395) Correct Javadoc errors on JDK8
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-2395?page=com.atlassian.jira.plugin.... ]
Tom Jenkinson commented on JBTM-2395:
-------------------------------------
The PR does disable the assembly of the javadocs into the distribution. Not sure if we want to keep that as we don't package the rest of the docs...
> Correct Javadoc errors on JDK8
> ------------------------------
>
> Key: JBTM-2395
> URL: https://issues.jboss.org/browse/JBTM-2395
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Components: Build System
> Reporter: Tom Jenkinson
> Assignee: Tom Jenkinson
> Fix For: 5.next
>
>
> WFLY has updated to JDK8, this means due to our build approach (early integration testing and BlackTie) we need to make sure we are compatible with JDK8. Currently there are Javadoc errors.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 1 month
[JBoss JIRA] (JBTM-2396) Investigate whether OCC implementation is unexpectedly checking locks during @WriteLock methods rather than delaying to commit time
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-2396?page=com.atlassian.jira.plugin.... ]
Tom Jenkinson updated JBTM-2396:
--------------------------------
Attachment: OptimisticLockUnitTest.java
I have attached a test that should show the error.
> Investigate whether OCC implementation is unexpectedly checking locks during @WriteLock methods rather than delaying to commit time
> -----------------------------------------------------------------------------------------------------------------------------------
>
> Key: JBTM-2396
> URL: https://issues.jboss.org/browse/JBTM-2396
> Project: JBoss Transaction Manager
> Issue Type: Task
> Components: STM
> Reporter: Tom Jenkinson
> Assignee: Mark Little
> Attachments: OptimisticLockUnitTest.java
>
>
> While checking out the STM work I tried to observe the difference between the OCC and PCC variants of the framework. During my testing I was not able to distinguish a difference between the two implementations.
> Consider the following test:
> {code}
> // gonna confess here, I am using two containers as I thought it might separate better but I don't actually think this is required to use the two containers to get the separation as the two transactions should I create should be doing that?
> Container<Atomic> theContainer1 = new Container<Atomic>();
> Container<Atomic> theContainer2 = new Container<Atomic>();
> // Setting up the two references to the STM object
> final Atomic obj1 = theContainer1.create(new ExampleSTM());
> final Atomic obj2 = theContainer2.clone(new ExampleSTM(), obj1);
> // Creating a transaction and calling the @WriteLock method set() on it but don't commit the tx
> AtomicAction a = new AtomicAction();
> a.begin();
> obj1.set(1234); // Don't commit this yet - I want to check that a conflicting set() doesn't get the lockexception until commit()
> // Setting up a second independent transaction and calling the @WriteLock method set() on it again
> AtomicAction.suspend();
> AtomicAction b = new AtomicAction();
> b.begin();
> // Now, in my understanding calling this method should only throw a LockException for pessimistic locking, but with the current implementation the Lock throws a conflict now
> obj2.set(12345); // This throws an exception even for @Optimisitic
> // the rest of the test commits the two txs, but IMO as I am using @Optimistic I should not have got the LockException on the second set() so the test failed
> {code}
> Whether the Atomic interface is annotated with either @Optimisitic (or ommitted/default @Pessimisitc) the second @WriteLock set(int) call results in a lockexception being thrown. This is contrary to my expectations for OCC where I would expect to observe the set() being allowed but validation to be performed during commit and the commit to fail.
> From my reading of STM/src/main/java/org/jboss/stm/internal/reflect/InvocationHandler.java I can see that this class (nor any of the other main classes) uses the OptimisticLock (which appears to suppress lock conflicts). I can see it uses OptimisticLockRecord which may do the validation correctly but my test fails because the second set(int) is allowed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 1 month
[JBoss JIRA] (JBTM-2396) Investigate whether OCC implementation is unexpectedly checking locks during @WriteLock methods rather than delaying to commit time
by Tom Jenkinson (JIRA)
Tom Jenkinson created JBTM-2396:
-----------------------------------
Summary: Investigate whether OCC implementation is unexpectedly checking locks during @WriteLock methods rather than delaying to commit time
Key: JBTM-2396
URL: https://issues.jboss.org/browse/JBTM-2396
Project: JBoss Transaction Manager
Issue Type: Task
Components: STM
Reporter: Tom Jenkinson
Assignee: Mark Little
While checking out the STM work I tried to observe the difference between the OCC and PCC variants of the framework. During my testing I was not able to distinguish a difference between the two implementations.
Consider the following test:
{code}
// gonna confess here, I am using two containers as I thought it might separate better but I don't actually think this is required to use the two containers to get the separation as the two transactions should I create should be doing that?
Container<Atomic> theContainer1 = new Container<Atomic>();
Container<Atomic> theContainer2 = new Container<Atomic>();
// Setting up the two references to the STM object
final Atomic obj1 = theContainer1.create(new ExampleSTM());
final Atomic obj2 = theContainer2.clone(new ExampleSTM(), obj1);
// Creating a transaction and calling the @WriteLock method set() on it but don't commit the tx
AtomicAction a = new AtomicAction();
a.begin();
obj1.set(1234); // Don't commit this yet - I want to check that a conflicting set() doesn't get the lockexception until commit()
// Setting up a second independent transaction and calling the @WriteLock method set() on it again
AtomicAction.suspend();
AtomicAction b = new AtomicAction();
b.begin();
// Now, in my understanding calling this method should only throw a LockException for pessimistic locking, but with the current implementation the Lock throws a conflict now
obj2.set(12345); // This throws an exception even for @Optimisitic
// the rest of the test commits the two txs, but IMO as I am using @Optimistic I should not have got the LockException on the second set() so the test failed
{code}
Whether the Atomic interface is annotated with either @Optimisitic (or ommitted/default @Pessimisitc) the second @WriteLock set(int) call results in a lockexception being thrown. This is contrary to my expectations for OCC where I would expect to observe the set() being allowed but validation to be performed during commit and the commit to fail.
>From my reading of STM/src/main/java/org/jboss/stm/internal/reflect/InvocationHandler.java I can see that this class (nor any of the other main classes) uses the OptimisticLock (which appears to suppress lock conflicts). I can see it uses OptimisticLockRecord which may do the validation correctly but my test fails because the second set(int) is allowed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 1 month
[JBoss JIRA] (JBTM-2395) Correct Javadoc errors on JDK8
by Tom Jenkinson (JIRA)
Tom Jenkinson created JBTM-2395:
-----------------------------------
Summary: Correct Javadoc errors on JDK8
Key: JBTM-2395
URL: https://issues.jboss.org/browse/JBTM-2395
Project: JBoss Transaction Manager
Issue Type: Bug
Components: Build System
Reporter: Tom Jenkinson
Assignee: Tom Jenkinson
Fix For: 5.next
WFLY has updated to JDK8, this means due to our build approach (early integration testing and BlackTie) we need to make sure we are compatible with JDK8. Currently there are Javadoc errors.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 1 month
[JBoss JIRA] (JBTM-2394) Transaction Reaper "abuses" in-use connection, leading to RPC protocol corruption
by Christian von Kutzleben (JIRA)
[ https://issues.jboss.org/browse/JBTM-2394?page=com.atlassian.jira.plugin.... ]
Christian von Kutzleben updated JBTM-2394:
------------------------------------------
Description:
The scenario is as follows:
An regular JBoss application thread uses the connection associated with an XAResource for it's work with the database backend. Start(xid) had been invoked, which means, the associated connection is "switched" to that xid. After finishing work, the regular JBoss application thread would eventually call end(xid, TMSUCCESS).
The application executes a query that takes longer than the configured timeout.
The work with the database is done via database specific RPC invocations, caused by a enterprise bean (which uses the JPA API and the JPA implementation eventually talks to the database via the database protocol).
The (lower-level) connection is a TCP/IP connection, and
the RPC protocol if of the form: "send data to server, then wait for server reply"
At this point it should be clear, that the TCP/IP connection should not be shared by another thread whilst in the middle of an RPC invocation, because neither does the database server expect any data at this moment on this particular TCP/IP connection nor does it work, that then 2 threads wait on the same socket to receive a reply. (E.g. a ClosedByInterruptException is likely, there might be other error though, depending how wrong data is interpreted, e.g. BufferUnderflowException).
Unfortunately, this is exactly the behavior of the JBoss "Transaction Reaper" thread, that uses the very same XAResource, that had been used by the regular JBoss application thread and is currently associated to a connection, "switched" to the current xid.
By definition of the XA specification, any other XAResource could be used to terminate that transaction branch, and it would be fine, if the transaction reaper thread would use any XAResource (regular one, or one from our recovery module) to do that.
This is a conceptual flaw of the transaction reaper implementation, and we can't implement a workaround (except extremely silly things like reaper thread recognition by comparing thread names ...)
We could synchronize access to the TCP/IP connection, to not allow another thread communication, while another thread is active doing so, however, that kind of defeats the purpose of the reaper thread, as it would be blocked indefinitely.
was:
The scenario is as follows:
An regular JBoss application thread uses the connection associated with an XAResource for it's work with the database backend. Start(xid) had been invoked, which means, the associated connection is "switched" to that xid. After finishing work, the regular JBoss application thread would eventually call end(xid, TMSUCCESS).
The application executes a query that takes longer than the configured timeout.
The work with the database is done via database specific RPC invocations, caused by a enterprise bean (which uses the JPA API and the JPA implementation eventually talks to the database via the database protocol).
The (lower-level) connection is --unsurprisingly-- a TCP/IP connection, and
the RPC protocol --also unsurprisingly-- if of the form: "send data to server, then wait for server reply"
At this point it should be clear, that the TCP/IP connection should not be shared by another thread whilst in the middle of an RPC invocation, because neither does the database server expect any data at this moment on this particular TCP/IP connection nor does it work, that then 2 threads wait on the same socket to receive a reply. (E.g. a ClosedByInterruptException is likely, there might be other error though, depending how wrong data is interpreted, e.g. BufferUnderflowException).
Unfortunately, this is exactly the behavior of the JBoss "Transaction Reaper" thread, that uses the very same XAResource, that had been used by the regular JBoss application thread and is currently associated to a connection, "switched" to the current xid.
By definition of the XA specification, any other XAResource could be used to terminate that transaction branch, and it would be fine, if the transaction reaper thread would use any XAResource (regular one, or one from our recovery module) to do that.
This is a conceptual flaw of the transaction reaper implementation, and we can't implement a workaround (except extremely silly things like reaper thread recognition by comparing thread names ...)
We could synchronized access to the TCP/IP connection, to not allow another thread communication, while another thread is active doing so, however, that kind of defeats the purpose of the reaper thread, as it would be blocked indefinitely.
> Transaction Reaper "abuses" in-use connection, leading to RPC protocol corruption
> ---------------------------------------------------------------------------------
>
> Key: JBTM-2394
> URL: https://issues.jboss.org/browse/JBTM-2394
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Reporter: Christian von Kutzleben
> Assignee: Tom Jenkinson
>
> The scenario is as follows:
> An regular JBoss application thread uses the connection associated with an XAResource for it's work with the database backend. Start(xid) had been invoked, which means, the associated connection is "switched" to that xid. After finishing work, the regular JBoss application thread would eventually call end(xid, TMSUCCESS).
> The application executes a query that takes longer than the configured timeout.
> The work with the database is done via database specific RPC invocations, caused by a enterprise bean (which uses the JPA API and the JPA implementation eventually talks to the database via the database protocol).
> The (lower-level) connection is a TCP/IP connection, and
> the RPC protocol if of the form: "send data to server, then wait for server reply"
> At this point it should be clear, that the TCP/IP connection should not be shared by another thread whilst in the middle of an RPC invocation, because neither does the database server expect any data at this moment on this particular TCP/IP connection nor does it work, that then 2 threads wait on the same socket to receive a reply. (E.g. a ClosedByInterruptException is likely, there might be other error though, depending how wrong data is interpreted, e.g. BufferUnderflowException).
> Unfortunately, this is exactly the behavior of the JBoss "Transaction Reaper" thread, that uses the very same XAResource, that had been used by the regular JBoss application thread and is currently associated to a connection, "switched" to the current xid.
> By definition of the XA specification, any other XAResource could be used to terminate that transaction branch, and it would be fine, if the transaction reaper thread would use any XAResource (regular one, or one from our recovery module) to do that.
> This is a conceptual flaw of the transaction reaper implementation, and we can't implement a workaround (except extremely silly things like reaper thread recognition by comparing thread names ...)
> We could synchronize access to the TCP/IP connection, to not allow another thread communication, while another thread is active doing so, however, that kind of defeats the purpose of the reaper thread, as it would be blocked indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 1 month