[jboss-as7-dev] JTA Synchronization.afterCompletion callback can occur in a background thread but JPA EntityManager must be accessed in single-threaded manner...

[Scott Marlow]

On 02/22/2013 02:53 PM, Jason Greene wrote:
> I had a discussion with Steve about this. Not sure if you guys talked about my suggestion.

I didn't know that but Steve did mention previously that he wanted to 
get your input.

>
> My suggestion was to simply detect if afterCompletion is running on the thread that created it. If the threads match proceed normally. If, however, they do not match then DO NOTHING (well at first):)
>
> Then have the EntityManager, immediately before certain actions such as a flush, detect if the Transaction has been rolled back (using Transaction.getStatus()). If it has been rolled back then at that point of time trigger an exception to the calling application code, with a message that the "Transaction manager has decided to revert the transaction, either due to a timeout or a resource failure. Check your log for details".

Yes, that sounds very familiar! :)

For HHH-7910, we can instrument enough code to do that and cover some 
other cases (replacing possible in-flight exception with an exception 
like your mentioning in the EntityManager method tail).  We also need to 
clear the EntityManager in each method start and tail (after detecting 
that this case previously occurred).

For AS7-6586, when the background thread runs the 
Synchronization.afterCompletion(), the application (transaction origin) 
thread may still be active or it may of completed its use of the 
transaction scoped EntityManager.  When the 
Synchronization.afterCompletion is invoked, the AS JPA container code is 
supposed to call EntityManager.close(), which shouldn't be called while 
another thread is (possibly) in a EntityManager invocation (more likely 
in the tx timeout case).

I started this thread discussion more to discuss the AS7-6586 case and 
ask if anyone knows of any other similar problems with other AS code + 
Synchronization.before/afterCompletion.

If we were to solve AS7-6586, at a higher level (via a container level 
queue), we might want to consider solving HHH-7910 the same way (I 
think).  If AS7-6586 is the only case we need to solve, we could do 
something custom (e.g. have every EntityManager call get an exclusive 
lock or something faster).

>
> On Feb 22, 2013, at 8:45 AM, Scott Marlow <smarlow at redhat.com> wrote:
>
>> Related jira issues are HHH-7910 + AS7-6586.  HHH-7910 has been around a
>> bit longer and contains comments about the current plan to address the
>> issue in Hibernate.
>>
>> As mentioned in HHH-7910, the JTA specification allows
>> Sychronization.afterCompletion callbacks to occur in a different thread
>> than the application thread that is using the transaction (typically for
>> transaction timeout or when a transaction is propagated to a remote thread).
>>
>> Since the JPA EntityManager is by design, not thread safe, invoking the
>> EntityManager.clear() or EntityManager.close() methods from a background
>> thread while the application thread may be in the middle of an
>> EntityManager invocation is not the best situation (IMO).
>>
>> One improvement that we talked to the JBossTM/TS team about, is adding a
>> TM policy that arranges for the Synchronization.afterCompletion to
>> always run in the application thread (some of the IRC discussion is
>> attached to HHH-7910).  The TM team doesn't think that they could do
>> that for both the TX timeout and tx propagating to a remote thread
>> (JBossTS) uses.
>>
>> One alternative solution, might be to create a top level container level
>> queue that the background thread Synchronization.afterCompletion defers
>> processing to.  As soon as the application thread returns control to the
>> top level, the queue is processed in FIFO order.
>>
>> *Does AS have any other uses of Synchronization.afterCompletion or
>> Synchronization.beforeCompletion that expect to run in the application
>> thread*?
>>
>>  From the JPA specification about thread unsafety:
>>
>> "
>> An entity manager must not be shared among multiple concurrently
>> executing threads, as the entity manager and persistence context are not
>> required to be threadsafe. Entity managers must only be accessed in a
>> single-threaded manner.
>> "
>> _______________________________________________
>> jboss-as7-dev mailing list
>> jboss-as7-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> --
> Jason T. Greene
> JBoss AS Lead / EAP Platform Architect
> JBoss, a division of Red Hat
>