[infinispan-dev] PutForExternalRead and autoCommit

[Slorg1]

Hi,
O.K. so here is what I found out (this is linked with the thread
titled "Re: [infinispan-dev] PutForExternalRead and autoCommit", a
copy of this message will be sent on it, though it does not close the
discussion on it).
First, let me remind everyone, that I am using:- Atomikos as a
transaction manager- Hibernate 3.5.5- infinispan 5.1.0 BETA5.- I have
the 2 nodes set up for invalidation on all caches except for the
timestamp cache which is in replication mode (for hibernate).
So, as discussed on the other thread, when autoCommit is set to false
in a transactional cache environment, an exception occurs since in a
few cases, the infinispan system suspends the ongoing transaction. In
this case, when it comes to effectively putting data (whether it is
"forExternalRead" or not, I will not get into that here) it show a
valid exception stating that a transaction is needed for a
transactional cache.
So here is what I had to do to move forward on that. Since I have a
smaller use case that what infinispan can handle, I could safely
remove the code that suspends the transaction where needed to test it
out.
Here is what I did:
1. In CacheImpl, I removed the logic in "final void
putForExternalRead(K key, V value, EnumSet<Flag> explicitFlags,
ClassLoader explicitClassLoader)"on lines 355, 357-359, 376-385
(removed all code linked with the suspension that I did not want given
that I wanted to ensure the autoCommit to false).
That worked and got a HUGE performance improvement, factor of 5 in the
execution/replication/invalidation time (as in literally 5 times
faster).
2. Then I realized that the BaseRegion class also had some logic that
I was running into that was suspending transactions. I changed the
methods "suspend" and "resume(Transaction tx)" to do nothing (suspends
returns systematically null and resume expect always null as an input
with my changes).
This change was what brought me to realize that there was a MAJOR
issue with hibernate (at least in 3.5.5) when doing that.
The timestamp cache/region has a special behaviour in "ActionQueue".
If you look at "AfterTransactionCompletionProcessQueue" the method
"public void afterTransactionCompletion(boolean success)" once all is
done, checks if there is a cache and pushes a set of invalidations.
The main issue here is that:1. my transaction was already successfully
committed by hibernate and these operations are done outside of the
scope of the transaction.2. consequence of #1: I had set autoCommit to
false, so no transactions were auto started for me, so the
invalidations failed systematically (I think it was correct for the
invalidations to fail).
(The changes to "fix" that are enclosed to the email, they are simple).
Thus, this leads me to say that a pure transactional cache with
autoCommit set to false cannot work at this point without the changes
I made to my own. I correctly works for me and after other changes I
got it to perform really well.
On that note, since we are talking performance, I explored the
batching functionality of infinispan (which is great!!). It does not
quite perform as is for all regions/cache. Indeed, if you run into the
issue I describe above in the timestamp cache/region, and your
autoCommit is set to true, no batching happens (you are indeed outside
of the transaction which started the batch) and the
invalidation/replication is super expensive: 2 commands/messages per
invalidation (one for prepare and one for commit). Implementing the
changes I described allowed the batching to work for my timestamp
region and that is when I got an additional 2x factor and everything
is lightening fast. When I say 2x factor, I mean a total of 10x faster
with all the changes I put in versus, the 5.1.0 BETA5 out of the box
(from 1 sec to 100ms per http transaction on my API !!). This is for
asynchronous invalidation, it is slower with synchronous.
While I am at it, I was wondering if it would be logical to have
infinispan automatically start batches in a transactional context. It
made sense to me and I have activated in my own code.
Please find enclosed the files changed.
Let me know if anything I said does not make sense or is wrong. I
tested it and so far it works well but again my use case is a small
subset of what infinispan can do.
Regards,
L.P.
PS: also, org/hibernate/cache/infinispan/util/AddressAdapterImpl.java
and org/hibernate/cache/infinispan/util/CacheHelper.java need public
no-arg constructors for the de serialization to work, they did not
have them in hibernate 3.5.5.
On Wed, Nov 23, 2011 at 08:33, Sanne Grinovero <sanne at infinispan.org> wrote:
> Thanks for clarifying that, it finally makes sense.
>
> On 23 November 2011 13:21, Manik Surtani <manik at jboss.org> wrote:
>> putForExternalRead() is only used when reading from a DB and saving the value in the cache.  So this does not affect updates.  Updates call put(), not pFER(), and put() will cause the tx to abort of the put() fails.
>>
>> On 22 Nov 2011, at 19:15, Sanne Grinovero wrote:
>>
>>> Why is there a general agreement about not aborting the transaction if
>>> we fail to update the cache?
>>>
>>> I can see why when we fail to write it for the first time, but if we
>>> fail to update a value which is stored already that's going to affect
>>> staleness of data read by subsequent transactions.
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Manik Surtani
>> manik at jboss.org
>> twitter.com/maniksurtani
>>
>> Lead, Infinispan
>> http://www.infinispan.org
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



-- 
Please consider the environment - do you really need to print this email ?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: changedFiles.tar.bz2
Type: application/x-bzip2
Size: 14466 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20111129/ac02a96a/attachment.bz2