It's on my TODO list to create a wiki on it: https://issues.jboss.org/browse/ISPN-1120 The javadoc should be clear enough but basically, it's a put operation geared towards best-effort, when you're trying to put somewhere read from an external source. http://www.slideshare.net/galderz/infinispan-for-dummies - slide 12 for characteristics of a PFER as opposed to a Put On Nov 17, 2011, at 9:36 AM, Sanne Grinovero wrote: -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache

The intention is the following: Imagine N threads trying to load the same piece of data from a database into the cache. With a normal put, they'd all be fighting against each other to get the locks for no reason. The idea here is that we want to store something in the cache which we know is not there, but someone, might concurrently try to retrieve it from the db too, so let's one of them do it and the rest, in further interactions, will be able to use the cached value directly. This is different to trying to update an entity. If multiple threads are trying to modify an entity, these need to go through the normal lock procedure. On Nov 17, 2011, at 11:45 AM, Sanne Grinovero wrote: -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache

On Nov 17, 2011, at 2:54 PM, Manik Surtani wrote: 2 does not affect anything but seems wasteful to me. Why start a transaction when I don't need one? -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache

On 17 Nov 2011, at 15:25, Galder Zamarreño wrote: I also think this might be confusing for users. What about: a) don't inject a transaction for PFER. That's an easy change. b) if autoCommit is off and PFER is not invoked in the scope of a tx we can either warn or throw an exception. Or just ignore. The problem is that we inject the transaction twice.

On 22 Nov 2011, at 19:13, Slorg1 wrote: autoCommit was added mainly for backward compatibility, as there are user's accessing the cache in a mixed mode. It is also an API-sugar for single-write transactions, as one doesn't have to explicitly write then begin/commit. no necessarily - just me trying to get to the root of the problem :)

On Nov 17, 2011, at 4:39 PM, Slorg1 wrote: It's an optimisation since we're reading from the external source of data. This optimisation avoids locking issues as a result of trying to read the same piece of data into the cache when executed concurrently in different nodes. Read the putForExternal read javadoc to find out more. Again, we're reading from the source of data and putting it in the cache. The source of data is the database which contains committed data, so no probs here. It'd be interesting to know if this drop in performance is purely down to this, or there are more factors involved. -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache

On 17 Nov 2011, at 15:39, Slorg1 wrote: putForExternalRead(PFER) is an operation that has a special semantic different than put - and that's what would do, i.e. enlist in the calling transaction. One of reasons PFER was added was to try and do the put, but if not possible the ongoing transaction should not be affected.

putForExternalRead() is only used when reading from a DB and saving the value in the cache. So this does not affect updates. Updates call put(), not pFER(), and put() will cause the tx to abort of the put() fails. On 22 Nov 2011, at 19:15, Sanne Grinovero wrote: -- Manik Surtani manik(a)jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org

Hi, O.K. so here is what I found out (this is linked with the thread titled "Re: [infinispan-dev] PutForExternalRead and autoCommit", a copy of this message will be sent on it, though it does not close the discussion on it). First, let me remind everyone, that I am using:- Atomikos as a transaction manager- Hibernate 3.5.5- infinispan 5.1.0 BETA5.- I have the 2 nodes set up for invalidation on all caches except for the timestamp cache which is in replication mode (for hibernate). So, as discussed on the other thread, when autoCommit is set to false in a transactional cache environment, an exception occurs since in a few cases, the infinispan system suspends the ongoing transaction. In this case, when it comes to effectively putting data (whether it is "forExternalRead" or not, I will not get into that here) it show a valid exception stating that a transaction is needed for a transactional cache. So here is what I had to do to move forward on that. Since I have a smaller use case that what infinispan can handle, I could safely remove the code that suspends the transaction where needed to test it out. Here is what I did: 1. In CacheImpl, I removed the logic in "final void putForExternalRead(K key, V value, EnumSet<Flag> explicitFlags, ClassLoader explicitClassLoader)"on lines 355, 357-359, 376-385 (removed all code linked with the suspension that I did not want given that I wanted to ensure the autoCommit to false). That worked and got a HUGE performance improvement, factor of 5 in the execution/replication/invalidation time (as in literally 5 times faster). 2. Then I realized that the BaseRegion class also had some logic that I was running into that was suspending transactions. I changed the methods "suspend" and "resume(Transaction tx)" to do nothing (suspends returns systematically null and resume expect always null as an input with my changes). This change was what brought me to realize that there was a MAJOR issue with hibernate (at least in 3.5.5) when doing that. The timestamp cache/region has a special behaviour in "ActionQueue". If you look at "AfterTransactionCompletionProcessQueue" the method "public void afterTransactionCompletion(boolean success)" once all is done, checks if there is a cache and pushes a set of invalidations. The main issue here is that:1. my transaction was already successfully committed by hibernate and these operations are done outside of the scope of the transaction.2. consequence of #1: I had set autoCommit to false, so no transactions were auto started for me, so the invalidations failed systematically (I think it was correct for the invalidations to fail). (The changes to "fix" that are enclosed to the email, they are simple). Thus, this leads me to say that a pure transactional cache with autoCommit set to false cannot work at this point without the changes I made to my own. I correctly works for me and after other changes I got it to perform really well. On that note, since we are talking performance, I explored the batching functionality of infinispan (which is great!!). It does not quite perform as is for all regions/cache. Indeed, if you run into the issue I describe above in the timestamp cache/region, and your autoCommit is set to true, no batching happens (you are indeed outside of the transaction which started the batch) and the invalidation/replication is super expensive: 2 commands/messages per invalidation (one for prepare and one for commit). Implementing the changes I described allowed the batching to work for my timestamp region and that is when I got an additional 2x factor and everything is lightening fast. When I say 2x factor, I mean a total of 10x faster with all the changes I put in versus, the 5.1.0 BETA5 out of the box (from 1 sec to 100ms per http transaction on my API !!). This is for asynchronous invalidation, it is slower with synchronous. While I am at it, I was wondering if it would be logical to have infinispan automatically start batches in a transactional context. It made sense to me and I have activated in my own code. Please find enclosed the files changed. Let me know if anything I said does not make sense or is wrong. I tested it and so far it works well but again my use case is a small subset of what infinispan can do. Regards, L.P. PS: also, org/hibernate/cache/infinispan/util/AddressAdapterImpl.java and org/hibernate/cache/infinispan/util/CacheHelper.java need public no-arg constructors for the de serialization to work, they did not have them in hibernate 3.5.5. On Wed, Nov 23, 2011 at 08:33, Sanne Grinovero <sanne(a)infinispan.org&gt; wrote: -- Please consider the environment - do you really need to print this email ?