[infinispan-dev] transaction over hotrod

Wed Jul 21 09:49:10 EDT 2010

On Jul 21, 2010, at 3:36 PM, Galder Zamarreño wrote:

> 
> On Jul 21, 2010, at 3:09 PM, Mircea Markus wrote:
> 
>> 
>> On 21 Jul 2010, at 14:03, Galder Zamarreño wrote:
>> 
>>> 
>>> On Jul 21, 2010, at 12:35 PM, Mircea Markus wrote:
>>> 
>>>> 
>>>> On 21 Jul 2010, at 09:36, Galder Zamarreño wrote:
>>>> 
>>>>> 
>>>>> On Jul 20, 2010, at 6:14 PM, Mircea Markus wrote:
>>>>> 
>>>>>> 
>>>>>> On 20 Jul 2010, at 14:07, Galder Zamarreño wrote:
>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> First of all, thanks Mircea for writing this up. These are my comments:
>>>>>>> 
>>>>>>> I'm not sure I understand the meaning of or the point you're trying to make with: "Further on, it is  possible (and not difficult) to build local transactions on top of global transaction: Infinispan's batching API does just that internally.".
>>>>>> Infinispan batching is a way to execute local transactions. 
>>>>>> cache.startBatch(); //start local transactions
>>>>>> //do stuff
>>>>>> cache.endBatch(true); //this would commit/rollback the local transaction
>>>>>> Batching functionality (i.e. local transaction) is implemented by starting a JTA transaction(i.e. global transaction) within BatchingInterceptor.
>>>>> 
>>>>> Sure, but what I'm not sure I understand this in the context of the section. You're trying to decide whether Hot Rod should use global or local transactions. It's clear that it needs global transactions, or JTA transactions. I think talking about batching confuses things here.
>>>> The point I want to make is that local transactions can be easily achieved once global transactions are in place(batching API are local transactions really).  JTA spec does not enforce a resource adapter to implement local transactions, but it encourages it. So we can do it at a further point if needed.
>>> 
>>> Ok, I get it now.
>>> 
>>>>> Batching AFAIK should be used to batch replication messages independent of JTA transactions (http://community.jboss.org/wiki/Batching).
>>>> yes, they are local transactions and should be used when you don't have an TransactionManager configured, or when you simply don't participate in distributed transactions.
>>> 
>>> It might be worth adding a note to the http://community.jboss.org/docs/DOC-14901 wiki indicating that when "   cache.endBatch(true); // This will now replicate the modifications since the batch was started." occurs, the semantics are not the same as if a JTA tx was in use.
>>> IOW, there're won't be a 2PC guaranteeing the replication...etc. Correct?
>> There will be a 2PC actually:  first a multicast to acquire the locks(PrepareCommand); if all this succeeds commit is multicasted to the list of nodes(CommitCommand).
> 
> When you talk about local transactions, you refer to 1PC in http://community.jboss.org/wiki/TransactionsOverHotRod. And batching is done with local transactions. So, my logic says that batching is 1PC unless I'm missing something.
> 
> If batching does 2PC, what's the difference with global/distributed transactions?

Aside from these questions, I think http://community.jboss.org/docs/DOC-14901 could do with having more information. Brian's http://community.jboss.org/wiki/BatchModeTransactionManager contains some info on what's the rationale behind batching and I suspect suspect some of that is useful in Infinispan as well.

On top of that, I think http://community.jboss.org/docs/DOC-14901 should answer:
- Are locks maintained in a batch?
- What happens at commit time in a local cache?
- What happens at commit time in a clustered cache? 1PC or 2PC
- What happens if a cache loader is involved? Are updates atomic on the cacheloader?
- When/Why would someone use batching instead of JTA transactions? And viceversa?
...etc

> 
>>> 
>>>>> I don't think it's a good idea to tie up batching as a way to build transactions cos that's not the use case of batching.
>>>> Batching are Infinispan's local transactions with a fancy name :)
>>>>> If you need transactions, use JTA transactions.
>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> Transactionable clients doesn't sound very well to me. I'd go for 'transactional clients'.
>>>>>> +1
>>>>>>> 
>>>>>>> I don't think we should support this: "Through HotRod, operations associated with same transaction might be dispatched to multiple nodes. ". I think this is rather messy and will cause problems. Think of sticky sessions. Instead, I think transactional client implementations will need a new load balance policy which is transaction sticky. IOW, if you call begin tx on node A, you want the rest of transaction operations to be directed there.
>>>>>> Good point. Thinking some more we can pool the connection to the server so that we won't keep a TCP connection for the entire duration of the transaction, which would be bad. 
>>>>>>> Otherwise, it gets very messy if the prepare lands on node B and commit on node C. So, wherever the beginTx lands, that's the node that should be used for the duration of the transaction. IOW, my vote is definitely for solution 1 which is simpler and avoids potential lock ups resulting from sending operations in the same tx to diff nodes.
>>>>>> I've just started t o like 1 more as well :) Just to clarify one thing with 2: the tx would reside on one server only. If the prepare lands on B it is forwarded (through something like a FrowardCommand) to C where it would be executed. More complex though. 
>>>>> 
>>>>> Indeed more complex. 
>>>>> 
>>>>> 
>>>>>>> 
>>>>>>> How are we gonna deal with situations where client sends a commitTx which is applied correctly in the target server and any other involved members in the cluster, but there's a failure when commitTx response is sent back to client?
>>>>>> XAResource on the client won't confirm the transaction commit to the TransactionManager(XAResouce.commit would throw an XAException with an specific exception code). From there on it is with the TXManager. 
>>>>> 
>>>>> To which TransactionManager? A transaction manager running on the client or the server? Another question, if there's an XAResource on the client, the client itself must have a TM running there?
>>>>> 
>>>>>>> The client could think that the commit failed but this worked fine on the server.
>>>>>>> 
>>>>>>> I think we need something other than client intelligence for determining whether a transaction is present or not for the following reason: Imagine that as part of transactional operation the server figures out that the client has a stale view. If client sends 4 as client intelligence, what is the server gonna reply in the topology change header? Is it gonna reply with no cluster info? or hash aware topology header? I get the feeling that we're trying to use client intelligence as way to signal that the operation sent is transactional: "Base on client's intelligence, the server should be able to determine weather these fields are present or not." and I think this is not correct. Let's leave client intelligence as it is and let's not try to give it a different meaning.
>>>>>> Yes, I agree.
>>>>>>> 
>>>>>>> Instead, let's use [tx_id length] to signal transactions. First of all, I think [tx_id length] [tx_id] should be part of the header since it's something common to all operations,
>>>>>> but ping, but ping is not really relevant :) 
>>>>> 
>>>>> Yeah, ping is exceptional.
>>>>> 
>>>>>>> rather than appending it at the end of the command where we store command specific information. On top of that, [tx_id length] can easily be used to signal a transaction. If [tx_id length] is 0, no tx is being sent. If not 0, a tx_id follows and hence the operation is transactional.
>>>>>>> 
>>>>>> Point taken. This flag is in trunk and is ignored for now, as we've discussed.
>>>>>> 
>>>>>> Thanks a lot for the feedback, this changes the design significantly. I'll update the doc and let you know.
>>>>> 
>>>>> You're welcome :)
>>>>> 
>>>>>>> Cheers,
>>>>>>> 
>>>>>>> On Jul 20, 2010, at 11:45 AM, Mircea Markus wrote:
>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> I'be just added a design draft for transactions over hotrod[1]. Feedback appreciated!
>>>>>>>> 
>>>>>>>> Cheers,
>>>>>>>> Mircea
>>>>>>>> 
>>>>>>>> [1]http://community.jboss.org/wiki/TransactionsOverHotRod
>>>>>>>> _______________________________________________
>>>>>>>> infinispan-dev mailing list
>>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>> 
>>>>>>> --
>>>>>>> Galder Zamarreño
>>>>>>> Sr. Software Engineer
>>>>>>> Infinispan, JBoss Cache
>>>>>>> 
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> 
>>>>> --
>>>>> Galder Zamarreño
>>>>> Sr. Software Engineer
>>>>> Infinispan, JBoss Cache
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> --
>>> Galder Zamarreño
>>> Sr. Software Engineer
>>> Infinispan, JBoss Cache
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache