11:16 a.m.
Hi,
These is a draft of the config changes I plan to add with ISPN-1131, can you please
comment?
1.
<transaction lockingMode="optimistic"/>
The "lockingMode" attribute can have two values: optimistic(default) and
pessimistic. If pessimistic is specified then this hase the same meaning as
"useEagerLocking=true" (useEagerLocking to be deprecated).
The fluent API looks as follows:
- config.fluent().transaction().lockingModeOptimistic();
- config.fluent().transaction().lockingModePessimistic();
2.
An important restriction introduced by ISPN-1131 is the fact that a cache cannot be used
in a mixed mode: i.e. all operations run within transactions or non does.
- by specifying a "transactionManagerLookup" the user marks the cache as
transactional.
- there also is an "autoCommit" attribute: <transaction
autoCommit="true"/>
- if true (default), then user doesn't have to explicitly start and finish
single-operation transactions, but a transaction is started under the hood on user's
behalf
- if false and a call is made out of a transaction's scope then an exception
is thrown
- if the cache is not transactional then autoCommit is ignored
- a cache that supports batch operations is implicitly transactional. That's because
batching is based on transactions.
and the fluent API:
config.fluent().transaction().autoCommit(true);
Cheers,
Mircea
11:36 a.m.
Hi Mircea,
what you propose has a strong impact on existing use cases.
Not having batching kills the Lucene performance, and using
transactions is not an option unless the whole state of the index can
fit in memory, which is not the use case we're targeting: I need to be
able to use both on the same Cache.
Isn't it possible to have a batching implementation which doesn't rely
on transactions, or why are you needing to add this limitation?
If not, I'll have to start thinking about a different design.. not
happy about it since we got much testing from the community and I
would consider it quite stable now, but especially the performance
tests are not something I can easily reproduce on my own as it implies
big indexes, real clusters and real world use cases.
What we could do, is implement batching in the Lucene Directory itself
( a layer above Infinispan); that could work, but then again I don't
see why this could not be generalized and provide to everyone, keeping
the change less traumatic.. I'm likely not the only user of this API.
Sanne
2011/8/1 Mircea Markus <mircea.markus(a)jboss.com>:
Hi,
These is a draft of the config changes I plan to add with ISPN-1131, can you please
comment?
1.
<transaction lockingMode="optimistic"/>
The "lockingMode" attribute can have two values: optimistic(default) and
pessimistic. If pessimistic is specified then this hase the same meaning as
"useEagerLocking=true" (useEagerLocking to be deprecated).
The fluent API looks as follows:
- config.fluent().transaction().lockingModeOptimistic();
- config.fluent().transaction().lockingModePessimistic();
2.
An important restriction introduced by ISPN-1131 is the fact that a cache cannot be used
in a mixed mode: i.e. all operations run within transactions or non does.
- by specifying a "transactionManagerLookup" the user marks the cache as
transactional.
- there also is an "autoCommit" attribute: <transaction
autoCommit="true"/>
- if true (default), then user doesn't have to explicitly start and finish
single-operation transactions, but a transaction is started under the hood on user's
behalf
- if false and a call is made out of a transaction's scope then an exception
is thrown
- if the cache is not transactional then autoCommit is ignored
- a cache that supports batch operations is implicitly transactional. That's because
batching is based on transactions.
and the fluent API:
config.fluent().transaction().autoCommit(true);
Cheers,
Mircea
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
2:49 p.m.
2011/8/1 Sanne Grinovero <sanne(a)infinispan.org>:
Hi Mircea,
what you propose has a strong impact on existing use cases.
Not having batching kills the Lucene performance, and using
transactions is not an option unless the whole state of the index can
fit in memory, which is not the use case we're targeting: I need to be
able to use both on the same Cache.
Isn't it possible to have a batching implementation which doesn't rely
on transactions, or why are you needing to add this limitation?
If not, I'll have to start thinking about a different design.. not
happy about it since we got much testing from the community and I
would consider it quite stable now, but especially the performance
tests are not something I can easily reproduce on my own as it implies
big indexes, real clusters and real world use cases.
What we could do, is implement batching in the Lucene Directory itself
( a layer above Infinispan); that could work, but then again I don't
see why this could not be generalized and provide to everyone, keeping
the change less traumatic.. I'm likely not the only user of this API.
Sorry, not sure what I was thinking: I just realized that I can't
possibly implement batching on a different layer.
So I have no way to workaround this other than re-designing some core concepts:
please allow batching without transactions.
Sanne
4:09 a.m.
On 1 Aug 2011, at 17:36, Sanne Grinovero wrote:
Hi Mircea,
what you propose has a strong impact on existing use cases.
Not having batching kills the Lucene performance, and using
transactions is not an option unless the whole state of the index can
fit in memory, which is not the use case we're targeting: I need to be
able to use both on the same Cache.
So you access the cache in two ways:
- through batching api and
- directly, outside the scope of a transaction
Am I correct?
Isn't it possible to have a batching implementation which doesn't rely
on transactions, or why are you needing to add this limitation?
Supporting a mixed
way of accessing the cache might cause problems [1] (thanks to Paolo R. for this paper)
and is also not consistent with the JSR 107's way of doing things, which doesn't
go for mixed cache.
I'd like to get a better grasp of your use case and let's catch up from there.
IRC?
[1] http://www.cis.upenn.edu/acg/papers/cal06_atomic_semantics.pdf
4:44 a.m.
2011/8/2 Mircea Markus <mircea.markus(a)jboss.com>:
On 1 Aug 2011, at 17:36, Sanne Grinovero wrote:
Hi Mircea,
what you propose has a strong impact on existing use cases.
Not having batching kills the Lucene performance, and using
transactions is not an option unless the whole state of the index can
fit in memory, which is not the use case we're targeting: I need to be
able to use both on the same Cache.
So you access the cache in two ways:
- through batching api and
- directly, outside the scope of a transaction
Am I correct?
Isn't it possible to have a batching implementation which doesn't rely
on transactions, or why are you needing to add this limitation?
Supporting a mixed way of accessing the cache might cause problems [1]
(thanks to Paolo R. for this paper) and is also not consistent with the JSR
107's way of doing things, which doesn't go for mixed cache.
I'd like to get a better grasp of your use case and let's catch up from
there. IRC?
[1] http://www.cis.upenn.edu/acg/papers/cal06_atomic_semantics.pdf
I'm afraid we're mixing concerns, I didn't ask to allow both
transactional and non-transactional operations on the same cache, I
understand there are good reasons to not support that.
What I need is control for batching, i.e. be able to send a set of
writes on different keys at one specific point in time, combined with
the capability to avoid repeatable reads in the same cache: definitely
there are many ways to emulate a similar behaviour, I could collect
all writes in a map and then send async operations and block on them
all, or open transactions for the span of single operations when I
don't need to batch.
Consider as well that I might be pushing a thousand of packages of
10MB each, I'm not of the idea that keeping these in a distributed
transaction log is healthy for performance since I don't need rollback
capabilities, being this already handled by Lucene.
Definitely I have alternative options to explore, but it will take
more time and I don't think it would be reasonable to demand this for
5.1; also I think that batching should be supported by Infinispan
core. More importantly, we exposed batching before both
as a feature and as an API, I don't think we can take it away in a
minor release.
10:45 a.m.
I'm afraid we're mixing concerns, I didn't ask to allow
both
transactional and non-transactional operations on the same cache, I
understand there are good reasons to not support that.
What I need is control for batching, i.e. be able to send a set of
writes on different keys at one specific point in time, combined with
the capability to avoid repeatable reads in the same cache: definitely
there are many ways to emulate a similar behaviour, I could collect
all writes in a map and then send async operations and block on them
all, or open transactions for the span of single operations when I
don't need to batch.
Consider as well that I might be pushing a thousand of packages of
10MB each, I'm not of the idea that keeping these in a distributed
transaction log is healthy for performance since I don't need rollback
capabilities, being this already handled by Lucene.
Right, after a long discussion
and brain storming here is the proposed solution:
Short version: you'll have to set autoCommit=true and
use1PcForInducedTransaction=true. Actually this will be the default value for the two
attributes so you won't have to do any change to your existing code, nor
configuration. The existing performance will be preserved, i.e. out-of-tx calls are
handled within 1RPC.
Long version: https://issues.jboss.org/browse/ISPN-1297
10:51 a.m.
2011/8/2 Mircea Markus <mircea.markus(a)jboss.com>:
I'm afraid we're mixing concerns, I didn't ask to allow
both
transactional and non-transactional operations on the same cache, I
understand there are good reasons to not support that.
What I need is control for batching, i.e. be able to send a set of
writes on different keys at one specific point in time, combined with
the capability to avoid repeatable reads in the same cache: definitely
there are many ways to emulate a similar behaviour, I could collect
all writes in a map and then send async operations and block on them
all, or open transactions for the span of single operations when I
don't need to batch.
Consider as well that I might be pushing a thousand of packages of
10MB each, I'm not of the idea that keeping these in a distributed
transaction log is healthy for performance since I don't need rollback
capabilities, being this already handled by Lucene.
Right, after a long discussion and brain storming here is the proposed
solution:
Short version: you'll have to set autoCommit=true and
use1PcForInducedTransaction=true. Actually this will be the default value
for the two attributes so you won't have to do any change to your existing
code, nor configuration. The existing performance will be preserved, i.e.
out-of-tx calls are handled within 1RPC.
Long version: https://issues.jboss.org/browse/ISPN-1297
Thanks! that sounds very good.
as always, we had very constructive discussions on irc.
(translation: was a though fight :D )
11:55 a.m.
On 1 août 2011, at 18:16, Mircea Markus wrote:
Hi,
These is a draft of the config changes I plan to add with ISPN-1131, can you please
comment?
1.
<transaction lockingMode="optimistic"/>
The "lockingMode" attribute can have two values: optimistic(default) and
pessimistic. If pessimistic is specified then this hase the same meaning as
"useEagerLocking=true" (useEagerLocking to be deprecated).
The fluent API looks as follows:
- config.fluent().transaction().lockingModeOptimistic();
- config.fluent().transaction().lockingModePessimistic();
config.fluent().transaction().lockingMode(OPTIMISTIC) //Enum are nice to see alternative
options
4:59 a.m.
On 1 Aug 2011, at 17:55, Emmanuel Bernard wrote:
On 1 août 2011, at 18:16, Mircea Markus wrote:
> Hi,
>
> These is a draft of the config changes I plan to add with ISPN-1131, can you please
comment?
>
> 1.
> <transaction lockingMode="optimistic"/>
> The "lockingMode" attribute can have two values: optimistic(default) and
pessimistic. If pessimistic is specified then this hase the same meaning as
"useEagerLocking=true" (useEagerLocking to be deprecated).
> The fluent API looks as follows:
> - config.fluent().transaction().lockingModeOptimistic();
> - config.fluent().transaction().lockingModePessimistic();
config.fluent().transaction().lockingMode(OPTIMISTIC) //Enum are nice to see alternative
options
Point taken - thanks!
5177
days inactive
5178
days old
infinispan-dev@lists.jboss.org
8 comments
3 participants
participants (3)
-
Emmanuel Bernard -
Mircea Markus -
Sanne Grinovero