Hiberante OGM Infinispan Remote dialect - Execute the command queue as a single Infispan Remote Server Task
2:24 a.m.
I'm Fabio, nice to meet you.
Speaking of the current implementation of the Infinispan remote dialect of
Hibernate OGM, working on issue OGM-1206 and talking with Davide I noticed
that the unit of work commands are executed (flushed to datastore) at the
end of the transaction itself.
In particular I noticed that the commands are stored in a transaction
scoped object of type org.hibernate.ogm.dialect.batch.spi.OperationsQueue.
Instead of perfom one remote invocation for each command in the method
org.hibernate.ogm.dialect.impl.BatchOperationsDelegator::executeBatch
maybe we could invoke a proper Infispan Remote Server Task to execute the
command queue on server side as a bulk operation.
Moving the execution of the server-side command list (Infinispan) we would
have the advantage of reducing remote interactions. Moreover and above all
the execution of the command queue would be a transactional work unit, on
which could be apply a Repeteable Read isolation level, for instance.
The solution would not solve the need for an XA client instead, but it
could be adopted by customers interested in local transactions.
What do you think about it?
Can I open a Jira issue?
Fabio
--
FABIO MASSIMO ERCOLI
Senior Software Engineer - Hibernate & Data Platform
Red Hat <https://www.redhat.com/>
fabio.ercoli(a)redhat.com M: (+39)-329.8681441
<http://redhatemailsignature-marketing.itos.redhat.com/>
<https://red.ht/sig>
2:53 a.m.
Hi Fabio,
thinking about the cons, Hot Rod can intelligently pick the server where
should a given key reside, to reduce the number of hops (therefore,
latency). RemoteCache does not expose any way to route according to some
key; feel free to file a feature request in Infinispan JIRA.
Note that in order to reduce the number of round-trips, Infinispan 9.2
will support true-async operations: Previously the putAsync et all just
moved the execution to different thread, since 9.2.0.CR2 it sends the
request to the wire right await and the response is received through a
CompletableFuture. At this moment multiple requests will need a distinct
TCP connection for each concurrent operation, but this limitation is
likely to be removed in 9.3. The goal is to let you write the batch down
one-by-one using async API and just wait for all the operations to complete.
I know this won't help for all the types of operation (a lack of
client-side API sometimes forces OGM to use get() + CAS in a loop).
I am not trying to talk you out of the remote execution API, just
spreading news about the emerging alternatives.
Radim
On 02/02/2018 09:24 AM, Fabio Massimo Ercoli wrote:
I'm Fabio, nice to meet you.
Speaking of the current implementation of the Infinispan remote dialect of
Hibernate OGM, working on issue OGM-1206 and talking with Davide I noticed
that the unit of work commands are executed (flushed to datastore) at the
end of the transaction itself.
In particular I noticed that the commands are stored in a transaction
scoped object of type org.hibernate.ogm.dialect.batch.spi.OperationsQueue.
Instead of perfom one remote invocation for each command in the method
org.hibernate.ogm.dialect.impl.BatchOperationsDelegator::executeBatch
maybe we could invoke a proper Infispan Remote Server Task to execute the
command queue on server side as a bulk operation.
Moving the execution of the server-side command list (Infinispan) we would
have the advantage of reducing remote interactions. Moreover and above all
the execution of the command queue would be a transactional work unit, on
which could be apply a Repeteable Read isolation level, for instance.
The solution would not solve the need for an XA client instead, but it
could be adopted by customers interested in local transactions.
What do you think about it?
Can I open a Jira issue?
Fabio
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
9:05 a.m.
Hi Radim,
thank you for the alternative, it is very interesting.
I've to study deeply the feature :P
Fabio
On Mon, Feb 5, 2018 at 9:53 AM, Radim Vansa <rvansa(a)redhat.com> wrote:
Hi Fabio,
thinking about the cons, Hot Rod can intelligently pick the server where
should a given key reside, to reduce the number of hops (therefore,
latency). RemoteCache does not expose any way to route according to some
key; feel free to file a feature request in Infinispan JIRA.
Note that in order to reduce the number of round-trips, Infinispan 9.2
will support true-async operations: Previously the putAsync et all just
moved the execution to different thread, since 9.2.0.CR2 it sends the
request to the wire right await and the response is received through a
CompletableFuture. At this moment multiple requests will need a distinct
TCP connection for each concurrent operation, but this limitation is
likely to be removed in 9.3. The goal is to let you write the batch down
one-by-one using async API and just wait for all the operations to
complete.
I know this won't help for all the types of operation (a lack of
client-side API sometimes forces OGM to use get() + CAS in a loop).
I am not trying to talk you out of the remote execution API, just
spreading news about the emerging alternatives.
Radim
On 02/02/2018 09:24 AM, Fabio Massimo Ercoli wrote:
> I'm Fabio, nice to meet you.
>
> Speaking of the current implementation of the Infinispan remote dialect
of
> Hibernate OGM, working on issue OGM-1206 and talking with Davide I
noticed
> that the unit of work commands are executed (flushed to datastore) at the
> end of the transaction itself.
> In particular I noticed that the commands are stored in a transaction
> scoped object of type org.hibernate.ogm.dialect.
batch.spi.OperationsQueue.
>
> Instead of perfom one remote invocation for each command in the method
> org.hibernate.ogm.dialect.impl.BatchOperationsDelegator::executeBatch
> maybe we could invoke a proper Infispan Remote Server Task to execute the
> command queue on server side as a bulk operation.
>
> Moving the execution of the server-side command list (Infinispan) we
would
> have the advantage of reducing remote interactions. Moreover and above
all
> the execution of the command queue would be a transactional work unit, on
> which could be apply a Repeteable Read isolation level, for instance.
>
> The solution would not solve the need for an XA client instead, but it
> could be adopted by customers interested in local transactions.
>
> What do you think about it?
> Can I open a Jira issue?
>
> Fabio
>
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
_______________________________________________
hibernate-dev mailing list
hibernate-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
--
FABIO MASSIMO ERCOLI
Senior Software Engineer - Hibernate & Data Platform
Red Hat <https://www.redhat.com/>
fabio.ercoli(a)redhat.com M: (+39)-329.8681441
<http://redhatemailsignature-marketing.itos.redhat.com/>
<https://red.ht/sig>
7:55 a.m.
Thanks for starting this, great ideas.
We should start exploring such options but I am thinking of some more
limitations that we'll have to overcome. To keep the discussion from
getting hopelessly complex, let's try to clarify the purpose:
The goal is to solve two problems which are strongly related:
# performance / efficiency
# better ACIDity of "transactions"
# Efficiency
Radim makes a great point, it would be nice to be able to address the
optimal node for any "task" we might want to perform.
JIRA:
- https://issues.jboss.org/browse/ISPN-8770
However while we wait for the Infinispan team to provide such
refinements, I think we should go ahead with this plan already, not
least for the other reasons, but also to be able to eventually provide
quick feedback on such improvements.
Clearly this means we'll incur (possibly?) a performance hit when
running a simple task - such as writing a single key - but it
shouldn't matter much for more complex operations: especially with a
small sized cluster chances are high to hit a node owning at least one
of these.
For the moment our client could detect if it's preferrable to run a
task or a "simple op"; essentially we could implement some of
ISPN-8770 in our Dialect but rather than making this too complex on
our side I'd contribute to the Infinispan improvement first.
# "one transaction, one task" ?
Just to clarify, we can't promise that the ORM won't need to flush
multiple times before the transaction is finally committed. Make sure
you read about auto-flush strategies, either in our docs or there are
many good articles about the topic.
In a nutshell, if there are dirty entities of some type A, and then a
query is run on that same type A, the ORM engine needs to flush the
pending changes on all As first to make sure the queries are accurate.
We could debate if this is appropriate and maybe we shouldn't do it on
all NoSQL stores, but it's certainly debatable as it's a change in
semantics compared to what people are used to.
It's not entirely bad news though; while a single transaction might
actually be performed by multiple, independent although ordered "Task
RPC"s this will give us some opportunity to at least make sure that
operations which really need to be atomic will happen atomically; for
example when we delete an entity we can make sure that all references
are cleared up correctly. However it's not a real transaction still
and we need to be clear about that, no big deal as "real transactions"
will come soon in Infinispan Remote too.
# Usability
Just a note on this. I hope we can try to work on this while expecting
that the Infinispan Server won't need to have specific code deployed
which depends on the specific application, such as the object model.
If it helps we can think of an "OGM extensions" to be included with
the Infinispan Server, but the code we include would then need to be
able to work with any schema. Ideally, even with multiple versions of
OGM, so I'd hope that we can design the models of each task we'll need
as Protobuf Entities.
Fabio, do you want to start working on such a POC?
Thanks,
Sanne
On 5 February 2018 at 15:05, Fabio Massimo Ercoli <fercoli(a)redhat.com> wrote:
Hi Radim,
thank you for the alternative, it is very interesting.
I've to study deeply the feature :P
Fabio
On Mon, Feb 5, 2018 at 9:53 AM, Radim Vansa <rvansa(a)redhat.com> wrote:
> Hi Fabio,
>
> thinking about the cons, Hot Rod can intelligently pick the server where
> should a given key reside, to reduce the number of hops (therefore,
> latency). RemoteCache does not expose any way to route according to some
> key; feel free to file a feature request in Infinispan JIRA.
>
> Note that in order to reduce the number of round-trips, Infinispan 9.2
> will support true-async operations: Previously the putAsync et all just
> moved the execution to different thread, since 9.2.0.CR2 it sends the
> request to the wire right await and the response is received through a
> CompletableFuture. At this moment multiple requests will need a distinct
> TCP connection for each concurrent operation, but this limitation is
> likely to be removed in 9.3. The goal is to let you write the batch down
> one-by-one using async API and just wait for all the operations to
> complete.
>
> I know this won't help for all the types of operation (a lack of
> client-side API sometimes forces OGM to use get() + CAS in a loop).
>
> I am not trying to talk you out of the remote execution API, just
> spreading news about the emerging alternatives.
>
> Radim
>
> On 02/02/2018 09:24 AM, Fabio Massimo Ercoli wrote:
> > I'm Fabio, nice to meet you.
> >
> > Speaking of the current implementation of the Infinispan remote dialect
> of
> > Hibernate OGM, working on issue OGM-1206 and talking with Davide I
> noticed
> > that the unit of work commands are executed (flushed to datastore) at the
> > end of the transaction itself.
> > In particular I noticed that the commands are stored in a transaction
> > scoped object of type org.hibernate.ogm.dialect.
> batch.spi.OperationsQueue.
> >
> > Instead of perfom one remote invocation for each command in the method
> > org.hibernate.ogm.dialect.impl.BatchOperationsDelegator::executeBatch
> > maybe we could invoke a proper Infispan Remote Server Task to execute the
> > command queue on server side as a bulk operation.
> >
> > Moving the execution of the server-side command list (Infinispan) we
> would
> > have the advantage of reducing remote interactions. Moreover and above
> all
> > the execution of the command queue would be a transactional work unit, on
> > which could be apply a Repeteable Read isolation level, for instance.
> >
> > The solution would not solve the need for an XA client instead, but it
> > could be adopted by customers interested in local transactions.
> >
> > What do you think about it?
> > Can I open a Jira issue?
> >
> > Fabio
> >
>
> --
> Radim Vansa <rvansa(a)redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
--
FABIO MASSIMO ERCOLI
Senior Software Engineer - Hibernate & Data Platform
Red Hat <https://www.redhat.com/>
fabio.ercoli(a)redhat.com M: (+39)-329.8681441
<http://redhatemailsignature-marketing.itos.redhat.com/>
<https://red.ht/sig>
_______________________________________________
hibernate-dev mailing list
hibernate-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
8:17 a.m.
On 02/06/2018 02:55 PM, Sanne Grinovero wrote:
Thanks for starting this, great ideas.
We should start exploring such options but I am thinking of some more
limitations that we'll have to overcome. To keep the discussion from
getting hopelessly complex, let's try to clarify the purpose:
The goal is to solve two problems which are strongly related:
# performance / efficiency
# better ACIDity of "transactions"
# Efficiency
Radim makes a great point, it would be nice to be able to address the
optimal node for any "task" we might want to perform.
JIRA:
- https://issues.jboss.org/browse/ISPN-8770
However while we wait for the Infinispan team to provide such
refinements, I think we should go ahead with this plan already, not
least for the other reasons, but also to be able to eventually provide
quick feedback on such improvements.
Clearly this means we'll incur (possibly?) a performance hit when
running a simple task - such as writing a single key - but it
shouldn't matter much for more complex operations: especially with a
small sized cluster chances are high to hit a node owning at least one
of these.
For the moment our client could detect if it's preferrable to run a
task or a "simple op"; essentially we could implement some of
ISPN-8770 in our Dialect but rather than making this too complex on
our side I'd contribute to the Infinispan improvement first.
# "one transaction, one task" ?
Just to clarify, we can't promise that the ORM won't need to flush
multiple times before the transaction is finally committed. Make sure
you read about auto-flush strategies, either in our docs or there are
many good articles about the topic.
In a nutshell, if there are dirty entities of some type A, and then a
query is run on that same type A, the ORM engine needs to flush the
pending changes on all As first to make sure the queries are accurate.
We could debate if this is appropriate and maybe we shouldn't do it on
all NoSQL stores, but it's certainly debatable as it's a change in
semantics compared to what people are used to.
It's not entirely bad news though; while a single transaction might
actually be performed by multiple, independent although ordered "Task
RPC"s this will give us some opportunity to at least make sure that
operations which really need to be atomic will happen atomically; for
example when we delete an entity we can make sure that all references
are cleared up correctly. However it's not a real transaction still
and we need to be clear about that, no big deal as "real transactions"
will come soon in Infinispan Remote too.
About 'real transactions': beware that current design for the Hot Rod
transactions does not involve executing scripts withing the scope of the
ongoing transaction. Since the server-side is already in master (for a
while now), the protocol that lacks support for script execution in
transaction is set. It was a 'start small' decision, so if this appears
to be critical for your use cases please shout aloud on infinispan-dev
list to influence that.
# Usability
Just a note on this. I hope we can try to work on this while expecting
that the Infinispan Server won't need to have specific code deployed
which depends on the specific application, such as the object model.
If it helps we can think of an "OGM extensions" to be included with
the Infinispan Server, but the code we include would then need to be
able to work with any schema. Ideally, even with multiple versions of
OGM, so I'd hope that we can design the models of each task we'll need
as Protobuf Entities.
I hope you won't end up with OGM generating javascript to be run on the
server :)
Radim
Fabio, do you want to start working on such a POC?
Thanks,
Sanne
On 5 February 2018 at 15:05, Fabio Massimo Ercoli <fercoli(a)redhat.com> wrote:
> Hi Radim,
>
> thank you for the alternative, it is very interesting.
> I've to study deeply the feature :P
>
> Fabio
>
>
> On Mon, Feb 5, 2018 at 9:53 AM, Radim Vansa <rvansa(a)redhat.com> wrote:
>
>> Hi Fabio,
>>
>> thinking about the cons, Hot Rod can intelligently pick the server where
>> should a given key reside, to reduce the number of hops (therefore,
>> latency). RemoteCache does not expose any way to route according to some
>> key; feel free to file a feature request in Infinispan JIRA.
>>
>> Note that in order to reduce the number of round-trips, Infinispan 9.2
>> will support true-async operations: Previously the putAsync et all just
>> moved the execution to different thread, since 9.2.0.CR2 it sends the
>> request to the wire right await and the response is received through a
>> CompletableFuture. At this moment multiple requests will need a distinct
>> TCP connection for each concurrent operation, but this limitation is
>> likely to be removed in 9.3. The goal is to let you write the batch down
>> one-by-one using async API and just wait for all the operations to
>> complete.
>>
>> I know this won't help for all the types of operation (a lack of
>> client-side API sometimes forces OGM to use get() + CAS in a loop).
>>
>> I am not trying to talk you out of the remote execution API, just
>> spreading news about the emerging alternatives.
>>
>> Radim
>>
>> On 02/02/2018 09:24 AM, Fabio Massimo Ercoli wrote:
>>> I'm Fabio, nice to meet you.
>>>
>>> Speaking of the current implementation of the Infinispan remote dialect
>> of
>>> Hibernate OGM, working on issue OGM-1206 and talking with Davide I
>> noticed
>>> that the unit of work commands are executed (flushed to datastore) at the
>>> end of the transaction itself.
>>> In particular I noticed that the commands are stored in a transaction
>>> scoped object of type org.hibernate.ogm.dialect.
>> batch.spi.OperationsQueue.
>>> Instead of perfom one remote invocation for each command in the method
>>> org.hibernate.ogm.dialect.impl.BatchOperationsDelegator::executeBatch
>>> maybe we could invoke a proper Infispan Remote Server Task to execute the
>>> command queue on server side as a bulk operation.
>>>
>>> Moving the execution of the server-side command list (Infinispan) we
>> would
>>> have the advantage of reducing remote interactions. Moreover and above
>> all
>>> the execution of the command queue would be a transactional work unit, on
>>> which could be apply a Repeteable Read isolation level, for instance.
>>>
>>> The solution would not solve the need for an XA client instead, but it
>>> could be adopted by customers interested in local transactions.
>>>
>>> What do you think about it?
>>> Can I open a Jira issue?
>>>
>>> Fabio
>>>
>> --
>> Radim Vansa <rvansa(a)redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> hibernate-dev mailing list
>> hibernate-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>
>
>
> --
>
> FABIO MASSIMO ERCOLI
>
> Senior Software Engineer - Hibernate & Data Platform
>
> Red Hat <https://www.redhat.com/>
>
> fabio.ercoli(a)redhat.com M: (+39)-329.8681441
> <http://redhatemailsignature-marketing.itos.redhat.com/>
> <https://red.ht/sig>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
2281
days inactive
2285
days old
4 comments
3 participants
participants (3)
-
Fabio Massimo Ercoli -
Radim Vansa -
Sanne Grinovero