11:46 p.m.
HI,
I always wandered why the identity generator needs to assign an identifier
prior to flushing.
Because of this behavior, the JDBC batching is disabled which is bad news
for MySQL where most setups use IDENTITY instead of TABLE generation type.
Since the AbstractSaveEventListener supports the shouldDelayIdentityInserts
option for when there is no transaction n progress, we could make this a
default anyway.
If the inserts are delayed to flush-time, we could batch them based on the
current dialect. Only the SQL Server dialect and Oracle 11g don't allow
fetching the resulting keys from a batch. But Oracle 12c (which supports
IDENTITY), PostgreSQL and MySQL can save a batch of inserts and fetch the
generated keys.
If I judge from a Stack Overflow perspective, most questions are in the
context of MySQL, so many Hibernate users use MySQL in fact.
What do you think? Should we make this change and allow MySQL users to
batch JDBC inserts even for IDENTITY.
Vlad
12:04 p.m.
As soon as a new entity becomes "managed", it needs to be tracked by
the Persistence Context (the container within each Session)..
obviously, as otherwise we wouldn't be able to know what needs to be
eventually flushed.
And within the PC each entity must have an id assigned: it would be
nice to have a data structure to be able to deal with reference
equality, but then how would you know when a "merge" operation on a
different object instance is meant to replace one of the previous
references?
I wonder if we could handle all corner cases like a merge operation by
tracking references too (as an alternative to ids), and essentially
have ids assigned as late as possible.
But I'm pretty sure that we have an API contract that mandates that a
user can read the Id after the entity has been "saved", even if we
didn't commit yet, which implies the entities should also be
instrumented so that a getId() can trigger a lazy initialization of
the id.
Since this lazy initialization needs to happen on the instances passed
(and constructed) by the end user, I suspect such a feature would only
be safe to be enabled for those using bytecode enhanced entities?
Seems like a big one.. you think it's worth it? People might simply
use an in-memory id generation strategy.
interesting thoughts!
Sanne
On 6 December 2015 at 05:46, Vlad Mihalcea <mihalcea.vlad(a)gmail.com> wrote:
HI,
I always wandered why the identity generator needs to assign an identifier
prior to flushing.
Because of this behavior, the JDBC batching is disabled which is bad news
for MySQL where most setups use IDENTITY instead of TABLE generation type.
Since the AbstractSaveEventListener supports the shouldDelayIdentityInserts
option for when there is no transaction n progress, we could make this a
default anyway.
If the inserts are delayed to flush-time, we could batch them based on the
current dialect. Only the SQL Server dialect and Oracle 11g don't allow
fetching the resulting keys from a batch. But Oracle 12c (which supports
IDENTITY), PostgreSQL and MySQL can save a batch of inserts and fetch the
generated keys.
If I judge from a Stack Overflow perspective, most questions are in the
context of MySQL, so many Hibernate users use MySQL in fact.
What do you think? Should we make this change and allow MySQL users to
batch JDBC inserts even for IDENTITY.
Vlad
_______________________________________________
hibernate-dev mailing list
hibernate-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
12:45 p.m.
On Sat, Dec 5, 2015 at 11:46 PM Vlad Mihalcea <mihalcea.vlad(a)gmail.com>
wrote:
HI,
I always wandered why the identity generator needs to assign an identifier
prior to flushing.
In general, how else do you propose to obtain the identifier with which we
can associate the entity into the PersistenceContext? That has to come
from the insert. Just look at the "hacks" that are in place to support
this today in terms of (1) generating a "stand-in identifier", (2) tracking
these in the PC and (3) going back and adjusting them after the eventual
insert. I guess probably I am missing what you are really asking here...
Because of this behavior, the JDBC batching is disabled which is bad news
for MySQL where most setups use IDENTITY instead of TABLE generation
type.
Since the AbstractSaveEventListener supports the shouldDelayIdentityInserts
option for when there is no transaction n progress, we could make this a
default anyway.
Well that is *part of* the resolution of shouldDelayIdentityInserts. The
other part is whether the source of this entity becoming persistent/managed
requires immediate access to the id; the typical case here is the
difference between save() and persist() - save() returns the id as the
method return, whereas persist() does not. So if save() was used the
insert *has to* happen immediately.
If the inserts are delayed to flush-time, we could batch them based on the
current dialect. Only the SQL Server dialect and Oracle 11g don't
allow
fetching the resulting keys from a batch. But Oracle 12c (which supports
IDENTITY), PostgreSQL and MySQL can save a batch of inserts and fetch the
generated keys.
If I judge from a Stack Overflow perspective, most questions are in the
context of MySQL, so many Hibernate users use MySQL in fact.
What do you think? Should we make this change and allow MySQL users to
batch JDBC inserts even for IDENTITY.
As you point out, the proper solution here would require consultation with
the Dialect to determine whether getGeneratedKeys works properly with
batched statements. From there AbstractEntityPersister would need changes
to allow these inserts to be batched based on what the Dialect reports. If
that is all something you want to take on, I don't see an issue with it -
provided the transaction and requiresImmediateIdAccess checks in
AbstractSaveEventListener stay in place. The requiresImmediateIdAccess
requirement is obvious. Maybe I am just letting my loathing of 2 bad ideas
(auto-commit and IDENTITY) combined cloud my judgement in regards to the
transaction "requirement" ;)
12:57 p.m.
I'm sure it require a significant change since the persistence context is
very much tied to the entity identifier.
Theoretically, it could work because the identifier is required for
database operations and for merging as well.
Because the EntityIdentityInsertAction already has the isDelayed property,
maybe we could better reason if we could eventually make this a reasonable
default.
Anyway, it's worth judging if it's worth the development cost and if we
can't afford the change, maybe we could add a Hibernate-native Batching API
to offer an alternative.
Vlad
On Mon, Dec 7, 2015 at 8:04 PM, Sanne Grinovero <sanne(a)hibernate.org> wrote:
As soon as a new entity becomes "managed", it needs to be
tracked by
the Persistence Context (the container within each Session)..
obviously, as otherwise we wouldn't be able to know what needs to be
eventually flushed.
And within the PC each entity must have an id assigned: it would be
nice to have a data structure to be able to deal with reference
equality, but then how would you know when a "merge" operation on a
different object instance is meant to replace one of the previous
references?
I wonder if we could handle all corner cases like a merge operation by
tracking references too (as an alternative to ids), and essentially
have ids assigned as late as possible.
But I'm pretty sure that we have an API contract that mandates that a
user can read the Id after the entity has been "saved", even if we
didn't commit yet, which implies the entities should also be
instrumented so that a getId() can trigger a lazy initialization of
the id.
Since this lazy initialization needs to happen on the instances passed
(and constructed) by the end user, I suspect such a feature would only
be safe to be enabled for those using bytecode enhanced entities?
Seems like a big one.. you think it's worth it? People might simply
use an in-memory id generation strategy.
interesting thoughts!
Sanne
On 6 December 2015 at 05:46, Vlad Mihalcea <mihalcea.vlad(a)gmail.com>
wrote:
> HI,
>
> I always wandered why the identity generator needs to assign an
identifier
> prior to flushing.
> Because of this behavior, the JDBC batching is disabled which is bad news
> for MySQL where most setups use IDENTITY instead of TABLE generation
type.
>
> Since the AbstractSaveEventListener supports the
shouldDelayIdentityInserts
> option for when there is no transaction n progress, we could make this a
> default anyway.
>
> If the inserts are delayed to flush-time, we could batch them based on
the
> current dialect. Only the SQL Server dialect and Oracle 11g don't allow
> fetching the resulting keys from a batch. But Oracle 12c (which supports
> IDENTITY), PostgreSQL and MySQL can save a batch of inserts and fetch the
> generated keys.
>
> If I judge from a Stack Overflow perspective, most questions are in the
> context of MySQL, so many Hibernate users use MySQL in fact.
>
> What do you think? Should we make this change and allow MySQL users to
> batch JDBC inserts even for IDENTITY.
>
> Vlad
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
1:04 p.m.
WDYM by "Hibernate-native Batching API"?
On Mon, Dec 7, 2015 at 12:57 PM Vlad Mihalcea <mihalcea.vlad(a)gmail.com>
wrote:
I'm sure it require a significant change since the persistence
context is
very much tied to the entity identifier.
Theoretically, it could work because the identifier is required for
database operations and for merging as well.
Because the EntityIdentityInsertAction already has the isDelayed property,
maybe we could better reason if we could eventually make this a reasonable
default.
Anyway, it's worth judging if it's worth the development cost and if we
can't afford the change, maybe we could add a Hibernate-native Batching API
to offer an alternative.
Vlad
On Mon, Dec 7, 2015 at 8:04 PM, Sanne Grinovero <sanne(a)hibernate.org>
wrote:
> As soon as a new entity becomes "managed", it needs to be tracked by
> the Persistence Context (the container within each Session)..
> obviously, as otherwise we wouldn't be able to know what needs to be
> eventually flushed.
>
> And within the PC each entity must have an id assigned: it would be
> nice to have a data structure to be able to deal with reference
> equality, but then how would you know when a "merge" operation on a
> different object instance is meant to replace one of the previous
> references?
>
> I wonder if we could handle all corner cases like a merge operation by
> tracking references too (as an alternative to ids), and essentially
> have ids assigned as late as possible.
>
> But I'm pretty sure that we have an API contract that mandates that a
> user can read the Id after the entity has been "saved", even if we
> didn't commit yet, which implies the entities should also be
> instrumented so that a getId() can trigger a lazy initialization of
> the id.
>
> Since this lazy initialization needs to happen on the instances passed
> (and constructed) by the end user, I suspect such a feature would only
> be safe to be enabled for those using bytecode enhanced entities?
>
> Seems like a big one.. you think it's worth it? People might simply
> use an in-memory id generation strategy.
>
> interesting thoughts!
>
> Sanne
>
>
> On 6 December 2015 at 05:46, Vlad Mihalcea <mihalcea.vlad(a)gmail.com>
> wrote:
> > HI,
> >
> > I always wandered why the identity generator needs to assign an
> identifier
> > prior to flushing.
> > Because of this behavior, the JDBC batching is disabled which is bad
news
> > for MySQL where most setups use IDENTITY instead of TABLE generation
> type.
> >
> > Since the AbstractSaveEventListener supports the
> shouldDelayIdentityInserts
> > option for when there is no transaction n progress, we could make this
a
> > default anyway.
> >
> > If the inserts are delayed to flush-time, we could batch them based on
> the
> > current dialect. Only the SQL Server dialect and Oracle 11g don't allow
> > fetching the resulting keys from a batch. But Oracle 12c (which
supports
> > IDENTITY), PostgreSQL and MySQL can save a batch of inserts and fetch
the
> > generated keys.
> >
> > If I judge from a Stack Overflow perspective, most questions are in the
> > context of MySQL, so many Hibernate users use MySQL in fact.
> >
> > What do you think? Should we make this change and allow MySQL users to
> > batch JDBC inserts even for IDENTITY.
> >
> > Vlad
> > _______________________________________________
> > hibernate-dev mailing list
> > hibernate-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
_______________________________________________
hibernate-dev mailing list
hibernate-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
1:08 p.m.
MySQL is the first choice for many projects (even if PostgreSQL is more
advanced) and the AUTO_INCREMENT is the only choice a DBA will approve.
The TABLE identifier generator is too heavyweight and any external DB user
will have to know how it works for inserting new rows.
I'll run some tests for TABLE identifier and see how the locking and
transaction context switch overhead affects performance.
If this that delaying identity entities to flush-time is a doable task, I
think we should add a Jira issue for this.
Vlad
On Mon, Dec 7, 2015 at 8:45 PM, Steve Ebersole <steve(a)hibernate.org> wrote:
On Sat, Dec 5, 2015 at 11:46 PM Vlad Mihalcea <mihalcea.vlad(a)gmail.com>
wrote:
> HI,
>
> I always wandered why the identity generator needs to assign an identifier
> prior to flushing.
>
In general, how else do you propose to obtain the identifier with which we
can associate the entity into the PersistenceContext? That has to come
from the insert. Just look at the "hacks" that are in place to support
this today in terms of (1) generating a "stand-in identifier", (2) tracking
these in the PC and (3) going back and adjusting them after the eventual
insert. I guess probably I am missing what you are really asking here...
Because of this behavior, the JDBC batching is disabled which is bad news
> for MySQL where most setups use IDENTITY instead of TABLE generation type.
>
> Since the AbstractSaveEventListener supports the
> shouldDelayIdentityInserts
> option for when there is no transaction n progress, we could make this a
> default anyway.
>
Well that is *part of* the resolution of shouldDelayIdentityInserts. The
other part is whether the source of this entity becoming persistent/managed
requires immediate access to the id; the typical case here is the
difference between save() and persist() - save() returns the id as the
method return, whereas persist() does not. So if save() was used the
insert *has to* happen immediately.
If the inserts are delayed to flush-time, we could batch them based on the
> current dialect. Only the SQL Server dialect and Oracle 11g don't allow
> fetching the resulting keys from a batch. But Oracle 12c (which supports
> IDENTITY), PostgreSQL and MySQL can save a batch of inserts and fetch the
> generated keys.
>
> If I judge from a Stack Overflow perspective, most questions are in the
> context of MySQL, so many Hibernate users use MySQL in fact.
>
> What do you think? Should we make this change and allow MySQL users to
> batch JDBC inserts even for IDENTITY.
>
As you point out, the proper solution here would require consultation with
the Dialect to determine whether getGeneratedKeys works properly with
batched statements. From there AbstractEntityPersister would need changes
to allow these inserts to be batched based on what the Dialect reports. If
that is all something you want to take on, I don't see an issue with it -
provided the transaction and requiresImmediateIdAccess checks in
AbstractSaveEventListener stay in place. The requiresImmediateIdAccess
requirement is obvious. Maybe I am just letting my loathing of 2 bad ideas
(auto-commit and IDENTITY) combined cloud my judgement in regards to the
transaction "requirement" ;)
2:11 p.m.
I meant something like:
session.persistAll(List<?> entities);
Because all entities are passed in a single method call, the persistence
context can start managing those and also try to use batching if the
configuration suggests using JDBC batching.
That could be much easier than delaying the IDENTITY insert to flush-time.
Vlad
On Mon, Dec 7, 2015 at 9:04 PM, Steve Ebersole <steve(a)hibernate.org> wrote:
WDYM by "Hibernate-native Batching API"?
On Mon, Dec 7, 2015 at 12:57 PM Vlad Mihalcea <mihalcea.vlad(a)gmail.com>
wrote:
> I'm sure it require a significant change since the persistence context is
> very much tied to the entity identifier.
> Theoretically, it could work because the identifier is required for
> database operations and for merging as well.
>
> Because the EntityIdentityInsertAction already has the isDelayed property,
> maybe we could better reason if we could eventually make this a reasonable
> default.
> Anyway, it's worth judging if it's worth the development cost and if we
> can't afford the change, maybe we could add a Hibernate-native Batching
> API
> to offer an alternative.
>
> Vlad
>
> On Mon, Dec 7, 2015 at 8:04 PM, Sanne Grinovero <sanne(a)hibernate.org>
> wrote:
>
> > As soon as a new entity becomes "managed", it needs to be tracked by
> > the Persistence Context (the container within each Session)..
> > obviously, as otherwise we wouldn't be able to know what needs to be
> > eventually flushed.
> >
> > And within the PC each entity must have an id assigned: it would be
> > nice to have a data structure to be able to deal with reference
> > equality, but then how would you know when a "merge" operation on a
> > different object instance is meant to replace one of the previous
> > references?
> >
> > I wonder if we could handle all corner cases like a merge operation by
> > tracking references too (as an alternative to ids), and essentially
> > have ids assigned as late as possible.
> >
> > But I'm pretty sure that we have an API contract that mandates that a
> > user can read the Id after the entity has been "saved", even if we
> > didn't commit yet, which implies the entities should also be
> > instrumented so that a getId() can trigger a lazy initialization of
> > the id.
> >
> > Since this lazy initialization needs to happen on the instances passed
> > (and constructed) by the end user, I suspect such a feature would only
> > be safe to be enabled for those using bytecode enhanced entities?
> >
> > Seems like a big one.. you think it's worth it? People might simply
> > use an in-memory id generation strategy.
> >
> > interesting thoughts!
> >
> > Sanne
> >
> >
> > On 6 December 2015 at 05:46, Vlad Mihalcea <mihalcea.vlad(a)gmail.com>
> > wrote:
> > > HI,
> > >
> > > I always wandered why the identity generator needs to assign an
> > identifier
> > > prior to flushing.
> > > Because of this behavior, the JDBC batching is disabled which is bad
> news
> > > for MySQL where most setups use IDENTITY instead of TABLE generation
> > type.
> > >
> > > Since the AbstractSaveEventListener supports the
> > shouldDelayIdentityInserts
> > > option for when there is no transaction n progress, we could make
> this a
> > > default anyway.
> > >
> > > If the inserts are delayed to flush-time, we could batch them based on
> > the
> > > current dialect. Only the SQL Server dialect and Oracle 11g don't
> allow
> > > fetching the resulting keys from a batch. But Oracle 12c (which
> supports
> > > IDENTITY), PostgreSQL and MySQL can save a batch of inserts and fetch
> the
> > > generated keys.
> > >
> > > If I judge from a Stack Overflow perspective, most questions are in
> the
> > > context of MySQL, so many Hibernate users use MySQL in fact.
> > >
> > > What do you think? Should we make this change and allow MySQL users to
> > > batch JDBC inserts even for IDENTITY.
> > >
> > > Vlad
> > > _______________________________________________
> > > hibernate-dev mailing list
> > > hibernate-dev(a)lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
2670
days inactive
2671
days old
6 comments
3 participants
participants (3)
-
Sanne Grinovero -
Steve Ebersole -
Vlad Mihalcea