Is this somehow different than the "attempt to load stuff into the PC during flush" scenarios I see in any of these related issues? - Steve Ebersole Project Lead http://hibernate.org steve(a)hibernate.org Principal Software Engineer JBoss, a division of Red Hat http://jboss.com http://redhat.com steve.ebersole(a)jboss.com steve.ebersole(a)redhat.com On Thu, 2009-03-05 at 19:14 -0500, Emmanuel Bernard wrote: > http://opensource.atlassian.com/projects/hibernate/browse/HHH-3225 > > Steve, any chance you could look at this one, it seems to hit HSearch > users on a regular basis. > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev

Not sure what you mean by your "In theory it should not"...  The very nature of @PostUpdate is that it is going to be getting called during a flush cycle... ---- wrt "is it possible to move the post* event after the flush?"... There are really 2 answers. 1) According to the JPA spec, can we do this?  The quote from the current spec says: <quote> The PreUpdate and PostUpdate callbacks occur before and after the database update operations to entity data respectively. These database operations may occur at the time the entity state is updated or they may occur at the time state is flushed to the database (which may be at the end of the transaction). </quote> I don't really see anything there that discusses the time-relationship between the SQL UPDATE execution and the @PostUpdate callback other than the fact that (obviously) @PostUpdate callback should come after the SQL UPDATE is issued; but it does not seem to limit *how long after*.  So I think this is OK from the perspective of the spec. 2) Can Hibernate be changed to do this?  Well AnythingIsPossible in programming, so I guess the question really is *should* we change Hibernate to do this.  My main concern with this change is the extra queueing it would require and the corollary memory requirements.  What happens right now is that those callbacks are executed during the action (org.hibernate.action.Executable) execution.  Flush puts them into a queue of actions (org.hibernate.engine.ActionQueue), from which they are removed as they are executed.  We decided to put the post callbacks in the actions themselves for assurance-of-execution as well as encapsulation purposes, which I think are both still worthwhile.  What I could see as a potential solution would be to do something like we do for Actions which have "after transaction" tasks to perform: http://fisheye.jboss.org/browse/Hibernate/core/trunk/core/src/main/java/o... The "executions" list here is a queue of actions which we need to keep around for later.  I can see something like that in conjunction with a method on ActionQueue to process that internal 'callbacks' queue after the entire flush is complete.  Note that this does not address @PreUpdate. We can investigate that though and see what we are talking about in specific. - Steve Ebersole Project Lead http://hibernate.org steve(a)hibernate.org Principal Software Engineer JBoss, a division of Red Hat http://jboss.com http://redhat.com steve.ebersole(a)jboss.com steve.ebersole(a)redhat.com On Fri, 2009-03-06 at 09:07 -0500, Emmanuel Bernard wrote: > Ahhh > In theory it should not as Hibernate Search reads data in the > beforeCompletion phase. > Unless people do not apply changes in a transaction in which case we > need to execute the read in the post* event. > > We will check whether or not people use surrounding transactions > (Hibernate aware Tx either through JTA or via the direct Hibernate > Transaction API). > Alternatively, is it possible to move the post* event after the flush? > Or create noew events for that? That would solve everybody's issue. > > Emmanuel > > On  Mar 5, 2009, at 22:54, Steve Ebersole wrote: > > > Is this somehow different than the "attempt to load stuff into the PC > > during flush" scenarios I see in any of these related issues? > > > > - > > > > Steve Ebersole > > Project Lead > > http://hibernate.org > > steve(a)hibernate.org > > > > Principal Software Engineer > > JBoss, a division of Red Hat > > http://jboss.com > > http://redhat.com > > steve.ebersole(a)jboss.com > > steve.ebersole(a)redhat.com > > > > > > On Thu, 2009-03-05 at 19:14 -0500, Emmanuel Bernard wrote: > >> http://opensource.atlassian.com/projects/hibernate/browse/HHH-3225 > >> > >> Steve, any chance you could look at this one, it seems to hit HSearch > >> users on a regular basis. > >> _______________________________________________ > >> hibernate-dev mailing list > >> hibernate-dev(a)lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

We discussed the issue with Sanne and for Hibernate Search we have a workaround solution that does not penalize Hibernate Core. This solution can be applied by everybody but it's not the easiest thing on Earth. The idea is to queue as you said but inside custom event listeners. In our case some Post* event listeners. This queue is "flushed" in a FlushEventListener. This new flush event listener must be registered *after* the default FlushEventListener. All this does not require Hibernate Core change and requires minimal change to the Hibernate Search code and architecture. On  Mar 6, 2009, at 10:15, Steve Ebersole wrote: > Not sure what you mean by your "In theory it should not"...  The very > nature of @PostUpdate is that it is going to be getting called during a > flush cycle... > > ---- > > wrt "is it possible to move the post* event after the flush?"... > > There are really 2 answers. > > 1) According to the JPA spec, can we do this?  The quote from the > current spec says: > <quote> > The PreUpdate and PostUpdate callbacks occur before and after the > database update operations to > entity data respectively. These database operations may occur at the > time the entity state is updated or > they may occur at the time state is flushed to the database (which may > be at the end of the transaction). > </quote> > I don't really see anything there that discusses the time-relationship > between the SQL UPDATE execution and the @PostUpdate callback other than > the fact that (obviously) @PostUpdate callback should come after the SQL > UPDATE is issued; but it does not seem to limit *how long after*.  So I > think this is OK from the perspective of the spec. > > 2) Can Hibernate be changed to do this?  Well AnythingIsPossible in > programming, so I guess the question really is *should* we change > Hibernate to do this.  My main concern with this change is the extra > queueing it would require and the corollary memory requirements.  What > happens right now is that those callbacks are executed during the action > (org.hibernate.action.Executable) execution.  Flush puts them into a > queue of actions (org.hibernate.engine.ActionQueue), from which they are > removed as they are executed.  We decided to put the post callbacks in > the actions themselves for assurance-of-execution as well as > encapsulation purposes, which I think are both still worthwhile.  What I > could see as a potential solution would be to do something like we do > for Actions which have "after transaction" tasks to perform: > > http://fisheye.jboss.org/browse/Hibernate/core/trunk/core/src/main/java/o... > > The "executions" list here is a queue of actions which we need to keep > around for later.  I can see something like that in conjunction with a > method on ActionQueue to process that internal 'callbacks' queue after > the entire flush is complete.  Note that this does not address > @PreUpdate. > > We can investigate that though and see what we are talking about in > specific. > > - > > Steve Ebersole > Project Lead > http://hibernate.org > steve(a)hibernate.org > > Principal Software Engineer > JBoss, a division of Red Hat > http://jboss.com > http://redhat.com > steve.ebersole(a)jboss.com > steve.ebersole(a)redhat.com > > > On Fri, 2009-03-06 at 09:07 -0500, Emmanuel Bernard wrote: >> >> Ahhh >> In theory it should not as Hibernate Search reads data in the >> beforeCompletion phase. >> Unless people do not apply changes in a transaction in which case we >> need to execute the read in the post* event. >> >> We will check whether or not people use surrounding transactions >> (Hibernate aware Tx either through JTA or via the direct Hibernate >> Transaction API). >> Alternatively, is it possible to move the post* event after the flush? >> Or create noew events for that? That would solve everybody's issue. >> >> Emmanuel >> >> On  Mar 5, 2009, at 22:54, Steve Ebersole wrote: >> >>> Is this somehow different than the "attempt to load stuff into the PC >>> during flush" scenarios I see in any of these related issues? >>> >>> - >>> >>> Steve Ebersole >>> Project Lead >>> http://hibernate.org >>> steve(a)hibernate.org >>> >>> Principal Software Engineer >>> JBoss, a division of Red Hat >>> http://jboss.com >>> http://redhat.com >>> steve.ebersole(a)jboss.com >>> steve.ebersole(a)redhat.com >>> >>> >>> On Thu, 2009-03-05 at 19:14 -0500, Emmanuel Bernard wrote: >>>> >>>> http://opensource.atlassian.com/projects/hibernate/browse/HHH-3225 >>>> >>>> Steve, any chance you could look at this one, it seems to hit HSearch >>>> users on a regular basis. >>>> _______________________________________________ >>>> hibernate-dev mailing list >>>> hibernate-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> >> > _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

hello On  Mar 13, 2009, at 17:54, Sanne Grinovero wrote: > Hi, > about this issue (HSEARCH-178) I've implemented a patch following your > directions and > your idea is working very well, but I'm having some trouble about the Cool > > configuration > of listeners. > > An additional flush listener is needed; I've patched the > autoregistration but people > not using annotations will have to specify both > DefaultFlushEventListener and the > new IndexWorkFlushEventListener in their configuration. yes. Actually you might have to listen to both flush and auto-flush.

> > I've been able to specify the pair of listeners using programmatic > configuration > and using hibernate.cfg.xml but is this also possible with > hibernate.properties no > > and persistence.xml ? yes add the property hibernate.ejb.event.flush = org.hibernate.ejb.event.EJB3FlushEventListener, o.h.s.e. IndexWorkFlushEventListener http://hibernate.org/hib_docs/entitymanager/reference/en/html_single/#d0e500

> > I couldn't find any docs or examples to register two event listeners for > the > same event in JPA, I'm wondering if instead of adding a listener I should > not extend or wrap the DefaultFlushEventListener so to have only one > listener? I don't like the idea, we introduce arrays of event listeners for that purpose. > > > Would this work for JPA also or should I have to extend the > EJB3FlushEventListener > instead? I see it's different. Yes you would need a different one or different two because of (EJB3AutoFlushEventListener). so not a good idea :) > > > In case the JPA listener should be different than the hibernate > version, how can I > detect the listener I should register in the EventListenerRegister > autoregistration > routine? Another good reason why it's a bad idea.

> > > To be backwards-compatible with our own configuration I've slightly > modified > the patch to work as the old way (loading collections in flush) when > the listener > is not found; a warning is logged saying the listener should be > registered. I don't quite understand why, the new EventListenerREgister will be bundled with the IndexWork Listener always right? What backward compatible mode do you have?

BTW, you should put a warning in the log when this event listener is used. "Applying change to the full-text index before transaction completion. Please use a Hibernate aware transaction (eg org.hibernate.Transaction, javax.persistence.EntityTransaction, JTA transaction with the proper TransactionFactory setting)"

> > > hope we can fix this, > Sanne > > > 2009/3/7 Emmanuel Bernard <emmanuel(a)hibernate.org&gt;: >> >> We discussed the issue with Sanne and for Hibernate Search we have a >> workaround solution that does not penalize Hibernate Core. This solution >> can >> be applied by everybody but it's not the easiest thing on Earth. >> >> The idea is to queue as you said but inside custom event listeners. In >> our >> case some Post* event listeners. This queue is "flushed" in a >> FlushEventListener. This new flush event listener must be registered >> *after* >> the default FlushEventListener. >> >> All this does not require Hibernate Core change and requires minimal >> change >> to the Hibernate Search code and architecture. >> >> On  Mar 6, 2009, at 10:15, Steve Ebersole wrote: >> >>> Not sure what you mean by your "In theory it should not"...  The very >>> nature of @PostUpdate is that it is going to be getting called during a >>> flush cycle... >>> >>> ---- >>> >>> wrt "is it possible to move the post* event after the flush?"... >>> >>> There are really 2 answers. >>> >>> 1) According to the JPA spec, can we do this?  The quote from the >>> current spec says: >>> <quote> >>> The PreUpdate and PostUpdate callbacks occur before and after the >>> database update operations to >>> entity data respectively. These database operations may occur at the >>> time the entity state is updated or >>> they may occur at the time state is flushed to the database (which may >>> be at the end of the transaction). >>> </quote> >>> I don't really see anything there that discusses the time-relationship >>> between the SQL UPDATE execution and the @PostUpdate callback other than >>> the fact that (obviously) @PostUpdate callback should come after the SQL >>> UPDATE is issued; but it does not seem to limit *how long after*.  So I >>> think this is OK from the perspective of the spec. >>> >>> 2) Can Hibernate be changed to do this?  Well AnythingIsPossible in >>> programming, so I guess the question really is *should* we change >>> Hibernate to do this.  My main concern with this change is the extra >>> queueing it would require and the corollary memory requirements.  What >>> happens right now is that those callbacks are executed during the action >>> (org.hibernate.action.Executable) execution.  Flush puts them into a >>> queue of actions (org.hibernate.engine.ActionQueue), from which they are >>> removed as they are executed.  We decided to put the post callbacks in >>> the actions themselves for assurance-of-execution as well as >>> encapsulation purposes, which I think are both still worthwhile.  What I >>> could see as a potential solution would be to do something like we do >>> for Actions which have "after transaction" tasks to perform: >>> >>> >>> http://fisheye.jboss.org/browse/Hibernate/core/trunk/core/src/main/java/o... >>> >>> The "executions" list here is a queue of actions which we need to keep >>> around for later.  I can see something like that in conjunction with a >>> method on ActionQueue to process that internal 'callbacks' queue after >>> the entire flush is complete.  Note that this does not address >>> @PreUpdate. >>> >>> We can investigate that though and see what we are talking about in >>> specific. >>> >>> - >>> >>> Steve Ebersole >>> Project Lead >>> http://hibernate.org >>> steve(a)hibernate.org >>> >>> Principal Software Engineer >>> JBoss, a division of Red Hat >>> http://jboss.com >>> http://redhat.com >>> steve.ebersole(a)jboss.com >>> steve.ebersole(a)redhat.com >>> >>> >>> On Fri, 2009-03-06 at 09:07 -0500, Emmanuel Bernard wrote: >>>> >>>> Ahhh >>>> In theory it should not as Hibernate Search reads data in the >>>> beforeCompletion phase. >>>> Unless people do not apply changes in a transaction in which case we >>>> need to execute the read in the post* event. >>>> >>>> We will check whether or not people use surrounding transactions >>>> (Hibernate aware Tx either through JTA or via the direct Hibernate >>>> Transaction API). >>>> Alternatively, is it possible to move the post* event after the flush? >>>> Or create noew events for that? That would solve everybody's issue. >>>> >>>> Emmanuel >>>> >>>> On  Mar 5, 2009, at 22:54, Steve Ebersole wrote: >>>> >>>>> Is this somehow different than the "attempt to load stuff into the PC >>>>> during flush" scenarios I see in any of these related issues? >>>>> >>>>> - >>>>> >>>>> Steve Ebersole >>>>> Project Lead >>>>> http://hibernate.org >>>>> steve(a)hibernate.org >>>>> >>>>> Principal Software Engineer >>>>> JBoss, a division of Red Hat >>>>> http://jboss.com >>>>> http://redhat.com >>>>> steve.ebersole(a)jboss.com >>>>> steve.ebersole(a)redhat.com >>>>> >>>>> >>>>> On Thu, 2009-03-05 at 19:14 -0500, Emmanuel Bernard wrote: >>>>>> >>>>>> http://opensource.atlassian.com/projects/hibernate/browse/HHH-3225 >>>>>> >>>>>> Steve, any chance you could look at this one, it seems to hit HSearch >>>>>> users on a regular basis. >>>>>> _______________________________________________ >>>>>> hibernate-dev mailing list >>>>>> hibernate-dev(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>>>> >>>> >>> >> >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>