From gunnar at hibernate.org Mon Aug 4 04:30:00 2014 From: gunnar at hibernate.org (Gunnar Morling) Date: Mon, 4 Aug 2014 10:30:00 +0200 Subject: [hibernate-dev] Exposing Hibernate statistics via JMX In-Reply-To: References: Message-ID: Anyone with an idea of what's the recommended way of exposing Hibernate statistics via JMX as of 4.3? Thanks, --Gunnar 2014-07-31 14:02 GMT+02:00 Gunnar Morling : > Steve, all, > > I've been asked about the Hibernate statistics functionality which used to > be accessible via JMX. > > There used to be StatisticsService which was an MBean and which one could > register with the JMX platform. This has been removed a while ago, and > there is the related issue HHH-6190 which still is open. > > Until this is resolved, what is the recommended strategy towards making > the statistics available via JMX? Should users create and export their own > MBean based on SessionFactory#getStatistics()? Or is there any other > replacement for StatisticsService? > > Thanks, > > --Gunnar > > [1] https://hibernate.atlassian.net/browse/HHH-6190 > > From smarlow at redhat.com Tue Aug 5 10:52:10 2014 From: smarlow at redhat.com (Scott Marlow) Date: Tue, 05 Aug 2014 10:52:10 -0400 Subject: [hibernate-dev] JdbcSession proposal In-Reply-To: <53AA3284.3070309@redhat.com> References: <529F65EE.3060308@hibernate.org> <53A1D588.3070204@redhat.com> <53AA3284.3070309@redhat.com> Message-ID: <53E0EF9A.4060306@redhat.com> Steve, This is the test case that I was talking about on IRC. It simulates what happens when several remote EJB calls occur using the same JTA transaction. This case is probably common enough that we should solve it. The workaround is to set hibernate.jta.track_by_thread=false. On 06/24/2014 10:23 PM, Scott Marlow wrote: > I pushed a test case that simulates what can happen with remote EJB > invocations that share the same JTA transaction & EntityManager. The > "Transaction was rolled back in a different thread!" error [2] is thrown > but shouldn't be, since the active application thread has changed to a > different thread (which can happen). The workaround is to set > "hibernate.jta.track_by_thread" to false. > > I think that we need to save the thread id at the start of every > application call into the EntityManager. > > Scott > > [1] https://github.com/scottmarlow/hibernate-orm/tree/sametx_multithreads > > [2] https://gist.github.com/scottmarlow/63c4ab686368853d759b > _______________________________________________ > hibernate-dev mailing list > hibernate-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > From smarlow at redhat.com Tue Aug 5 11:32:46 2014 From: smarlow at redhat.com (Scott Marlow) Date: Tue, 05 Aug 2014 11:32:46 -0400 Subject: [hibernate-dev] JdbcSession proposal In-Reply-To: <53E0EF9A.4060306@redhat.com> References: <529F65EE.3060308@hibernate.org> <53A1D588.3070204@redhat.com> <53AA3284.3070309@redhat.com> <53E0EF9A.4060306@redhat.com> Message-ID: <53E0F91E.6040005@redhat.com> More in-line below. On 08/05/2014 10:52 AM, Scott Marlow wrote: > Steve, > > This is the test case that I was talking about on IRC. It simulates > what happens when several remote EJB calls occur using the same JTA > transaction. This case is probably common enough that we should solve it. > > The workaround is to set hibernate.jta.track_by_thread=false. > > > > On 06/24/2014 10:23 PM, Scott Marlow wrote: >> I pushed a test case that simulates what can happen with remote EJB >> invocations that share the same JTA transaction & EntityManager. The >> "Transaction was rolled back in a different thread!" error [2] is thrown >> but shouldn't be, since the active application thread has changed to a >> different thread (which can happen). The workaround is to set >> "hibernate.jta.track_by_thread" to false. >> >> I think that we need to save the thread id at the start of every >> application call into the EntityManager. I'm wondering if a different approach than using the thread id, would cover more cases (specifically the distributed + remote cases). For example, adding a configuration option for a session level lock that the SessionImpl implements, to protect against two threads calling into the same session concurrently. >> >> Scott >> >> [1] https://github.com/scottmarlow/hibernate-orm/tree/sametx_multithreads >> >> [2] https://gist.github.com/scottmarlow/63c4ab686368853d759b >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > > _______________________________________________ > hibernate-dev mailing list > hibernate-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > From smarlow at redhat.com Wed Aug 6 17:18:21 2014 From: smarlow at redhat.com (Scott Marlow) Date: Wed, 06 Aug 2014 17:18:21 -0400 Subject: [hibernate-dev] [renamed] multiple invocations on same transaction, separate thread is not working unless is changed to false... In-Reply-To: <53E0F91E.6040005@redhat.com> References: <529F65EE.3060308@hibernate.org> <53A1D588.3070204@redhat.com> <53AA3284.3070309@redhat.com> <53E0EF9A.4060306@redhat.com> <53E0F91E.6040005@redhat.com> Message-ID: <53E29B9D.1090305@redhat.com> Either the subject or email isn't working to discuss this. Hence, the rename. On 08/05/2014 11:32 AM, Scott Marlow wrote: > More in-line below. > > On 08/05/2014 10:52 AM, Scott Marlow wrote: >> Steve, >> >> This is the test case that I was talking about on IRC. It simulates >> what happens when several remote EJB calls occur using the same JTA >> transaction. This case is probably common enough that we should solve it. >> >> The workaround is to set hibernate.jta.track_by_thread=false. >> >> >> >> On 06/24/2014 10:23 PM, Scott Marlow wrote: >>> I pushed a test case that simulates what can happen with remote EJB >>> invocations that share the same JTA transaction & EntityManager. The >>> "Transaction was rolled back in a different thread!" error [2] is thrown >>> but shouldn't be, since the active application thread has changed to a >>> different thread (which can happen). The workaround is to set >>> "hibernate.jta.track_by_thread" to false. >>> >>> I think that we need to save the thread id at the start of every >>> application call into the EntityManager. > > I'm wondering if a different approach than using the thread id, would > cover more cases (specifically the distributed + remote cases). For > example, adding a configuration option for a session level lock that the > SessionImpl implements, to protect against two threads calling into the > same session concurrently. > >>> >>> Scott >>> >>> [1] https://github.com/scottmarlow/hibernate-orm/tree/sametx_multithreads >>> >>> [2] https://gist.github.com/scottmarlow/63c4ab686368853d759b >>> _______________________________________________ >>> hibernate-dev mailing list >>> hibernate-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> >> >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > > _______________________________________________ > hibernate-dev mailing list > hibernate-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > From jlesinge at gmail.com Thu Aug 7 04:10:15 2014 From: jlesinge at gmail.com (John Worrell) Date: Thu, 7 Aug 2014 09:10:15 +0100 Subject: [hibernate-dev] Contributing to OGM / Cassandra In-Reply-To: References: <6B5FA68A-59CD-4CAD-81EB-3AC2CCBBE55C@hibernate.org> Message-ID: Hi Emmanuel et al., My apologies for the log radio silence. I've taken a look at the code-base on Jon Halliday's repo, and have set up a nick on freenode - #jlesinge. On the time-series question I was wondering how you envisaged the data stored: I tend to think of a single row under an primary key with an object-instance per column. Now what we have typically done (generally the data has been immutable) is to store the data serialized as a blob (JSON or XML), but I understand you do not favour this approach. With this sort of model I imagine the collection is then all the objects stored in the row, and the challenge is to page through the objects in the row. An approach we have often taken is to create multiple copies of data in different (obviously works well only for immutable objects) or better to create a table of keys to a main table where in either approach the row-keys are effectively a foreign-key and there is column per object associated through the foreign-key. Another approach though might be to use a column with type list (or set, or map) to contain keys to the associated objects - this would be a little like the extensions Oracle have for mapping 1-* associations, though with the caveat that a column of collection type may only contain 64k elements. I wondered if some though had been given to this strategy (which I must admit I have not yet used myself). It seems very likely that different mapping strategies should be specifiable, but then I have still to understand how these might fit with treiid. Can I ask about assumptions: is it fair to assume that for Cassandra, OGM will target only CQL 3 (which means Cassandra 2 or maybe 1.2)? This would certainly make life simpler. An issue I don't see addressed is the choice of consistency-level (read or write) and I wondered if there was a plan for this? Assumptions can be made on a per table basis, but, certainly for ad hoc queries, it is important think to have the flexibility to specify on a per-query basis. Those are my thoughts so far... I'll see about doing a POC of some of what I have described above Cheers, John On Mon, Jul 21, 2014 at 4:48 PM, John Worrell wrote: > Hi Emmanuel, > > I'll take a look at what is there, and I'll get up and running on IRC. > > I'll particularly look at the time-series issue - non-trivial I think. > > Cheers, > > John > > > On Mon, Jul 21, 2014 at 1:06 PM, Emmanuel Bernard > wrote: > >> Hi John, >> >> I thought I had replied to you on Friday but apparently the email never >> went through :/ >> >> That is good news :) >> Jonathan worked on a Cassandra prototype but had to drop due to other >> duties. He pushed everything at >> https://github.com/jhalliday/hibernate-ogm/tree/jonathan_cassandra >> >> Have a look at what he has done and come ask any question to Gunnar, >> Davide or me. There are a bunch of moving pieces. We are mostly on >> freenode?s #hibernate-dev ( you need a freenode login >> http://freenode.net/faq.shtml#nicksetup ). If you are allergic to IRC, >> let me know and we will find alternatives. >> >> The most interesting challenge will be to see how we can map time series >> into a collection and make sure we let the user decide how much he wants to >> load. >> >> Emmanuel >> >> On 16 Jul 2014, at 13:17, John Worrell wrote: >> >> > Hi, >> > >> > I'm interested in contributing to the Cassandra module of Hibernate-OGM >> - >> > what would be the baest way to go about this? >> > >> > Thanks, >> > >> > John >> > _______________________________________________ >> > hibernate-dev mailing list >> > hibernate-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/hibernate-dev >> >> > From sanne at hibernate.org Thu Aug 7 11:02:29 2014 From: sanne at hibernate.org (Sanne Grinovero) Date: Thu, 7 Aug 2014 16:02:29 +0100 Subject: [hibernate-dev] [renamed] multiple invocations on same transaction, separate thread is not working unless is changed to false... In-Reply-To: <53E29B9D.1090305@redhat.com> References: <529F65EE.3060308@hibernate.org> <53A1D588.3070204@redhat.com> <53AA3284.3070309@redhat.com> <53E0EF9A.4060306@redhat.com> <53E0F91E.6040005@redhat.com> <53E29B9D.1090305@redhat.com> Message-ID: Steve, we had a conversation about this on IRC, I think we came to a ? hopefully? nice ? ?solution . [15:18] andrigtmiller: I still think you're locking out the TXM, not making it possible to legitimately timeout queries.. [15:19] (but I don't have the full picture.. not really my area) [15:19] that is a transaction manager concern but they already handled calling other synchronization call backs. currently, Arjuna holds a lock on the collection of synchronizations at that time, so there should be no conflict [15:20] I asked about your concern yesterday to make sure we wouldn't deadlock [15:20] I wasn't actually thinking about deadlock, I just don't see what you're aiming at with the single lock on the Session [15:22] to avoid calling Hibernate session.clear from a background thread while the application thread is also doing a Hibernate session invocation. which can lead to random exceptions [15:22] I'm aiming at solving the concurrency concern ^ [15:22] But I guess you'd need to talk with sebersole, this lock thing you're proposing comes out of the blue for me :) [15:22] yes I get that [15:23] but you should allow the session.clear() to proceed, not lock it out [15:23] sure, I'm just trying to talk to who ever listens and have feedback. Before I get anyway near code changes, Sebersole needs to be on board (he isn't yet) [15:24] I'm neither :) willing to discuss for sake of interest, but can't replace sebersole on this I think :) [15:24] I posted on as7 dev mailing list a few years ago and the only feedback that I got was that only Hibernate was not handling concurrency but there could be others [15:25] what is the effect you expect to have users see when the background thread kills the current session? [15:25] I cross posted as well [15:25] users shouldn't get NPE errors or unexpected exceptions [15:26] sure, but what kind of effect would you propose? [15:26] I guess a different exception with a better error message? [15:26] my goal is ensuring that only one thread is invoking the Hibernate session at a time [15:27] No that's not true, as you're saying that you want to allow the TXM to rollback the transaction [15:27] well, want is a strong word :) [15:28] IBM/Oracle/JBoss all do that [15:28] and that's an implementation detail what you just explained :) [15:28] what I'm asking is what you expect the user to experience when this needs to happen [15:29] when the transaction is rolled back, depending on where the application code is in the Hibernate session invocation, a JDBC error might occur or something related to that. The goal is to avoid mutating the Hibernate session from two different threads concurrently [15:30] you're not answering my question, that's an implementation detail :) [15:30] if we can avoid violating concurrency of the hibernate session, I think we will be more robust. [15:31] The goal of the TXM timeout, is to kill stuff which is taking too long.. not allowing it to finish in a safe lock. andrigtmiller am I understanding the basics correctly? [15:31] currently, what the user will experience is poor. The scope of turning that into a more pleasant experience is a good question but involves many moving and separate parts [15:32] so you actually _need_ to allow concurrent access, to kill and rollback the operations which are being done [15:32] the goal of TXM timeout handling in JBoss/Oracle/IBM is to also handle deadlocks that might not otherwise be recovered from [15:32] the concurrent access happens at the resource level [15:33] from the background, with components that handle concurrency. Hibernate doesn't [15:33] that's just one part of the things [15:33] and we can definitely have Hibernate handle some concurrent events [15:33] Hibernate sessions are not supposed to handle concurrency, so its a design flaw [15:33] the TXM shouldn't invoke clear() (which is public API) but invoke a specific method which could provide the needed semantics [15:33] its not some events, its many events [15:34] why do you say it's a design flaw? it's not, for the "normal" public API usage [15:34] the TXM, calls the Hibernate Synchronization.afterCompletion(int) and Hibernate detaches entities [15:34] and Hibernate does other cleaning up as well [15:35] its a design flaw as its not handled in our system [15:35] I don't feel I'm making progress in the conversation if you keep repeating the implementation details, we can fix that as we please [15:35] but you have to explain the expected user experience [15:36] you mean if we didn't have any constraints of any of the existing specs that we implement? [15:36] yes, just explain what you think it should do please, that would help me understand, and I think I can propose you something more concrete [15:37] if we ignore XA, JTA + JPA, I can answer the question better and will attempt that [15:37] the problem is the user sees an NPE right? [15:38] could be an NPE or any unexpected exception that comes from using a thread-unsafe component from multiple threads [15:38] so the basic question is what do you think the user should "see".. I guess another exception, say HibernateRolledBackException("The Transaction Manager decided we run out of time") [15:38] sure. would you like the above proposal ^ [15:39] Because implementing that is easy :) [15:41] ? If you're busy I'm happy to talk later, I am 3h late with my lunch [15:41] in a perfect world, 1) the user registers an application event listener that tells that the transaction timed out. 2) whether the application thread catches a signal that the transaction is about to be rolled back. 3) The application thread then catches a signal that the transaction was rolled back [15:41] that doesn't explain what you expect the client code to experience. [15:41] 4) the application thread then catches a signal that Synchronization call backs are going to happen to clean up after the roll back [15:42] say the user was incoking a "Query.list()" .. what do we return? [15:42] what could they expect? They could be in the middle of code that doesn't use the Hibernate session or could be in code that does [15:43] Exactly my point, so we'd return an exception like I proposed above right? [15:43] if you say so but no one knows how to do that [15:43] I mean, I'm not sure what returns that [15:44] we had tried doing what you suggest but it was too performance expensive and covered too few cases [15:44] but sure, throwing an EJB rolledback exception is ideal [15:45] you can do it in an efficient way, but yes I agree you'd have to patch several areas of code. [15:45] I'd do it incrementally though, start with the Session and Query APIs, see if people like it [15:45] our previous attempt that failed, added a pre-check at the front of every Hibernate session call, tail end and middle [15:46] but that didn't work because it didn't handle remote transactions or distributed transactions [15:46] the EntityManager implementation has some kind of "exception translator" [15:46] and performance suffered [15:46] all you need to do is catch exceptions, and re-throw the better explanation [15:46] so, we have less of the checking today and still don't handle remote transactions and distributed transactions [15:47] the TXM just needs to flag a volatile field in the Session to inject the error it wants us to tell [15:47] so on the optimal path you just have a volatile field read operation, which is a zero cost essentially [15:47] (optimal path I mean the case in which there are no errors) [15:48] then we need to poll for that flag in the start/middle/end (or so) if every Hibernate method [15:48] no [15:48] or maybe in the finally clause of every method [15:48] you just need to catch exceptions [15:48] and we happen to already do that, because you need to translate all Hibernate specific exceptions to JPA specific ones [15:49] so it's actually a very simple patch with zero overhead I think [15:49] at worse, we catch exceptions (including NPE) and notice the flag and then eat the cause exception? [15:49] +1 [15:49] if we show the cause, its confusing [15:49] you don't need the finally method either though [15:49] if we eat it, its confusing [15:49] right, don't show the cause. [15:49] I hate when exceptions are eaten [15:50] well.. the TXM should provide you a sensible error message, like "aborted because of ... ", and we take that as explanation. [15:50] but only if that flag is set [15:50] of course! [15:50] sounds like a worthwhile idea, thanks [15:51] np, I'm very happy if it works :) [15:51] the exception translation is already done somewhere in the EM project [15:51] I don't remember the details, but I'm sure you're more familiar with it [15:51] just make sure you look that up [15:52] it catches all exceptions from Hibernate to wrap them in something else, to satisfy the spec requirements -- Sanne From gunnar at hibernate.org Thu Aug 7 11:26:24 2014 From: gunnar at hibernate.org (Gunnar Morling) Date: Thu, 7 Aug 2014 17:26:24 +0200 Subject: [hibernate-dev] Using raw vs. wildcard types Message-ID: Hi, every now and then I run into usages of raw types in our code bases: * Class entityType (instead of Class) * Set querySpaces (instead of Set) ... I first thought that'd be left-overs from ancient times but apparently this is also done in newly written code. I don't like it as in some cases it's clearly a loss of information which requires investigation in surrounding code to find out e.g. which types are put into a raw Set. In other cases (e.g. Class) I prefer to use the wildcard type (Class) as it avoids many raw-type warnings in the IDE (which otherwise may obfuscate legitimate warnings). Is there any reason for using raw types in new code rather than full generic or wildcard types? Personally I try to fix existing usages of raw types and avoid to add new ones if possible, but sometimes I can't (e.g. one cannot override a method expecting a Class with a method expecting a Class). Thanks, --Gunnar From smarlow at redhat.com Thu Aug 7 11:33:29 2014 From: smarlow at redhat.com (Scott Marlow) Date: Thu, 07 Aug 2014 11:33:29 -0400 Subject: [hibernate-dev] [renamed] multiple invocations on same transaction, separate thread is not working unless is changed to false... In-Reply-To: References: <529F65EE.3060308@hibernate.org> <53A1D588.3070204@redhat.com> <53AA3284.3070309@redhat.com> <53E0EF9A.4060306@redhat.com> <53E0F91E.6040005@redhat.com> <53E29B9D.1090305@redhat.com> Message-ID: <53E39C49.5010309@redhat.com> Sanne, One question that I didn't ask before, when the Hibernate Synchronization.afterCompletion(int) is called (with rolledback TX status), how does Hibernate know if the transaction was rolled back from the transaction reaper thread? I tried checking the thread name in WildFly integration code but that is wrong, as the thread name may be a transaction manager communications thread that is invoked on behalf of a remote (JVM) reaper thread. Currently, are checking thread ids which does not work for the distributed case involving multiple jvms. The current approach also does not work for remote invocations that use the same JTA transaction (as the below test case simulates). Scott On 08/07/2014 11:02 AM, Sanne Grinovero wrote: > Steve, > we had a conversation about this on IRC, I think we came to a > ? hopefully? > nice > ? > ?solution > . > > [15:18] andrigtmiller: I still think you're locking out > the TXM, not making it possible to legitimately timeout queries.. > [15:19] (but I don't have the full picture.. not really > my area) > [15:19] that is a transaction manager concern but they already > handled calling other synchronization call backs. currently, Arjuna > holds a lock on the collection of synchronizations at that time, so > there should be no conflict > [15:20] I asked about your concern yesterday to make sure we > wouldn't deadlock > [15:20] I wasn't actually thinking about deadlock, I > just don't see what you're aiming at with the single lock on the Session > [15:22] to avoid calling Hibernate session.clear from a > background thread while the application thread is also doing a Hibernate > session invocation. which can lead to random exceptions > [15:22] I'm aiming at solving the concurrency concern ^ > [15:22] But I guess you'd need to talk with sebersole, > this lock thing you're proposing comes out of the blue for me :) > [15:22] yes I get that > [15:23] but you should allow the session.clear() to > proceed, not lock it out > [15:23] sure, I'm just trying to talk to who ever listens and > have feedback. Before I get anyway near code changes, Sebersole needs > to be on board (he isn't yet) > [15:24] I'm neither :) willing to discuss for sake of > interest, but can't replace sebersole on this I think :) > [15:24] I posted on as7 dev mailing list a few years ago and > the only feedback that I got was that only Hibernate was not handling > concurrency but there could be others > [15:25] what is the effect you expect to have users see > when the background thread kills the current session? > [15:25] I cross posted as well > [15:25] users shouldn't get NPE errors or unexpected exceptions > [15:26] sure, but what kind of effect would you propose? > [15:26] I guess a different exception with a better > error message? > [15:26] my goal is ensuring that only one thread is invoking > the Hibernate session at a time > [15:27] No that's not true, as you're saying that you > want to allow the TXM to rollback the transaction > [15:27] well, want is a strong word :) > [15:28] IBM/Oracle/JBoss all do that > [15:28] and that's an implementation detail what you > just explained :) > [15:28] what I'm asking is what you expect the user to > experience when this needs to happen > [15:29] when the transaction is rolled back, depending on > where the application code is in the Hibernate session invocation, a > JDBC error might occur or something related to that. The goal is to > avoid mutating the Hibernate session from two different threads concurrently > [15:30] you're not answering my question, that's an > implementation detail :) > [15:30] if we can avoid violating concurrency of the hibernate > session, I think we will be more robust. > [15:31] The goal of the TXM timeout, is to kill stuff > which is taking too long.. not allowing it to finish in a safe lock. > andrigtmiller am I understanding the basics correctly? > [15:31] currently, what the user will experience is poor. The > scope of turning that into a more pleasant experience is a good question > but involves many moving and separate parts > [15:32] so you actually _need_ to allow concurrent > access, to kill and rollback the operations which are being done > [15:32] the goal of TXM timeout handling in JBoss/Oracle/IBM > is to also handle deadlocks that might not otherwise be recovered from > [15:32] the concurrent access happens at the resource level > [15:33] from the background, with components that handle > concurrency. Hibernate doesn't > [15:33] that's just one part of the things > [15:33] and we can definitely have Hibernate handle > some concurrent events > [15:33] Hibernate sessions are not supposed to handle > concurrency, so its a design flaw > [15:33] the TXM shouldn't invoke clear() (which is > public API) but invoke a specific method which could provide the needed > semantics > [15:33] its not some events, its many events > [15:34] why do you say it's a design flaw? it's not, > for the "normal" public API usage > [15:34] the TXM, calls the Hibernate > Synchronization.afterCompletion(int) and Hibernate detaches entities > [15:34] and Hibernate does other cleaning up as well > [15:35] its a design flaw as its not handled in our system > [15:35] I don't feel I'm making progress in the > conversation if you keep repeating the implementation details, we can > fix that as we please > [15:35] but you have to explain the expected user > experience > [15:36] you mean if we didn't have any constraints of any of > the existing specs that we implement? > [15:36] yes, just explain what you think it should do > please, that would help me understand, and I think I can propose you > something more concrete > [15:37] if we ignore XA, JTA + JPA, I can answer the question > better and will attempt that > [15:37] the problem is the user sees an NPE right? > [15:38] could be an NPE or any unexpected exception that comes > from using a thread-unsafe component from multiple threads > [15:38] so the basic question is what do you think the > user should "see".. I guess another exception, say > HibernateRolledBackException("The Transaction Manager decided we run out > of time") > [15:38] sure. would you like the above proposal ^ > [15:39] Because implementing that is easy :) > [15:41] ? If you're busy I'm happy to talk later, I am > 3h late with my lunch > [15:41] in a perfect world, 1) the user registers an > application event listener that tells that the transaction timed out. > 2) whether the application thread catches a signal that the > transaction is about to be rolled back. 3) The application thread then > catches a signal that the transaction was rolled back > [15:41] that doesn't explain what you expect the client > code to experience. > [15:41] 4) the application thread then catches a signal that > Synchronization call backs are going to happen to clean up after the > roll back > [15:42] say the user was incoking a "Query.list()" .. > what do we return? > [15:42] what could they expect? They could be in the middle > of code that doesn't use the Hibernate session or could be in code that does > [15:43] Exactly my point, so we'd return an exception > like I proposed above right? > [15:43] if you say so but no one knows how to do that > [15:43] I mean, I'm not sure what returns that > [15:44] we had tried doing what you suggest but it was too > performance expensive and covered too few cases > [15:44] but sure, throwing an EJB rolledback exception is ideal > [15:45] you can do it in an efficient way, but yes I > agree you'd have to patch several areas of code. > [15:45] I'd do it incrementally though, start with the > Session and Query APIs, see if people like it > [15:45] our previous attempt that failed, added a pre-check at > the front of every Hibernate session call, tail end and middle > [15:46] but that didn't work because it didn't handle remote > transactions or distributed transactions > [15:46] the EntityManager implementation has some kind > of "exception translator" > [15:46] and performance suffered > [15:46] all you need to do is catch exceptions, and > re-throw the better explanation > [15:46] so, we have less of the checking today and still don't > handle remote transactions and distributed transactions > [15:47] the TXM just needs to flag a volatile field in > the Session to inject the error it wants us to tell > [15:47] so on the optimal path you just have a volatile > field read operation, which is a zero cost essentially > [15:47] (optimal path I mean the case in which there > are no errors) > [15:48] then we need to poll for that flag in the > start/middle/end (or so) if every Hibernate method > [15:48] no > [15:48] or maybe in the finally clause of every method > [15:48] you just need to catch exceptions > [15:48] and we happen to already do that, because you > need to translate all Hibernate specific exceptions to JPA specific ones > [15:49] so it's actually a very simple patch with zero > overhead I think > [15:49] at worse, we catch exceptions (including NPE) and > notice the flag and then eat the cause exception? > [15:49] +1 > [15:49] if we show the cause, its confusing > [15:49] you don't need the finally method either though > [15:49] if we eat it, its confusing > [15:49] right, don't show the cause. > [15:49] I hate when exceptions are eaten > [15:50] well.. the TXM should provide you a sensible > error message, like "aborted because of ... ", and we take that as > explanation. > [15:50] but only if that flag is set > [15:50] of course! > [15:50] sounds like a worthwhile idea, thanks > [15:51] np, I'm very happy if it works :) > [15:51] the exception translation is already done > somewhere in the EM project > [15:51] I don't remember the details, but I'm sure > you're more familiar with it > [15:51] just make sure you look that up > [15:52] it catches all exceptions from Hibernate to > wrap them in something else, to satisfy the spec requirements > > -- Sanne From smarlow at redhat.com Thu Aug 7 11:43:19 2014 From: smarlow at redhat.com (Scott Marlow) Date: Thu, 07 Aug 2014 11:43:19 -0400 Subject: [hibernate-dev] [renamed] multiple invocations on same transaction, separate thread is not working unless is changed to false... In-Reply-To: <53E39C49.5010309@redhat.com> References: <529F65EE.3060308@hibernate.org> <53A1D588.3070204@redhat.com> <53AA3284.3070309@redhat.com> <53E0EF9A.4060306@redhat.com> <53E0F91E.6040005@redhat.com> <53E29B9D.1090305@redhat.com> <53E39C49.5010309@redhat.com> Message-ID: <53E39E97.3010603@redhat.com> Also, detecting that the background thread may of violated the Hibernate session concurrency requirements for only one thread to use it at a time, is not as good as preventing the concurrency violation. If anyone wants to also give feedback on the Transaction Manager side of this, the forum thread is https://community.jboss.org/thread/243360. On 08/07/2014 11:33 AM, Scott Marlow wrote: > Sanne, > > One question that I didn't ask before, when the Hibernate > Synchronization.afterCompletion(int) is called (with rolledback TX > status), how does Hibernate know if the transaction was rolled back from > the transaction reaper thread? I tried checking the thread name in > WildFly integration code but that is wrong, as the thread name may be a > transaction manager communications thread that is invoked on behalf of a > remote (JVM) reaper thread. > > Currently, are checking thread ids which does not work for the > distributed case involving multiple jvms. The current approach also > does not work for remote invocations that use the same JTA transaction > (as the below test case simulates). > > Scott > > On 08/07/2014 11:02 AM, Sanne Grinovero wrote: >> Steve, >> we had a conversation about this on IRC, I think we came to a >> ? hopefully? >> nice >> ? >> ?solution >> . >> >> [15:18] andrigtmiller: I still think you're locking out >> the TXM, not making it possible to legitimately timeout queries.. >> [15:19] (but I don't have the full picture.. not really >> my area) >> [15:19] that is a transaction manager concern but they already >> handled calling other synchronization call backs. currently, Arjuna >> holds a lock on the collection of synchronizations at that time, so >> there should be no conflict >> [15:20] I asked about your concern yesterday to make sure we >> wouldn't deadlock >> [15:20] I wasn't actually thinking about deadlock, I >> just don't see what you're aiming at with the single lock on the Session >> [15:22] to avoid calling Hibernate session.clear from a >> background thread while the application thread is also doing a Hibernate >> session invocation. which can lead to random exceptions >> [15:22] I'm aiming at solving the concurrency concern ^ >> [15:22] But I guess you'd need to talk with sebersole, >> this lock thing you're proposing comes out of the blue for me :) >> [15:22] yes I get that >> [15:23] but you should allow the session.clear() to >> proceed, not lock it out >> [15:23] sure, I'm just trying to talk to who ever listens and >> have feedback. Before I get anyway near code changes, Sebersole needs >> to be on board (he isn't yet) >> [15:24] I'm neither :) willing to discuss for sake of >> interest, but can't replace sebersole on this I think :) >> [15:24] I posted on as7 dev mailing list a few years ago and >> the only feedback that I got was that only Hibernate was not handling >> concurrency but there could be others >> [15:25] what is the effect you expect to have users see >> when the background thread kills the current session? >> [15:25] I cross posted as well >> [15:25] users shouldn't get NPE errors or unexpected exceptions >> [15:26] sure, but what kind of effect would you propose? >> [15:26] I guess a different exception with a better >> error message? >> [15:26] my goal is ensuring that only one thread is invoking >> the Hibernate session at a time >> [15:27] No that's not true, as you're saying that you >> want to allow the TXM to rollback the transaction >> [15:27] well, want is a strong word :) >> [15:28] IBM/Oracle/JBoss all do that >> [15:28] and that's an implementation detail what you >> just explained :) >> [15:28] what I'm asking is what you expect the user to >> experience when this needs to happen >> [15:29] when the transaction is rolled back, depending on >> where the application code is in the Hibernate session invocation, a >> JDBC error might occur or something related to that. The goal is to >> avoid mutating the Hibernate session from two different threads concurrently >> [15:30] you're not answering my question, that's an >> implementation detail :) >> [15:30] if we can avoid violating concurrency of the hibernate >> session, I think we will be more robust. >> [15:31] The goal of the TXM timeout, is to kill stuff >> which is taking too long.. not allowing it to finish in a safe lock. >> andrigtmiller am I understanding the basics correctly? >> [15:31] currently, what the user will experience is poor. The >> scope of turning that into a more pleasant experience is a good question >> but involves many moving and separate parts >> [15:32] so you actually _need_ to allow concurrent >> access, to kill and rollback the operations which are being done >> [15:32] the goal of TXM timeout handling in JBoss/Oracle/IBM >> is to also handle deadlocks that might not otherwise be recovered from >> [15:32] the concurrent access happens at the resource level >> [15:33] from the background, with components that handle >> concurrency. Hibernate doesn't >> [15:33] that's just one part of the things >> [15:33] and we can definitely have Hibernate handle >> some concurrent events >> [15:33] Hibernate sessions are not supposed to handle >> concurrency, so its a design flaw >> [15:33] the TXM shouldn't invoke clear() (which is >> public API) but invoke a specific method which could provide the needed >> semantics >> [15:33] its not some events, its many events >> [15:34] why do you say it's a design flaw? it's not, >> for the "normal" public API usage >> [15:34] the TXM, calls the Hibernate >> Synchronization.afterCompletion(int) and Hibernate detaches entities >> [15:34] and Hibernate does other cleaning up as well >> [15:35] its a design flaw as its not handled in our system >> [15:35] I don't feel I'm making progress in the >> conversation if you keep repeating the implementation details, we can >> fix that as we please >> [15:35] but you have to explain the expected user >> experience >> [15:36] you mean if we didn't have any constraints of any of >> the existing specs that we implement? >> [15:36] yes, just explain what you think it should do >> please, that would help me understand, and I think I can propose you >> something more concrete >> [15:37] if we ignore XA, JTA + JPA, I can answer the question >> better and will attempt that >> [15:37] the problem is the user sees an NPE right? >> [15:38] could be an NPE or any unexpected exception that comes >> from using a thread-unsafe component from multiple threads >> [15:38] so the basic question is what do you think the >> user should "see".. I guess another exception, say >> HibernateRolledBackException("The Transaction Manager decided we run out >> of time") >> [15:38] sure. would you like the above proposal ^ >> [15:39] Because implementing that is easy :) >> [15:41] ? If you're busy I'm happy to talk later, I am >> 3h late with my lunch >> [15:41] in a perfect world, 1) the user registers an >> application event listener that tells that the transaction timed out. >> 2) whether the application thread catches a signal that the >> transaction is about to be rolled back. 3) The application thread then >> catches a signal that the transaction was rolled back >> [15:41] that doesn't explain what you expect the client >> code to experience. >> [15:41] 4) the application thread then catches a signal that >> Synchronization call backs are going to happen to clean up after the >> roll back >> [15:42] say the user was incoking a "Query.list()" .. >> what do we return? >> [15:42] what could they expect? They could be in the middle >> of code that doesn't use the Hibernate session or could be in code that does >> [15:43] Exactly my point, so we'd return an exception >> like I proposed above right? >> [15:43] if you say so but no one knows how to do that >> [15:43] I mean, I'm not sure what returns that >> [15:44] we had tried doing what you suggest but it was too >> performance expensive and covered too few cases >> [15:44] but sure, throwing an EJB rolledback exception is ideal >> [15:45] you can do it in an efficient way, but yes I >> agree you'd have to patch several areas of code. >> [15:45] I'd do it incrementally though, start with the >> Session and Query APIs, see if people like it >> [15:45] our previous attempt that failed, added a pre-check at >> the front of every Hibernate session call, tail end and middle >> [15:46] but that didn't work because it didn't handle remote >> transactions or distributed transactions >> [15:46] the EntityManager implementation has some kind >> of "exception translator" >> [15:46] and performance suffered >> [15:46] all you need to do is catch exceptions, and >> re-throw the better explanation >> [15:46] so, we have less of the checking today and still don't >> handle remote transactions and distributed transactions >> [15:47] the TXM just needs to flag a volatile field in >> the Session to inject the error it wants us to tell >> [15:47] so on the optimal path you just have a volatile >> field read operation, which is a zero cost essentially >> [15:47] (optimal path I mean the case in which there >> are no errors) >> [15:48] then we need to poll for that flag in the >> start/middle/end (or so) if every Hibernate method >> [15:48] no >> [15:48] or maybe in the finally clause of every method >> [15:48] you just need to catch exceptions >> [15:48] and we happen to already do that, because you >> need to translate all Hibernate specific exceptions to JPA specific ones >> [15:49] so it's actually a very simple patch with zero >> overhead I think >> [15:49] at worse, we catch exceptions (including NPE) and >> notice the flag and then eat the cause exception? >> [15:49] +1 >> [15:49] if we show the cause, its confusing >> [15:49] you don't need the finally method either though >> [15:49] if we eat it, its confusing >> [15:49] right, don't show the cause. >> [15:49] I hate when exceptions are eaten >> [15:50] well.. the TXM should provide you a sensible >> error message, like "aborted because of ... ", and we take that as >> explanation. >> [15:50] but only if that flag is set >> [15:50] of course! >> [15:50] sounds like a worthwhile idea, thanks >> [15:51] np, I'm very happy if it works :) >> [15:51] the exception translation is already done >> somewhere in the EM project >> [15:51] I don't remember the details, but I'm sure >> you're more familiar with it >> [15:51] just make sure you look that up >> [15:52] it catches all exceptions from Hibernate to >> wrap them in something else, to satisfy the spec requirements >> >> -- Sanne > > _______________________________________________ > hibernate-dev mailing list > hibernate-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > From steve at hibernate.org Thu Aug 7 13:10:39 2014 From: steve at hibernate.org (Steve Ebersole) Date: Thu, 7 Aug 2014 12:10:39 -0500 Subject: [hibernate-dev] [renamed] multiple invocations on same transaction, separate thread is not working unless is changed to false... In-Reply-To: References: <529F65EE.3060308@hibernate.org> <53A1D588.3070204@redhat.com> <53AA3284.3070309@redhat.com> <53E0EF9A.4060306@redhat.com> <53E0F91E.6040005@redhat.com> <53E29B9D.1090305@redhat.com> Message-ID: Don't y'all believe in summarization? ;) I'll read through this tonight/tomorrow... On Thu, Aug 7, 2014 at 10:02 AM, Sanne Grinovero wrote: > Steve, > we had a conversation about this on IRC, I think we came to a > ? hopefully? > nice > ? > ?solution > . > > [15:18] andrigtmiller: I still think you're locking out > the TXM, not making it possible to legitimately timeout queries.. > [15:19] (but I don't have the full picture.. not really > my area) > [15:19] that is a transaction manager concern but they already > handled calling other synchronization call backs. currently, Arjuna holds > a lock on the collection of synchronizations at that time, so there should > be no conflict > [15:20] I asked about your concern yesterday to make sure we > wouldn't deadlock > [15:20] I wasn't actually thinking about deadlock, I just > don't see what you're aiming at with the single lock on the Session > [15:22] to avoid calling Hibernate session.clear from a > background thread while the application thread is also doing a Hibernate > session invocation. which can lead to random exceptions > [15:22] I'm aiming at solving the concurrency concern ^ > [15:22] But I guess you'd need to talk with sebersole, > this lock thing you're proposing comes out of the blue for me :) > [15:22] yes I get that > [15:23] but you should allow the session.clear() to > proceed, not lock it out > [15:23] sure, I'm just trying to talk to who ever listens and > have feedback. Before I get anyway near code changes, Sebersole needs to > be on board (he isn't yet) > [15:24] I'm neither :) willing to discuss for sake of > interest, but can't replace sebersole on this I think :) > [15:24] I posted on as7 dev mailing list a few years ago and the > only feedback that I got was that only Hibernate was not handling > concurrency but there could be others > [15:25] what is the effect you expect to have users see > when the background thread kills the current session? > [15:25] I cross posted as well > [15:25] users shouldn't get NPE errors or unexpected exceptions > [15:26] sure, but what kind of effect would you propose? > [15:26] I guess a different exception with a better error > message? > [15:26] my goal is ensuring that only one thread is invoking the > Hibernate session at a time > [15:27] No that's not true, as you're saying that you > want to allow the TXM to rollback the transaction > [15:27] well, want is a strong word :) > [15:28] IBM/Oracle/JBoss all do that > [15:28] and that's an implementation detail what you just > explained :) > [15:28] what I'm asking is what you expect the user to > experience when this needs to happen > [15:29] when the transaction is rolled back, depending on where > the application code is in the Hibernate session invocation, a JDBC error > might occur or something related to that. The goal is to avoid mutating > the Hibernate session from two different threads concurrently > [15:30] you're not answering my question, that's an > implementation detail :) > [15:30] if we can avoid violating concurrency of the hibernate > session, I think we will be more robust. > [15:31] The goal of the TXM timeout, is to kill stuff > which is taking too long.. not allowing it to finish in a safe lock. > andrigtmiller am I understanding the basics correctly? > [15:31] currently, what the user will experience is poor. The > scope of turning that into a more pleasant experience is a good question > but involves many moving and separate parts > [15:32] so you actually _need_ to allow concurrent > access, to kill and rollback the operations which are being done > [15:32] the goal of TXM timeout handling in JBoss/Oracle/IBM is > to also handle deadlocks that might not otherwise be recovered from > [15:32] the concurrent access happens at the resource level > [15:33] from the background, with components that handle > concurrency. Hibernate doesn't > [15:33] that's just one part of the things > [15:33] and we can definitely have Hibernate handle some > concurrent events > [15:33] Hibernate sessions are not supposed to handle > concurrency, so its a design flaw > [15:33] the TXM shouldn't invoke clear() (which is public > API) but invoke a specific method which could provide the needed semantics > [15:33] its not some events, its many events > [15:34] why do you say it's a design flaw? it's not, for > the "normal" public API usage > [15:34] the TXM, calls the Hibernate > Synchronization.afterCompletion(int) and Hibernate detaches entities > [15:34] and Hibernate does other cleaning up as well > [15:35] its a design flaw as its not handled in our system > [15:35] I don't feel I'm making progress in the > conversation if you keep repeating the implementation details, we can fix > that as we please > [15:35] but you have to explain the expected user > experience > [15:36] you mean if we didn't have any constraints of any of the > existing specs that we implement? > [15:36] yes, just explain what you think it should do > please, that would help me understand, and I think I can propose you > something more concrete > [15:37] if we ignore XA, JTA + JPA, I can answer the question > better and will attempt that > [15:37] the problem is the user sees an NPE right? > [15:38] could be an NPE or any unexpected exception that comes > from using a thread-unsafe component from multiple threads > [15:38] so the basic question is what do you think the > user should "see".. I guess another exception, say > HibernateRolledBackException("The Transaction Manager decided we run out of > time") > [15:38] sure. would you like the above proposal ^ > [15:39] Because implementing that is easy :) > [15:41] ? If you're busy I'm happy to talk later, I am 3h > late with my lunch > [15:41] in a perfect world, 1) the user registers an application > event listener that tells that the transaction timed out. 2) whether the > application thread catches a signal that the transaction is about to be > rolled back. 3) The application thread then catches a signal that the > transaction was rolled back > [15:41] that doesn't explain what you expect the client > code to experience. > [15:41] 4) the application thread then catches a signal that > Synchronization call backs are going to happen to clean up after the roll > back > [15:42] say the user was incoking a "Query.list()" .. > what do we return? > [15:42] what could they expect? They could be in the middle of > code that doesn't use the Hibernate session or could be in code that does > [15:43] Exactly my point, so we'd return an exception > like I proposed above right? > [15:43] if you say so but no one knows how to do that > [15:43] I mean, I'm not sure what returns that > [15:44] we had tried doing what you suggest but it was too > performance expensive and covered too few cases > [15:44] but sure, throwing an EJB rolledback exception is ideal > [15:45] you can do it in an efficient way, but yes I > agree you'd have to patch several areas of code. > [15:45] I'd do it incrementally though, start with the > Session and Query APIs, see if people like it > [15:45] our previous attempt that failed, added a pre-check at > the front of every Hibernate session call, tail end and middle > [15:46] but that didn't work because it didn't handle remote > transactions or distributed transactions > [15:46] the EntityManager implementation has some kind of > "exception translator" > [15:46] and performance suffered > [15:46] all you need to do is catch exceptions, and > re-throw the better explanation > [15:46] so, we have less of the checking today and still don't > handle remote transactions and distributed transactions > [15:47] the TXM just needs to flag a volatile field in > the Session to inject the error it wants us to tell > [15:47] so on the optimal path you just have a volatile > field read operation, which is a zero cost essentially > [15:47] (optimal path I mean the case in which there are > no errors) > [15:48] then we need to poll for that flag in the > start/middle/end (or so) if every Hibernate method > [15:48] no > [15:48] or maybe in the finally clause of every method > [15:48] you just need to catch exceptions > [15:48] and we happen to already do that, because you > need to translate all Hibernate specific exceptions to JPA specific ones > [15:49] so it's actually a very simple patch with zero > overhead I think > [15:49] at worse, we catch exceptions (including NPE) and notice > the flag and then eat the cause exception? > [15:49] +1 > [15:49] if we show the cause, its confusing > [15:49] you don't need the finally method either though > [15:49] if we eat it, its confusing > [15:49] right, don't show the cause. > [15:49] I hate when exceptions are eaten > [15:50] well.. the TXM should provide you a sensible > error message, like "aborted because of ... ", and we take that as > explanation. > [15:50] but only if that flag is set > [15:50] of course! > [15:50] sounds like a worthwhile idea, thanks > [15:51]