[JBoss JIRA] (JBTM-529) Provide a way to disable connection pooling within a transaction
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-529?page=com.atlassian.jira.plugin.s... ]
Tom Jenkinson updated JBTM-529:
-------------------------------
Fix Version/s: 5.0.0.Final
(was: 5.0.0.M3)
> Provide a way to disable connection pooling within a transaction
> ----------------------------------------------------------------
>
> Key: JBTM-529
> URL: https://issues.jboss.org/browse/JBTM-529
> Project: JBoss Transaction Manager
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: Resource Manager
> Affects Versions: 4.5.0, 4.6.0
> Environment: JBossJTA + DataSource with connection pooling (i.e.: DBCP) + Spring
> Reporter: Mauro Molinari
> Assignee: Tom Jenkinson
> Priority: Critical
> Fix For: 5.0.0.Final
>
> Attachments: patch.txt
>
>
> Suppose you are using JBossTS within you webapp, configured using Spring.
> Suppose you're doing connection pooling over JBossTS: that is, you have your data source with connection pooling managing connections obtained from the TransactionalDriver of JBossJTA.
> As of now, com.arjuna.ats.jdbc.TransactionalDriver.connect(String, Properties) is calling com.arjuna.ats.internal.jdbc.ConnectionManager.create(String, Properties) to get a connection. The latter is behaving like this:
> - if an "equivalent" connection has been already created, it's not closed and it is associated with the same transaction, return that connection
> - otherwise create a new one
> In other words, as stated by the comments in ConnectionManager source, this class is pooling connections for the duration of a transaction.
> This however this can lead to severe and hard-to-find problems when you're doing your own connection pooling above JBossJTA using a data source like DBCP. These are two scenarios in which problems can occur:
> 1) suppose you have two different data sources that are modeling two logically different databases. However, suppose you're configuring both of them to point to the same physical database (that is: same URL, same username, same password) and you're off course using the same dynamic class to drive the TransactionalDriver to the actual JDBC driver.
> Suppose transaction 1 (tx1) is started and, within its duration, a connection is asked to data source 1 (ds1) and another connection is asked to data source 2 (ds2). ds1 and ds2 are initially empty, so the request leads to the creation of two new connections, through TransactionalDriver.connect. Therefore, ds1 asks the TransactionalDriver for a new connection (conn1) and the TransactionalDriver asks the ConnectionManager to create it. conn1 is created and put into ds1, then made available to the client code. But also ds2 is requesting a new connection to the TransactionalDriver: this asks the ConnectionManager, which in turns realizes that it has been requested a connection for the same transaction (tx1), for the same URL, username and password and using the same dynamic class... then, instead of creating another connection (conn2), thanks to its pooling algorythm it returns again conn1, which then goes into ds2! Anyway, tx1 terminates without problems, because the physical database is actually the same.
> However, after tx1 has terminated and new transactions are created, because conn1 is in both ds1 and ds2, it may happen that ds1 gives away conn1 to be used for a transaction tx2 while ds2 gives away conn1 to be used for another concurrent transaction tx3. Then, during tx2 or tx3 execution the "Checking transaction and found that this connection is already associated with a different transaction!" error occurs.
> To work around this problem you could configure each data source to use a different implementation (that is, different class names!) of the dynamic class. However, another problem could still happen.
> 2) suppose you work around problem 1) by using a different dynamic class implementation for each data source. Now, suppose tx4 is started and, during its execution, a thread requests two different connections to ds1. Because of the pooling done by ConnectionManager, it may happen that ds1 receives conn1 twice from the TransactionalDriver, while thinking to receive two DIFFERENT connections. Now, suppose that the data source implementation can handle that without corrupting its internal data structures (and, based on my experiences with DBCP, I wouldn't bet on this!), however this causes conn1 to appear twice in ds1. Then, if tx5 and tx6 are two new concurrent transactions during which requests to ds1 are made, it can happen that conn1 is given to be used in both tx5 and tx6, producing again the already mentioned error.
> So, my request is to support a configuration like the one I described by adding the ability to disable connection pooling at transaction level completely.
> I'm attaching a suggested patch that provides this functionality by:
> - adding support for a connection property named POOL_CONNECTIONS_WITHIN_TRANSACTION wich is assumed to be "true" by default but that can be set to "false" as needed
> - changing ConnectionManager.create(String, Properties) to honour that property: if it is set to "false", pooling is disabled and a new connection is always created
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 3 months
[JBoss JIRA] (JBTM-860) use XAResourceWrapper metadata for assume complete
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-860?page=com.atlassian.jira.plugin.s... ]
Tom Jenkinson updated JBTM-860:
-------------------------------
Fix Version/s: 5.0.0.Final
(was: 5.0.0.M3)
> use XAResourceWrapper metadata for assume complete
> --------------------------------------------------
>
> Key: JBTM-860
> URL: https://issues.jboss.org/browse/JBTM-860
> Project: JBoss Transaction Manager
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: Resource Manager
> Affects Versions: 4.15.0
> Reporter: Jonathan Halliday
> Assignee: Tom Jenkinson
> Fix For: 5.0.0.Final
>
>
> In the XA protocol a time window exists wherein the RM has committed and thus forgotten a tx branch but the TM has not yet deleted its log. A crash during this window currently results in an unrecoverable situation, as the TM assumes the branch belongs to an RM that is uncontactable and will retry recovery indefinitely. This stems from an inability to relate the Xid to a specific RM. It can be overridden globally with JTAEnvironmentBean.xaAssumeRecoveryComplete, but we would prefer more fine-grained control. With the availability of RM id information from XAResourceWrapper this becomes feasible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 3 months
[JBoss JIRA] (JBTM-764) ConnectionImple/RecoverableConnection leak in ConnectionManager because of the pooling
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-764?page=com.atlassian.jira.plugin.s... ]
Tom Jenkinson updated JBTM-764:
-------------------------------
Fix Version/s: 5.0.0.Final
(was: 5.0.0.M3)
> ConnectionImple/RecoverableConnection leak in ConnectionManager because of the pooling
> --------------------------------------------------------------------------------------
>
> Key: JBTM-764
> URL: https://issues.jboss.org/browse/JBTM-764
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Resource Manager
> Affects Versions: 4.6.1, 4.11.0
> Environment: JBossTS JTA embedded in a webapp with Spring
> Reporter: Mauro Molinari
> Assignee: Tom Jenkinson
> Priority: Critical
> Fix For: 5.0.0.Final
>
>
> This is indirectly related to JBTM-529.
> Suppose you are using JBossTS JTA within your webapp, configured using Spring and using the DynamicClass mechanism to get connections.
> Suppose you're doing connection pooling under JBossTS: that is, the DynamicClass implementation is pooling XA connections from the actual XADataSource provided by the DBMS JDBC driver. This is a different approach from the one described in JBTM-529 and we switched to this one because of the problems encountered with JBTM-529, thinking that it could be a more correct approach.
> In this scenario, the client application code is doing the following:
> - start a transaction
> - request a new JDBC connection: this can be done either by using the DriverManager or by using Spring SimpleDriverDataSource; JBossTS JTA TransactionalDriver actually creates a new ConnectionImple, which asks the DynamicClass implementation to get a new XAConnection and gets a physical JDBC connection from that XAConnection; because of the XAConnection pooling provided by the DynamicClass implementation, this XAConnection might be a new one or a reused one, but I think this is not relevant to the problem
> - do what you have to do with the connection
> - close the connection: this causes the invocation of ConnectionImple.close
> - do other things within the transaction
> - commit the transaction
> Now, what I see is this: com.arjuna.ats.internal.jdbc.ConnectionImple.close() does the following check:
> if (!_recoveryConnection.inuse())
> {
> ConnectionManager.remove(this); // finalize?
> }
> The problem is that since the transaction is still active, _recoveryConnection.inuse() returns true, because it checks for the XAResource associated with it: in the case of DirectRecoverableConnection, this is done when com.arjuna.ats.internal.jdbc.DirectRecoverableConnection.reset() is invoked, that is when the RecoverableConnection is closed, and so when the transaction terminates.
> The ultimate effect is that the ConnectionImple is never removed from the caching map of ConnectionManager, because it should have been when the connection was closed, BUT at that time it was not because its RecoverableConnection was in use.
> What we're actually seeing in our application is that com.arjuna.ats.internal.jdbc.ConnectionManager._connections grows indefinitely. This causes a memory leak and severe performance problems, also because the test for cache hits in ConnectionManager is made using a linear iteration over the map (this could be improved a lot).
> The workaround is to disable connection pooling within the transaction by using the patch from JBTM-529 (but also please see my last comment, too!).
> To fix the problem I think that some approaches could be followed:
> - the easiest one is to remove the old connections without any associated transaction when iterating over _connections: that is, if I'm checking a connection conn which has tx1 == null, I could remove conn from _connections; in this way I'm sure that sooner or later a ConnectionImple, which hasn't been removed from that map previously, will be when another connection is requested
> - implement com.arjuna.ats.internal.jdbc.ConnectionImple.equals(Object) and hashCode() so that the call to _connections.put(conn, conn) will "replace any old (closed) connection which had the same connection information", as the inline documentation says; this is not true right now, because equality is based on identity; this might be the less invasive solution, but I don't know how easy it would be to do and if there may be regressions in other areas
> - provide a mechanism to invoke com.arjuna.ats.internal.jdbc.ConnectionManager.remove(ConnectionImple) for all the ConnectionImple objects bound to a transaction when this is completed (committed or rolled back)
> In any case, a smarter use of _connections would be useful for performance (using a Transaction=>ConnectionImple mapping, for instance...).
> I would also consider the complete removal of in-transaction connection pooling: after all, every production use of JBossTS JTA will certainly use some sort of connection pooling by its own, so the benefits of using another level of pooling could be negligible. Even worse, performance could be compromised if the search for the hit in the pool is not efficient.
> PLEASE NOTE: this problem has been observed in our production environment, using JBossTS JTA 4.6.1_GA. I did some browsing of JBossTS JTA 4.11.0_Final and although some little changes (like the use of a HashSet instead of a Hashtable) it seems to me that the problem is still there. I couldn't do further investigation for now, but I will do soon.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 3 months
[JBoss JIRA] (JBTM-531) com.arjuna.ats.internal.jdbc.DynamicClass.shutdownDataSource(XADataSource) is never called
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-531?page=com.atlassian.jira.plugin.s... ]
Tom Jenkinson updated JBTM-531:
-------------------------------
Fix Version/s: 5.0.0.Final
(was: 5.0.0.M3)
> com.arjuna.ats.internal.jdbc.DynamicClass.shutdownDataSource(XADataSource) is never called
> ------------------------------------------------------------------------------------------
>
> Key: JBTM-531
> URL: https://issues.jboss.org/browse/JBTM-531
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Resource Manager
> Affects Versions: 4.5.0, 4.6.0
> Reporter: Mauro Molinari
> Assignee: Tom Jenkinson
> Priority: Critical
> Fix For: 5.0.0.Final
>
>
> com.arjuna.ats.internal.jdbc.DynamicClass.shutdownDataSource(XADataSource) is never called by com.arjuna.ats.internal.jdbc.DirectRecoverableConnection.
> From what I could understand (but I'm not sure of this), after calling com.arjuna.ats.internal.jdbc.DirectRecoverableConnection.closeCloseCurrentConnection() the DirectRecoverableConnection won't be used anymore, so that method could be a good candidate to call _dynamicConnection.shutDownDataSource(_theDataSource) and to null both _dynamicConnection and _theDataSource.
> This problem could be quite severe, because if the XADataSource needs to be shut down in order to release resources, this could lead to leaks.
> Moreover, I was wondering if a caching of the dynamic class wouldn't be a good idea... as of now, a NEW XA data source is created every time a new connection is requested... and this could be expensive...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 3 months
[JBoss JIRA] (JBTM-789) XA connections leak when no modifier available
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-789?page=com.atlassian.jira.plugin.s... ]
Tom Jenkinson updated JBTM-789:
-------------------------------
Fix Version/s: 5.0.0.Final
(was: 5.0.0.M3)
> XA connections leak when no modifier available
> ----------------------------------------------
>
> Key: JBTM-789
> URL: https://issues.jboss.org/browse/JBTM-789
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Resource Manager
> Affects Versions: 4.11.0
> Environment: JBoss Transactions 4.11, Tomcat, Spring
> Reporter: Mauro Molinari
> Assignee: Tom Jenkinson
> Priority: Critical
> Fix For: 5.0.0.Final
>
> Attachments: Patch JBTM-789.txt
>
>
> I recently upgraded JBossTS from 4.6.1 GA to 4.11 Final. I see that bug JBTM-532 that I opened against 4.5 and 4.6 should have been fixed in 4.8.0. However, besides still having a null _theModifier when using PostgreSQL, the current (4.11) implementation of com.arjuna.ats.internal.jdbc.ConnectionImple.close() has a problem.
>
> I mean, if _theModifier is null, a comment says: "no indication about connections, so assume close immediately". However in this case only _theConnection (java.sql.Connection) is closed and set to null, while _recoveryConnection.closeCloseCurrentConnection() is not called, leading to an XAConnection leak.
> What I would expect is that if _theModifier is null, a log entry were added but nothing else were done; in particular I would expect not to return and let the following "if (!delayClose)" (towards the end of the method) close both the connections immediately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 3 months
[JBoss JIRA] (JBTM-1211) Fix qa suite failure: org.jboss.jbossts.qa.junit.testgroup.TestGroup_jtsremote
by Gytis Trikleris (JIRA)
[ https://issues.jboss.org/browse/JBTM-1211?page=com.atlassian.jira.plugin.... ]
Gytis Trikleris commented on JBTM-1211:
---------------------------------------
http://172.17.131.2/job/jbossts-narayana-java6-ipv6-dualstack/68
> Fix qa suite failure: org.jboss.jbossts.qa.junit.testgroup.TestGroup_jtsremote
> ------------------------------------------------------------------------------
>
> Key: JBTM-1211
> URL: https://issues.jboss.org/browse/JBTM-1211
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: JTS, Testing, Transaction Core
> Affects Versions: 4.16.4
> Reporter: Paul Robinson
> Assignee: Michael Musgrove
> Fix For: 4.17.4, 5.0.0.M2
>
> Attachments: jbossts-narayana-java6-ipv6-dualstack.57.tar, jtsremote(1).zip, jtsremote.zip
>
> Original Estimate: 1 week
> Remaining Estimate: 1 week
>
> See: http://172.17.131.2/view/Narayana+BlackTie/job/jbossts-branch416-java6/232
> {code}
> Running org.jboss.jbossts.qa.junit.testgroup.TestGroup_jtsremote
> [junit] Tests run: 9, Failures: 1, Errors: 0, Time elapsed: 169.235 sec
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 3 months
[JBoss JIRA] (JBTM-1414) Facilitate TM record removal by adding record's file name to the warning message
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-1414?page=com.atlassian.jira.plugin.... ]
Michael Musgrove commented on JBTM-1414:
----------------------------------------
Also printing the file name is only useful with file based stores so this won't help with all the other store types, particularly the journalling store. Chapter 15 of
JBoss_Enterprise_Application_Platform-6-Administration_and_Configuration_Guide-en-US.pdf contains a nice description of how to delete transaction records via the tooling.
> Facilitate TM record removal by adding record's file name to the warning message
> ---------------------------------------------------------------------------------
>
> Key: JBTM-1414
> URL: https://issues.jboss.org/browse/JBTM-1414
> Project: JBoss Transaction Manager
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: Recovery
> Affects Versions: later
> Environment: JBoss EAP 6.x
> Reporter: Tom Ross
> Assignee: Tom Jenkinson
>
> When printing warnings about none recoverable transaction manager records in the object store it would help if the warning message included the record's physical file name as well. It would made the manual removal of those records much easier.
> {noformat}
> 15:19:51,003 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a2c0a0d:-140df526:50d1ddd3:ac883, node_name=1, branch_uid=0:ffff0a2c0a0d:-140df526:50d1ddd3:ac95c, subordinatenodename=null, eis_name=java:jboss/eis/Connection >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.7.0, jndiName: java:jboss/eis/Connection com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5e124b >
> 15:19:51,008 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a2c0a0d:-140df526:50d1ddd3:ac883, node_name=1, branch_uid=0:ffff0a2c0a0d:-140df526:50d1ddd3:ac95c, subordinatenodename=null, eis_name=java:jboss/eis/Connection >
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 3 months