9:49 a.m.
Wouter De Borger created AS7-4958:
-------------------------------------
Summary: Memory leak in Distributed Transaction System
Key: AS7-4958
URL: https://issues.jboss.org/browse/AS7-4958
Project: Application Server 7
Issue Type: Bug
Components: EJB
Affects Versions: 7.1.2.Final (EAP), 7.1.1.Final
Environment: centos6 in libvirt qemu/kvm vm
Reporter: Wouter De Borger
Assignee: jaikiran pai
When running distributed transaction over multiple clusters, memory is depleted in a
matter of minutes.
We are using EJB's without transaction annotations (default behavior) and remoting as
described in
https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+....
Heap dumps are available at http://people.cs.kuleuven.be/~bart.vanbrabant/jboss_dump/.
The IBM heap analyzer blames org/jboss/as/ejb3/remote/EJBRemoteTransactionsRepository as
the root cause, taking up 95.73% of memory, through the transaction map.
If more info is required, I can provided our test application.
Thanks,
Wouter
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:53 a.m.
[
https://issues.jboss.org/browse/AS7-4958?page=com.atlassian.jira.plugin.s...
]
jaikiran pai commented on AS7-4958:
-----------------------------------
{quote}
If more info is required, I can provided our test application.
{quote}
Yes, please.
Memory leak in Distributed Transaction System
---------------------------------------------
Key: AS7-4958
URL: https://issues.jboss.org/browse/AS7-4958
Project: Application Server 7
Issue Type: Bug
Components: EJB
Affects Versions: 7.1.1.Final, 7.1.2.Final (EAP)
Environment: centos6 in libvirt qemu/kvm vm
Reporter: Wouter De Borger
Assignee: jaikiran pai
When running distributed transaction over multiple clusters, memory is depleted in a
matter of minutes.
We are using EJB's without transaction annotations (default behavior) and remoting
as described in
https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+....
Heap dumps are available at http://people.cs.kuleuven.be/~bart.vanbrabant/jboss_dump/.
The IBM heap analyzer blames org/jboss/as/ejb3/remote/EJBRemoteTransactionsRepository as
the root cause, taking up 95.73% of memory, through the transaction map.
If more info is required, I can provided our test application.
Thanks,
Wouter
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:42 a.m.
[
https://issues.jboss.org/browse/AS7-4958?page=com.atlassian.jira.plugin.s...
]
jaikiran pai commented on AS7-4958:
-----------------------------------
Wouter, did you get a chance to create a sample application to reproduce this?
Memory leak in Distributed Transaction System
---------------------------------------------
Key: AS7-4958
URL: https://issues.jboss.org/browse/AS7-4958
Project: Application Server 7
Issue Type: Bug
Components: EJB
Affects Versions: 7.1.1.Final, 7.1.2.Final (EAP)
Environment: centos6 in libvirt qemu/kvm vm
Reporter: Wouter De Borger
Assignee: jaikiran pai
When running distributed transaction over multiple clusters, memory is depleted in a
matter of minutes.
We are using EJB's without transaction annotations (default behavior) and remoting
as described in
https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+....
Heap dumps are available at http://people.cs.kuleuven.be/~bart.vanbrabant/jboss_dump/.
The IBM heap analyzer blames org/jboss/as/ejb3/remote/EJBRemoteTransactionsRepository as
the root cause, taking up 95.73% of memory, through the transaction map.
If more info is required, I can provided our test application.
Thanks,
Wouter
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:27 a.m.
[
https://issues.jboss.org/browse/AS7-4958?page=com.atlassian.jira.plugin.s...
]
jaikiran pai commented on AS7-4958:
-----------------------------------
Quick question - In your application, is the invocation on the EJB happening from an EJB?
Memory leak in Distributed Transaction System
---------------------------------------------
Key: AS7-4958
URL: https://issues.jboss.org/browse/AS7-4958
Project: Application Server 7
Issue Type: Bug
Components: EJB
Affects Versions: 7.1.1.Final, 7.1.2.Final (EAP)
Environment: centos6 in libvirt qemu/kvm vm
Reporter: Wouter De Borger
Assignee: jaikiran pai
When running distributed transaction over multiple clusters, memory is depleted in a
matter of minutes.
We are using EJB's without transaction annotations (default behavior) and remoting
as described in
https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+....
Heap dumps are available at http://people.cs.kuleuven.be/~bart.vanbrabant/jboss_dump/.
The IBM heap analyzer blames org/jboss/as/ejb3/remote/EJBRemoteTransactionsRepository as
the root cause, taking up 95.73% of memory, through the transaction map.
If more info is required, I can provided our test application.
Thanks,
Wouter
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:05 a.m.
[
https://issues.jboss.org/browse/AS7-4958?page=com.atlassian.jira.plugin.s...
]
Wouter De Borger commented on AS7-4958:
---------------------------------------
Yes
Memory leak in Distributed Transaction System
---------------------------------------------
Key: AS7-4958
URL: https://issues.jboss.org/browse/AS7-4958
Project: Application Server 7
Issue Type: Bug
Components: EJB
Affects Versions: 7.1.1.Final, 7.1.2.Final (EAP)
Environment: centos6 in libvirt qemu/kvm vm
Reporter: Wouter De Borger
Assignee: jaikiran pai
When running distributed transaction over multiple clusters, memory is depleted in a
matter of minutes.
We are using EJB's without transaction annotations (default behavior) and remoting
as described in
https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+....
Heap dumps are available at http://people.cs.kuleuven.be/~bart.vanbrabant/jboss_dump/.
The IBM heap analyzer blames org/jboss/as/ejb3/remote/EJBRemoteTransactionsRepository as
the root cause, taking up 95.73% of memory, through the transaction map.
If more info is required, I can provided our test application.
Thanks,
Wouter
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:54 a.m.
[
https://issues.jboss.org/browse/AS7-4958?page=com.atlassian.jira.plugin.s...
]
Wouter De Borger commented on AS7-4958:
---------------------------------------
Hi,
The fix seems to work.
Under moderate load, the servers seem to hold out well, garbage collection time are
stable.
Under heavy load, the servers get stuck, but this is probably due to unrelated causes. We
will investigated further this week.
However, periodically, the tier that initializes the transactions prints out a massive
number of log messages of the form:
17:49:06,308 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find
new XAResource to use for recovering non-serializable XAResource XAResourceRecord <
resource:null, txid:< formatId=131077, gtrid_length=29, bqual_length=36,
tx_uid=0:ffffc0a87a0e:-22a62bdc:4fe021f9:936f, node_name=1,
branch_uid=0:ffffc0a87a0e:-22a62bdc:4fe021f9:9370, subordinatenodename=null,
eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@7b9428f7 >
17:49:06,309 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to
recover < formatId=131077, gtrid_length=29, bqual_length=36,
tx_uid=0:ffffc0a87a0e:-22a62bdc:4fe021f9:936f, node_name=1,
branch_uid=0:ffffc0a87a0e:-22a62bdc:4fe021f9:9370, subordinatenodename=null,
eis_name=unknown eis name >
Wouter
Memory leak in Distributed Transaction System
---------------------------------------------
Key: AS7-4958
URL: https://issues.jboss.org/browse/AS7-4958
Project: Application Server 7
Issue Type: Bug
Components: EJB
Affects Versions: 7.1.1.Final, 7.1.2.Final (EAP)
Environment: centos6 in libvirt qemu/kvm vm
Reporter: Wouter De Borger
Assignee: jaikiran pai
When running distributed transaction over multiple clusters, memory is depleted in a
matter of minutes.
We are using EJB's without transaction annotations (default behavior) and remoting
as described in
https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+....
Heap dumps are available at http://people.cs.kuleuven.be/~bart.vanbrabant/jboss_dump/.
The IBM heap analyzer blames org/jboss/as/ejb3/remote/EJBRemoteTransactionsRepository as
the root cause, taking up 95.73% of memory, through the transaction map.
If more info is required, I can provided our test application.
Thanks,
Wouter
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:34 a.m.
[
https://issues.jboss.org/browse/AS7-4958?page=com.atlassian.jira.plugin.s...
]
jaikiran pai commented on AS7-4958:
-----------------------------------
Wouter, thanks for testing the fix. I've sent a pull request to merge these changes
upstream.
As for the WARN messages you mention, under heavy load, if this turns out to be an issue
in the AS7 code, feel free to either open a forum thread or a JIRA with the details.
Memory leak in Distributed Transaction System
---------------------------------------------
Key: AS7-4958
URL: https://issues.jboss.org/browse/AS7-4958
Project: Application Server 7
Issue Type: Bug
Components: EJB
Affects Versions: 7.1.1.Final, 7.1.2.Final (EAP)
Environment: centos6 in libvirt qemu/kvm vm
Reporter: Wouter De Borger
Assignee: jaikiran pai
When running distributed transaction over multiple clusters, memory is depleted in a
matter of minutes.
We are using EJB's without transaction annotations (default behavior) and remoting
as described in
https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+....
Heap dumps are available at http://people.cs.kuleuven.be/~bart.vanbrabant/jboss_dump/.
The IBM heap analyzer blames org/jboss/as/ejb3/remote/EJBRemoteTransactionsRepository as
the root cause, taking up 95.73% of memory, through the transaction map.
If more info is required, I can provided our test application.
Thanks,
Wouter
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
5068
days inactive
5082
days old
18 comments
3 participants
participants (3)
-
Bart Vanbrabant (JIRA) -
jaikiran pai (JIRA)
-
Wouter De Borger (JIRA)