[JBoss JIRA] (JASSIST-163) RuntimeSupport.find2Methods a perf hotspot when proxy's methods are called at higher concurrency
8:27 a.m.
Nikita Tovstoles created JASSIST-163:
----------------------------------------
Summary: RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.16.1-GA, 3.15.0-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profiler|http://yourkit.com] is attached, salient screen shots also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:31 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Attachment: Tomcat-2012-03-28(2).zip
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profiler|http://yourkit.com] is attached, salient screen shots also
attached separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:35 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Attachment: monitor-backtraces.png
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: monitor-backtraces.png, monitor-backtraces.png,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profiler|http://yourkit.com] is attached, salient screen shots also
attached separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:35 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Attachment: monitor-backtraces.png
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: monitor-backtraces.png, monitor-backtraces.png,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profiler|http://yourkit.com] is attached, salient screen shots also
attached separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:37 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Description:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip], salient screen shots also attached separately
was:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profiler|http://yourkit.com] is attached, salient screen shots also attached
separately
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: monitor-backtraces.png, monitor-backtraces.png,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip], salient screen shots also attached separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:41 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Attachment: blocked-threads.png
find2methods-hotspot.png
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
monitor-backtraces.png, monitor-backtraces.png, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip], salient screen shots also attached separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:41 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Description:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
was:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip], salient screen shots also attached separately
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
monitor-backtraces.png, monitor-backtraces.png, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:43 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Description:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in [this screen shot|
^monitor-backtraces.png]
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
was:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement.
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
monitor-backtraces.png, monitor-backtraces.png, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in [this screen shot|
^monitor-backtraces.png]
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:45 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Description:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
was:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in [this screen shot|
https://issues.jboss.org/secure/thumbnail/12353025/_thumb_12353025.png]
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
monitor-backtraces.png, monitor-backtraces.png, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:45 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Description:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in [this screen shot|
https://issues.jboss.org/secure/thumbnail/12353025/_thumb_12353025.png]
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
was:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in [this screen shot|
^monitor-backtraces.png]
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
monitor-backtraces.png, monitor-backtraces.png, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in [this screen shot|
https://issues.jboss.org/secure/thumbnail/12353025/_thumb_12353025.png]
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:53 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
Nikita, please stop spamming.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
monitor-backtraces.png, monitor-backtraces.png, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:13 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Description:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead of repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
was:
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to 55,
260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values should
be cached (in a ConcurrentMap?) instead repeatedly executing the above synchronized
statement. Instead, currently, it's being called every time (?) a proxied method is
executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
I am happy to fix the above and submit a patch, but need a little help making sense of
ProxyFactory.makeForwarder as I am not well-versed in bytecode directives. Ideally, could
someone provide a plain java or pseudo-code translation of bytecode being generated in
makeForwarder - i can't quite understand what that method is doing? Specifically, what
is the value of Method[] methods when find2Methods is invoked (indirectly) from
makeForwarder...
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
monitor-backtraces.png, monitor-backtraces.png, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:07 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Attachment: Product_$$_javassist_0.java
I decompiled output of ProxyFactory.createClass(). Here's what makeForwarder does for
each proxied method (full decompiled src attached):
{code}
//target pojo has a getName() method
public final String getName()
{
Method[] arrayOfMethod = _methods_;
RuntimeSupport.find2Methods(this, "getName", "_d7getName", 14,
"()Ljava/lang/String;", arrayOfMethod);
return (String)this.handler.invoke(this, arrayOfMethod[14], arrayOfMethod[15], new
Object[0]);
}
{code}
..looks like static Method[] methods is lazily filled, perhaps as a perf/memory
optimization. Clearly repeat calls to find2Methods need to be avoided.
Not yet clear if, instead of caching, it'd make more sense to fill _methods_ eagerly
during during proxy class creation and avoid any calls to find2methods during calls
against proxy instance
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
monitor-backtraces.png, monitor-backtraces.png, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:30 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Attachment: jassist-163-fix.patch
Product_$$_javassist_0-post-patch.java
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:36 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles updated JASSIST-163:
-------------------------------------
Attachment: Product.java
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:38 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles commented on JASSIST-163:
------------------------------------------
I fixed this problem by moving calls to RuntimeSupport.find2methods from forwarder
methods' bodies to a static initializer of Method[] _methods_ array. Please compare
generated sources to see specific changes:
# [Product.java|https://issues.jboss.org/secure/attachment/12353075/Product....] - Sample
POJO to be proxied
# [Generated proxy - before
patch|https://issues.jboss.org/secure/attachment/12353038/Product_%24%24_...]
# [Generated proxy - after
patch|https://issues.jboss.org/secure/attachment/12353074/Product_%24%24_...]
# See Find2MethodsRelocatedTest.java (part of the patch) for code used to generate both
proxies
Confirmed that the perf. hotspot has been removed. Here's the patch itself (developed
from trunk revision 626):
https://issues.jboss.org/secure/attachment/12353073/jassist-163-fix.patch
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:45 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
Nikita,
Your patch looks good but filling Method[] eagerly may cause unnecessary memory
consumption and initialization overheads. I think the real problem is a jumbo lock. So
what about modifying find2Methods() as following:
{code}
public static void find2Methods(Object self, String superMethod,
String thisMethod, int index,
String desc, java.lang.reflect.Method[] methods)
{
// synchronized (methods) {
// if (methods[index] == null) {
// methods[index + 1] = thisMethod == null ? null
// : findMethod(self, thisMethod, desc);
// methods[index] = findSuperMethod(self, superMethod, desc);
// }
// }
if (methods[index] == null) {
Method m1 = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
Method m0 = findSuperMethod(self, superMethod, desc);
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = m1;
methods[index] = m0;
}
}
}
}
{code}
Could you check this fix also removes performance hotspot?
Thanks!
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:54 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow updated JASSIST-163:
---------------------------------
Forum Reference: http://lists.jboss.org/pipermail/jboss-as7-dev/2012-July/006274.html
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:07 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
The above change doesn't look safe, since other threads would be reading
methods[index] without synchronizing (methods[index] might contain non-null but
methods[index+1] could be null).
Would it be easier to use a different datastructure instead for holding the pair of
methods? Does the Methods[] methods array need to be longer than two elements or is it
always sized for two?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:42 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
I agree that it's not easy to make sure of the correctness of the code above. In this
sense, the code is not very good.
However, since methods[even-number] is always set after methods[odd-number], if
methods[index] is non null, methods[index + 1] is also non-null. (I suppose that the
JVM's memory model guarantees this fact.) Furthermore, the values of m0 and m1 are
always the same no matter which thread computes them. Am I correct?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:56 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
I think that when the synchronized block is ending, the methods[index] + methods[index+1]
changes will be flushed to Java memory. However, when the non-synchronized read of
methods[index] occurs in the other thread, I think the other thread may or may not also
see the expected methods[index+1] value.
If we could depend on the Java 5 memory model changes, we could look at a change to using
volatile fields of a class (instead of method array), which I think would give us
consistent ordering.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:09 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
Thank you for your comment! I carefully read the document of the memory model and changed
the source:
/* Once methods[index] and methods[index + 1] are set to non-null,
* then their values never change.
*/
if (methods[index] == null || methods[index + 1] == null) {
Method m1 = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
Method m0 = findSuperMethod(self, superMethod, desc);
Since the elements in methods are sort of final fields, the code above should work
correctly.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:53 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
How are the elements in methods sort of final fields?
I think that final fields have to be initialized during the referencing class constructor
or class static initializer.
Your change looks good to me. I'll paste the additional lines (from your change)
below for others to also review.
{code}
/**
* Finds two methods specified by the parameters and stores them
* into the given array.
*
* @throws RuntimeException if the methods are not found.
* @see javassist.util.proxy.ProxyFactory
*/
public static void find2Methods(Object self, String superMethod,
String thisMethod, int index,
String desc, java.lang.reflect.Method[] methods)
{
/* Once methods[index] and methods[index + 1] are set to non-null,
* then their values never change.
*/
if (methods[index] == null || methods[index + 1] == null) {
Method m1 = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
Method m0 = findSuperMethod(self, superMethod, desc);
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = m1;
methods[index] = m0;
}
}
}
}
{code}
If a another thread attempts to read methods[index] or methods[index+1], they will either
see one of the fields set to null or both fields initialized. If either field is null,
the thread will synchronize on methods and then only set both fields if they are still
null.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:26 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
Thanks for your comment. As for "sort of final fields", I mean, the elements of
methods never change once they are set.
My interpretation of the new code is the same as what you wrote above. But I am not sure
that the Method object contained in methods[index] is up-to-date when methods[index] is
non null. Is this guaranteed by the memory model? Since a reader thread may not
synchronize before reading methods[index], it may see that methods[index] is non null but
the Method object in methods[index] contains older values than the ones visible to the
writer thread, which sets methods[index].
Any advice?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:58 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles commented on JASSIST-163:
------------------------------------------
Count me biased, but I maintain the approach of eagerly initializing static methods[] -
submitted
[above|https://issues.jboss.org/browse/JASSIST-163?focusedCommentId=126810...]
is most practical:
* simpler runtime code in generated getters (see before-and-after comparison in the above
comment)
* shorter call-chain during getter invocation
My understanding is that the argument for filling methods[] lazily is memory savings. In
reality the current approach of lazy methods[] initialization will produce minimal-to-zero
memory savings as typically all properties of a proxied class are proxied, and, over the
lifecycle of JVM, will be invoked. In other words, after some time methods[] will be
fully-initialized in both implementations.
BTW, Method references in methods[] take up [either 4 or 8 bytes
each|http://stackoverflow.com/a/3733242/182808] or 8 or 16 bytes per property (32bit VM
vs. 64bit vm; 2 Methods per property). That's 8-16 bytes in allocation *delayed*, not
really *avoided*.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:08 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
Hi Nikita,
as typically all properties of a proxied class are proxied, and, over
the lifecycle of JVM, will be invoked.
Is this true in general? Not only in your particular use-cases?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:57 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
{quote}
My interpretation of the new code is the same as what you wrote above. But I am not sure
that the Method object contained in methods[index] is up-to-date when methods[index] is
non null. Is this guaranteed by the memory model? Since a reader thread may not
synchronize before reading methods[index], it may see that methods[index] is non null but
the Method object in methods[index] contains older values than the ones visible to the
writer thread, which sets methods[index].
{quote}
The Method object probably came from Class.getDeclaredMethods() or maybe
Class.getDeclaredMethod(). Since those calls are thread safe, the entire Method object
contents should be in main working memory.
{quote}
Count me biased, but I maintain the approach of eagerly initializing static methods[] -
submitted above is most practical:
{quote}
Would it be possible to support both lazy and eager initialization?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:22 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
bq. The Method object probably came from Class.getDeclaredMethods() or maybe
Class.getDeclaredMethod(). Since those calls are thread safe, the entire Method object
contents should be in main working memory.
Yes, but being thread safe means that a thread doing synchronization (or accessing a
volatile field) can safely use the method according to the memory model of the JVM. The
memory model does not guarantee anything about a thread doing no synchronization. Is this
correct? We can expect that some thread safety is preserved on real hardware for such a
thread that does no synchronization, though.
bq. Would it be possible to support both lazy and eager initialization?
This is a good idea.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:14 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
Good to hear that "eager initialization" is a good idea. :)
Is anyone working on adding eager initialization?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:16 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Brett Meyer commented on JASSIST-163:
-------------------------------------
There were a couple of commits, prior to most of the discussion here. What was the
resolution? Can this be re-opened?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:50 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles commented on JASSIST-163:
------------------------------------------
Hi, Brett & Scott:
I contributed [the fix
above|https://issues.jboss.org/browse/JASSIST-163?focusedCommentId=126810...]
(ie "eager initialization") awhile ago. This patch throws out lazy init
altogether as I wanted to avoid synchronization and did not think lazy initialization
would buy any memory savings at steady state as:
* most - if not all - ORM apps, in my view, end up using all mapped entities over the
course of the app (thus eventually same amount of heap is allocated in both eager and lazy
scenarios)
* even if some initialized properties are never used, the extra memory consumption is too
insignificant - a string per domain class property - to warrant added code complexity (and
potential lock contention) of
FWIW our web API, integrated into [sites with significant
traffic|http://ecorebates.com/our-clients/], has been running on this patch for 6+ months
without any issues. YMMV ;-
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:48 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Brett Meyer commented on JASSIST-163:
-------------------------------------
[~dukehoops], thanks for the clarification!
Any idea when 3.17.0-GA will be available? Hibernate needs this fix fairly soon, if
possible.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:16 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Nikita Tovstoles commented on JASSIST-163:
------------------------------------------
Sure, Brett. Sorry are you asking me about the GA date? I don't know - I don't
actually work on Javassist project - just a 'concerned user' :-)
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:25 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
I planed the GA release early this week but a problem was found in JASSIST-160.
The released date will be late this week or early next week.
However, the current snapshot does not contain the code for "eager"
initialization. Can I extend the release date to merge the code?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:26 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
Yes, I think its best to extend the release date to take additional time for also doing
the "eager initialization". That way, users can choose which fix they want.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:44 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
I have implemented the eager initialization and deleted all the code for the lazy one.
The implementation is slightly different from Nikita's so that the proxies will work
correctly with multiple class loaders.
I will release soon. Quick verification is welcome.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:50 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
I thought that you were going to leave lazy initialization in and also add eager
initialization. So, that existing code could continue to run as it used to (with lazy
initialization) and with code changes, applications could switch to eager initialization.
Is that possible?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:56 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
I was able to build Hibernate with the Javassist revision 676 changes. I couldn't
build with maven but ant worked. I then hacked the javassist.jar into my local maven repo
(hacked to be version=3.17.xxx).
{quote}
r676 | chiba | 2012-10-25 12:38:40 -0400 (Thu, 25 Oct 2012) | 1 line
fixed JASSIST-163. Eager initialization has been implemented.
{quote}
When I run the Hibernate (4.1 branch), I get a
org.hibernate.test.bytecode.javassist.ReflectionOptimizerTest failure for reasons that I
don't yet understand.
I was surprised that Hibernate was able to build as I thought you made Javassist api
changes.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:50 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
When I single stepped through the failing test case from the Intellij IDE, I get a:
{quote}
java.lang.VerifyError: Expecting a stackmap frame at branch target 111 in method
org.hibernate.test.bytecode.Bean_$$_bulkaccess_0.setPropertyValues(Ljava/lang/Object;[Ljava/lang/Object;)V
at ...{quote}
call stack http://pastie.org/5116601
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:06 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
using -XX:-UseSplitVerifier seems to workaround the above failure when running from
Intellij, I'll see if that helps with commandline Hibernate testing. Not sure why
that is needed with the Javassist latest changes.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:16 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
Can you post the dump got by
javap -c -verbose org.hibernate.test.... _bulkaccess_0
?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:27 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
Sure, I will get the dump and post it here.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: blocked-threads.png, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:08 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow updated JASSIST-163:
---------------------------------
Attachment: Bean_$$_bulkaccess_0.txt
Bean_$$_bulkaccess_0.txt contains the output from javap -c -verbose
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, blocked-threads.png,
find2methods-hotspot.png, jassist-163-fix.patch, monitor-backtraces.png,
monitor-backtraces.png, Product.java, Product_$$_javassist_0-post-patch.java,
Product_$$_javassist_0.java, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:04 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
Thanks.
The bug is that an appropriate stack map is not attached to setPropertyValue method but
this class does not look like a class generated by Javassist for a proxy. So it's not
relevant to this latest change. Maybe a bug of Hibernate?
BTW, the eager initialization does not change the API. I just changed the implementation
under the hood.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, blocked-threads.png,
find2methods-hotspot.png, jassist-163-fix.patch, monitor-backtraces.png,
monitor-backtraces.png, Product.java, Product_$$_javassist_0-post-patch.java,
Product_$$_javassist_0.java, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:20 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
To create a stackmap, call MethodInfo#rebuildStackMap(ClassPool) after constructing a
method body.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, blocked-threads.png,
find2methods-hotspot.png, jassist-163-fix.patch, monitor-backtraces.png,
monitor-backtraces.png, Product.java, Product_$$_javassist_0-post-patch.java,
Product_$$_javassist_0.java, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:38 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
In case it helps you to see it, the test case is here
https://github.com/hibernate/hibernate-orm/blob/master/hibernate-core/src...
The Bean_$$_bulkaccess_0.txt output is for this class
https://github.com/hibernate/hibernate-orm/blob/master/hibernate-core/src...
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, blocked-threads.png,
find2methods-hotspot.png, jassist-163-fix.patch, monitor-backtraces.png,
monitor-backtraces.png, Product.java, Product_$$_javassist_0-post-patch.java,
Product_$$_javassist_0.java, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:20 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba updated JASSIST-163:
----------------------------------
Attachment: BulkAccessorFactory.java.diff
This is a bug of org.hibernate.bytecode.internal.javassist.BulkAccessorFactory.java
(Javassist is no guilty!)
I wrote a fix but I haven't tested it yet. Can you check
BulkAccessorFactory.java.diff?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, blocked-threads.png,
BulkAccessorFactory.java.diff, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:15 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow updated JASSIST-163:
---------------------------------
Attachment: BulkAccessorFactory.patch
Still failed. Attached is BulkAccessorFactory.patch which is the output from doing a
"git diff --patch" after applying BulkAccessorFactory.java.diff. I'll check
to see if its the same verify error or something else.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, blocked-threads.png,
BulkAccessorFactory.java.diff, BulkAccessorFactory.patch, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:31 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
new failure is java.lang.VerifyError: Bad local variable type in method
org.hibernate.test.bytecode.Bean_$$_bulkaccess_0.setPropertyValues(Ljava/lang/Object;[Ljava/lang/Object;)V
at offset 119 http://pastie.org/5119100
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, blocked-threads.png,
BulkAccessorFactory.java.diff, BulkAccessorFactory.patch, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:33 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
I will generate the javap -c -verbose again also
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, blocked-threads.png,
BulkAccessorFactory.java.diff, BulkAccessorFactory.patch, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:33 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow edited comment on JASSIST-163 at 10/26/12 10:33 AM:
-----------------------------------------------------------------
I will generate the "javap -c -verbose" output and attach it.
was (Author: smarlow):
I will generate the javap -c -verbose again also
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, blocked-threads.png,
BulkAccessorFactory.java.diff, BulkAccessorFactory.patch, find2methods-hotspot.png,
jassist-163-fix.patch, monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:40 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow updated JASSIST-163:
---------------------------------
Attachment: Bean_$$_bulkaccess_0_Two.txt
Bean_$$_bulkaccess_0_Two.txt is from today's test with the "bad local
variable" error
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.patch,
find2methods-hotspot.png, jassist-163-fix.patch, monitor-backtraces.png,
monitor-backtraces.png, Product.java, Product_$$_javassist_0-post-patch.java,
Product_$$_javassist_0.java, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:42 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
FYI, JBossWS Native testing with JASSIST trunk (revision 676) is successful.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.patch,
find2methods-hotspot.png, jassist-163-fix.patch, monitor-backtraces.png,
monitor-backtraces.png, Product.java, Product_$$_javassist_0-post-patch.java,
Product_$$_javassist_0.java, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:46 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
thanks lots. I see why the test failed. I will fix this bug in a few days.
(I'm quite busy this week with my work.)
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.patch,
find2methods-hotspot.png, jassist-163-fix.patch, monitor-backtraces.png,
monitor-backtraces.png, Product.java, Product_$$_javassist_0-post-patch.java,
Product_$$_javassist_0.java, Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:39 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba updated JASSIST-163:
----------------------------------
Attachment: BulkAccessorFactory.java.diff2
I have updated the diff file. Could you try BulkAccessorFactory.java.diff2?
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:32 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
Thanks, I tested again with the BulkAccessorFactory.java.diff2 patch and the Hibernate
test that failed before is now passing! I pushed the patch to
https://github.com/scottmarlow/hibernate-orm/tree/4.1_HHH-7435_javassistu....
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:07 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
Good. Thanks.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:58 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
I have released 3.17.0-GA, including this fix.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:23 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Brett Meyer commented on JASSIST-163:
-------------------------------------
Thanks guys -- we'll update the version in Hibernate asap
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:11 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Scott Marlow commented on JASSIST-163:
--------------------------------------
Would it be possible to also have a release that targets Java 6? The 3.17.0-GA release
appears to target Java 7.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:32 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
The binary available from sourceforge is for JDK 1.4 or later.
Please try it. It is available from www.javassist.org
Otherwise, you can compile by yourself. Just run "ant jar" or "mvn
package"
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:49 a.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Donnchadh Ó Donnabháin commented on JASSIST-163:
------------------------------------------------
[~chiba], I think [~smarlow] was referring to the build in maven central.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:51 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
I see. So, which kind of binary do you think we should upload to maven, Java 6 or 7?
I'm not an expert of maven...
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:51 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Donnchadh Ó Donnabháin commented on JASSIST-163:
------------------------------------------------
Is there a reason that the source and target were changed to 1.7 in the following
changeset:
https://source.jboss.org/changelog/Javassist?cs=678
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:53 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Donnchadh Ó Donnabháin edited comment on JASSIST-163 at 11/14/12 1:51 PM:
--------------------------------------------------------------------------
Is there a reason that the source and target were changed to 1.7 in the following
changeset?:
https://source.jboss.org/changelog/Javassist?cs=678
was (Author: donnchadh):
Is there a reason that the source and target were changed to 1.7 in the following
changeset:
https://source.jboss.org/changelog/Javassist?cs=678
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:03 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Shigeru Chiba commented on JASSIST-163:
---------------------------------------
Ah,... maybe that's to compile the source on my Mac.
I will fix it. In fact, Javassist works on JDK 1.4 or later.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:24 p.m.
[
https://issues.jboss.org/browse/JASSIST-163?page=com.atlassian.jira.plugi...
]
Dan Fabulich commented on JASSIST-163:
--------------------------------------
I've filed JASSIST-178 on the Java 7 bug.
RuntimeSupport.find2Methods a perf hotspot when proxy's methods
are called at higher concurrency
------------------------------------------------------------------------------------------------
Key: JASSIST-163
URL: https://issues.jboss.org/browse/JASSIST-163
Project: Javassist
Issue Type: Enhancement
Affects Versions: 3.15.0-GA, 3.16.1-GA
Environment: hibernate-core 3.6.10.Final
Reporter: Nikita Tovstoles
Assignee: Shigeru Chiba
Fix For: 3.17.0-GA
Attachments: Bean_$$_bulkaccess_0.txt, Bean_$$_bulkaccess_0_Two.txt,
blocked-threads.png, BulkAccessorFactory.java.diff, BulkAccessorFactory.java.diff2,
BulkAccessorFactory.patch, find2methods-hotspot.png, jassist-163-fix.patch,
monitor-backtraces.png, monitor-backtraces.png, Product.java,
Product_$$_javassist_0-post-patch.java, Product_$$_javassist_0.java,
Tomcat-2012-03-28(2).zip
We've been profiling our Hibernate 3.6.10-based app and noticed a perf bottleneck in
javassist.util.proxy.RuntimeSupport.find2methods. Unfortunately, this method, which has a
synch. block, is being called on
every invocation of every proxied entity method (see
javassist.util.proxy.ProxyFactory.makeForwarder(), called indirectly by
ProxyFactory.createClass()).
In our testing, the result is that our service call's latency increases from 33 to
55, 260, 400ms as concurrency increases
1-10-20-30 users on a 4-core CPU. At 20 and 30 users 51% of CPU time is spent contending
for a monitor in RuntimeSupport.find2methods:
{code}
synchronized (methods) {
if (methods[index] == null) {
methods[index + 1] = thisMethod == null ? null
: findMethod(self, thisMethod, desc);
methods[index] = findSuperMethod(self, superMethod, desc);
}
}
{code}
Since find2methods merely interrogates class metadata, seems like its return values
should be cached (in a ConcurrentMap?) instead of repeatedly executing the above
synchronized statement. Instead, currently, it's being called every time (?) a proxied
method is executed - see *Invocation Count* in this screen shot:
https://issues.jboss.org/secure/attachment/12353025/monitor-backtraces.png
Full [YourKit profile|http://yourkit.com] is [attached as a ZIP
archive|^Tomcat-2012-03-28(2).zip]; key screen shots from the snapshot also attached
separately
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
4439
days inactive
4668
days old
67 comments
6 participants
participants (6)
-
Brett Meyer (JIRA) -
Dan Fabulich (JIRA)
-
Donnchadh Ó Donnabháin (JIRA)
-
Nikita Tovstoles (JIRA)
-
Scott Marlow (JIRA)
-
Shigeru Chiba (JIRA)