[JBoss JIRA] (SECURITY-734) Slow policy evaluation with a large number of policy sets
by Anil Saldhana (JIRA)
[ https://issues.jboss.org/browse/SECURITY-734?page=com.atlassian.jira.plug... ]
Anil Saldhana resolved SECURITY-734.
------------------------------------
Resolution: Done
> Slow policy evaluation with a large number of policy sets
> ---------------------------------------------------------
>
> Key: SECURITY-734
> URL: https://issues.jboss.org/browse/SECURITY-734
> Project: PicketBox
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Affects Versions: picketbox_xacml_2.0.8.Final
> Environment: RedHat Linux 6.0, WebSphere Application Server 7.0
> Reporter: Maxim Zinal
> Assignee: Anil Saldhana
> Fix For: picketbox_xacml_2.0.9.Final
>
> Attachments: fix-734.patch
>
>
> We suffer a performance problem in the evaluation of large XACML configuration. We use JBoss (Sun?) XACML library to process a large number of policy sets (over 2400 policy set files) using "deny-override" combining algorithm.
> Initially the number of policies was relatively small, but it increased with time, leading to performance degradation, with 100% CPU (single)core consumption.
> After running a code sample under Java profiler, we found that 99% of time is spent in method PolicySetFinderModule.findPolicy(URI, int, VersionConstraints, PolicyMetaData).
> This method walks through a list of AbstractPolicy objects, comparing an ID of each object with it's first argument. In our configuration the number of such objects is relatively large, leading to slow execution with high CPU consumption.
> We suggest replacing the list of AbstractPolicy with a HashMap. Our experiments show that total evaluation time reduces ~50 times with a HashMap over List implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (SECURITY-734) Slow policy evaluation with a large number of policy sets
by Maxim Zinal (JIRA)
[ https://issues.jboss.org/browse/SECURITY-734?page=com.atlassian.jira.plug... ]
Maxim Zinal commented on SECURITY-734:
--------------------------------------
Created a pull request. How strange and ugly that git is, uh... :(
Poor Linux developers.
> Slow policy evaluation with a large number of policy sets
> ---------------------------------------------------------
>
> Key: SECURITY-734
> URL: https://issues.jboss.org/browse/SECURITY-734
> Project: PicketBox
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Environment: RedHat Linux 6.0, WebSphere Application Server 7.0
> Reporter: Maxim Zinal
> Assignee: Anil Saldhana
> Attachments: fix-734.patch
>
>
> We suffer a performance problem in the evaluation of large XACML configuration. We use JBoss (Sun?) XACML library to process a large number of policy sets (over 2400 policy set files) using "deny-override" combining algorithm.
> Initially the number of policies was relatively small, but it increased with time, leading to performance degradation, with 100% CPU (single)core consumption.
> After running a code sample under Java profiler, we found that 99% of time is spent in method PolicySetFinderModule.findPolicy(URI, int, VersionConstraints, PolicyMetaData).
> This method walks through a list of AbstractPolicy objects, comparing an ID of each object with it's first argument. In our configuration the number of such objects is relatively large, leading to slow execution with high CPU consumption.
> We suggest replacing the list of AbstractPolicy with a HashMap. Our experiments show that total evaluation time reduces ~50 times with a HashMap over List implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (SECURITY-738) XACML DatabaseResourceAttributeLocator fails when used with Oracle 11g Driver
by Anil Saldhana (JIRA)
[ https://issues.jboss.org/browse/SECURITY-738?page=com.atlassian.jira.plug... ]
Anil Saldhana commented on SECURITY-738:
----------------------------------------
Please feel free to fork https://github.com/picketbox/security-xacml and submit a pull request.
> XACML DatabaseResourceAttributeLocator fails when used with Oracle 11g Driver
> -----------------------------------------------------------------------------
>
> Key: SECURITY-738
> URL: https://issues.jboss.org/browse/SECURITY-738
> Project: PicketBox
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: JBossXACML
> Affects Versions: picketbox_xacml_2.0.8.Final
> Environment: EAP 6.0.1
> Reporter: Hisanobu Okuda
> Assignee: Anil Saldhana
>
> We are using the DatabaseResourceAttributeLocator in jboss-xacml 2.0.8.Final for fetching a policy attribute from an Oracle 11g database. The Oracle JDBC driver is ojdbc6 11.2.0.x. However, the locator fails with an SQLException. The complete stacktrace is attached, but the interesting part is the following:
> {code}
> Caused by: java.sql.SQLException: error occurred during batching: batch must be either executed or cleared
> at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3435)
> at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3493)
> at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1491)
> at org.jboss.security.xacml.locators.attrib.DatabaseAttributeLocator.getColumnValue(DatabaseAttributeLocator.java:215)
> ... 48 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (SECURITY-734) Slow policy evaluation with a large number of policy sets
by Anil Saldhana (JIRA)
[ https://issues.jboss.org/browse/SECURITY-734?page=com.atlassian.jira.plug... ]
Anil Saldhana commented on SECURITY-734:
----------------------------------------
[~zinal] please feel free to fork https://github.com/picketbox/security-xacml and submit a pull request.
> Slow policy evaluation with a large number of policy sets
> ---------------------------------------------------------
>
> Key: SECURITY-734
> URL: https://issues.jboss.org/browse/SECURITY-734
> Project: PicketBox
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Environment: RedHat Linux 6.0, WebSphere Application Server 7.0
> Reporter: Maxim Zinal
> Assignee: Anil Saldhana
> Attachments: fix-734.patch
>
>
> We suffer a performance problem in the evaluation of large XACML configuration. We use JBoss (Sun?) XACML library to process a large number of policy sets (over 2400 policy set files) using "deny-override" combining algorithm.
> Initially the number of policies was relatively small, but it increased with time, leading to performance degradation, with 100% CPU (single)core consumption.
> After running a code sample under Java profiler, we found that 99% of time is spent in method PolicySetFinderModule.findPolicy(URI, int, VersionConstraints, PolicyMetaData).
> This method walks through a list of AbstractPolicy objects, comparing an ID of each object with it's first argument. In our configuration the number of such objects is relatively large, leading to slow execution with high CPU consumption.
> We suggest replacing the list of AbstractPolicy with a HashMap. Our experiments show that total evaluation time reduces ~50 times with a HashMap over List implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (JGRP-1618) missingMessageReceived() never called in NAKACK resulting in memory leak
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1618?page=com.atlassian.jira.plugin.... ]
Bela Ban resolved JGRP-1618.
----------------------------
Resolution: Won't Fix
See my comment on the case
> missingMessageReceived() never called in NAKACK resulting in memory leak
> ------------------------------------------------------------------------
>
> Key: JGRP-1618
> URL: https://issues.jboss.org/browse/JGRP-1618
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 2.8.1, 2.12.2
> Environment: Java 6
> Reporter: Harry Mark
> Assignee: Bela Ban
> Fix For: 3.2.9, 3.3
>
> Attachments: NakReceiverWindow.java
>
>
> We are using JGroups 2.8.1 and encountered a memory leak where it eventually ran out of CMS Old Gen memory. The heap dump revealed that the problem was in the xmit_stats ConcurrentHashMap of org.jgroups.protocols.pbcast.NAKACK.
> After much analysis here's what we found: when the system is under load, messages can start arriving out of order. When the receiver receives a higher sequence number than expected, it requests the sender retransmit the missing messages with the lower sequence numbers. The sender sends the missing message, however the bug in the NakReceiverWindow meant that the missing message was never purged from the Map that tracks missing messages (xmit_stats) because missingMessageReceived() was never invoked. Over time this Map grows and starts using up CMS Old Gen; the only way it would get reduced was when a server left the cluster and the missing messages were purged for that server.
> In JMX, the MissingMsgsReceived attribute of jgroups:cluster=*,protocol=NAKACK,type=protocol was always zero, confirming that it never purged any received "missing messages".
> When I looked at the most recent GA version 3.2.8 of NakReceiverWindow.java , it has the corrected logic that ensures that missingMessageReceived() is called. I checked the the most recent 2.x, which is 2.12.2, and found it also has the same bug in the logic as 2.8.1. This bug may apply to other 2.x but I did not check.
> Attached is the fixed NakReceiverWindow.java for 2.8.1.
> After applying the patch, the memory leak went away.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (JGRP-1618) missingMessageReceived() never called in NAKACK resulting in memory leak
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1618?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-1618:
--------------------------------
Hi Harry,
excellent analysis, you really did dig through the code didn't you !
However, I'm afraid I don't support 2.8.x or 2.12.x any longer, see [1] for reasons. I suggest you use this patch in a local build of 2.8.x and upgrade to a more recent version at some later point in time.
I also don't see any implementations of missingMessageReceived() in master. Besides, NakReceiverWindow has been replaced by Table and is only used in NAKACK. Note that NAKACK has been superceeded by NAKACK2, see my blog post on the diffs.
Had this been a bug that's also present in master or the 3.2 branch, I'd have fixed it, but xmit_stats was removed some time ago.
[1] https://community.jboss.org/docs/DOC-14454
> missingMessageReceived() never called in NAKACK resulting in memory leak
> ------------------------------------------------------------------------
>
> Key: JGRP-1618
> URL: https://issues.jboss.org/browse/JGRP-1618
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 2.8.1, 2.12.2
> Environment: Java 6
> Reporter: Harry Mark
> Assignee: Bela Ban
> Fix For: 3.2.9, 3.3
>
> Attachments: NakReceiverWindow.java
>
>
> We are using JGroups 2.8.1 and encountered a memory leak where it eventually ran out of CMS Old Gen memory. The heap dump revealed that the problem was in the xmit_stats ConcurrentHashMap of org.jgroups.protocols.pbcast.NAKACK.
> After much analysis here's what we found: when the system is under load, messages can start arriving out of order. When the receiver receives a higher sequence number than expected, it requests the sender retransmit the missing messages with the lower sequence numbers. The sender sends the missing message, however the bug in the NakReceiverWindow meant that the missing message was never purged from the Map that tracks missing messages (xmit_stats) because missingMessageReceived() was never invoked. Over time this Map grows and starts using up CMS Old Gen; the only way it would get reduced was when a server left the cluster and the missing messages were purged for that server.
> In JMX, the MissingMsgsReceived attribute of jgroups:cluster=*,protocol=NAKACK,type=protocol was always zero, confirming that it never purged any received "missing messages".
> When I looked at the most recent GA version 3.2.8 of NakReceiverWindow.java , it has the corrected logic that ensures that missingMessageReceived() is called. I checked the the most recent 2.x, which is 2.12.2, and found it also has the same bug in the logic as 2.8.1. This bug may apply to other 2.x but I did not check.
> Attached is the fixed NakReceiverWindow.java for 2.8.1.
> After applying the patch, the memory leak went away.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (AS7-4932) CLONE - Using session passivation results in WeldListener: java.lang.NullPointerException on normal operation
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/AS7-4932?page=com.atlassian.jira.plugin.s... ]
RH Bugzilla Integration commented on AS7-4932:
----------------------------------------------
Paul Ferraro <paul.ferraro(a)redhat.com> made a comment on [bug 900549|https://bugzilla.redhat.com/show_bug.cgi?id=900549]
Quick status update: I've finally managed to reproduce the issue locally. I'm still isolating the exact cause.
> CLONE - Using session passivation results in WeldListener: java.lang.NullPointerException on normal operation
> -------------------------------------------------------------------------------------------------------------
>
> Key: AS7-4932
> URL: https://issues.jboss.org/browse/AS7-4932
> Project: Application Server 7
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 7.1.2.Final (EAP)
> Reporter: Radoslav Husar
> Assignee: Paul Ferraro
> Priority: Blocker
> Labels: as713tracking
> Fix For: 7.1.3.Final (EAP)
>
>
> NPE is thrown in roughly 0.36% of HTTP session request processing in the test.
> This results in response code 503 returned to the client with the exception.
> The test is using passivation-enabled WAR of clusterbench
> https://github.com/rhusar/clusterbench
> Here is a shorter soak test run that uncovered the issue
> https://hudson.qa.jboss.com/hudson/view/EAP6/view/EAP6-Clustering-Soak/jo...
> {noformat}
> [JBossINF] 19:52:10,113 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host]] (ajp-perf20/10.16.90.58:8009-2570) Exception sending request initialized lifecycle event to listener instance of class org.jboss.weld.servlet.WeldListener: java.lang.NullPointerException
> [JBossINF] at org.jboss.as.web.session.ClusteredSession.update(ClusteredSession.java:972) [jboss-as-web-7.1.2.Final-redhat-1.jar:7.1.2.Final-redhat-1]
> [JBossINF] at org.jboss.as.web.session.DistributableSessionManager.loadSession(DistributableSessionManager.java:1377) [jboss-as-web-7.1.2.Final-redhat-1.jar:7.1.2.Final-redhat-1]
> [JBossINF] at org.jboss.as.web.session.DistributableSessionManager.findSession(DistributableSessionManager.java:673) [jboss-as-web-7.1.2.Final-redhat-1.jar:7.1.2.Final-redhat-1]
> [JBossINF] at org.jboss.as.web.session.DistributableSessionManager.findSession(DistributableSessionManager.java:84) [jboss-as-web-7.1.2.Final-redhat-1.jar:7.1.2.Final-redhat-1]
> [JBossINF] at org.apache.catalina.connector.Request.doGetSession(Request.java:2618) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at org.apache.catalina.connector.Request.getSession(Request.java:2375) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:841) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at org.jboss.weld.context.beanstore.http.LazySessionBeanStore.getSession(LazySessionBeanStore.java:72) [weld-core-1.1.8.Final-redhat-1.jar:1.1.8.Final-redhat-1]
> [JBossINF] at org.jboss.weld.context.beanstore.http.LazySessionBeanStore.<init>(LazySessionBeanStore.java:58) [weld-core-1.1.8.Final-redhat-1.jar:1.1.8.Final-redhat-1]
> [JBossINF] at org.jboss.weld.context.http.HttpSessionContextImpl.associate(HttpSessionContextImpl.java:31) [weld-core-1.1.8.Final-redhat-1.jar:1.1.8.Final-redhat-1]
> [JBossINF] at org.jboss.weld.context.http.HttpSessionContextImpl.associate(HttpSessionContextImpl.java:16) [weld-core-1.1.8.Final-redhat-1.jar:1.1.8.Final-redhat-1]
> [JBossINF] at org.jboss.weld.servlet.WeldListener.requestInitialized(WeldListener.java:134) [weld-core-1.1.8.Final-redhat-1.jar:1.1.8.Final-redhat-1]
> [JBossINF] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:452) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:931) [jbossweb-7.0.16.Final-redhat-1.jar:]
> [JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (JGRP-1619) Fix the OSGi manifest entries
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1619?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-1619:
--------------------------------
Which file is this ? jgroups.bnd ?
Can you attach the modified file here, or - even better - submit a pull request ?
> Fix the OSGi manifest entries
> -----------------------------
>
> Key: JGRP-1619
> URL: https://issues.jboss.org/browse/JGRP-1619
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.3
> Reporter: Markus Wolf
> Assignee: Bela Ban
> Fix For: 3.3
>
>
> Currently the headers import the packages org.apache.log4j and org.apache.logging.log4j not optionally.
> But in the documentation it is stated, that the log framework could be chosen by the implementor.
> This is currently not the case when deploy to an osgi runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months