[JBoss JIRA] (JGRP-2075) SYM/ASYM_ENCRYPT: don't use WeakHashMap for old ciphers
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2075?page=com.atlassian.jira.plugin.... ]
Bela Ban edited comment on JGRP-2075 at 9/5/16 12:28 PM:
---------------------------------------------------------
Simple solution: use {{BoundedHashMap}} with a fixed size. The max size is set via {{key_map_max_size}}
was (Author: belaban):
Simple solution: use {{BoundedHashMap}} with a fixed size.
> SYM/ASYM_ENCRYPT: don't use WeakHashMap for old ciphers
> -------------------------------------------------------
>
> Key: JGRP-2075
> URL: https://issues.jboss.org/browse/JGRP-2075
> Project: JGroups
> Issue Type: Task
> Reporter: Bela Ban
> Assignee: Bela Ban
> Priority: Minor
> Fix For: 3.6.11, 4.0
>
>
> Currently we use WeakHashMap, but should not, reasons outlined below. We could replace it with a LazyRemovalCache. Andrew's email refers to SecretKeys but this probably also applies to Ciphers.
> Andrew Haley's email:
> {quote}
> TL/DR: Please don't use WeakReferences, SoftReferences, etc. to cache
> any data which might point to native memory. In particular, never do
> this with instances of java.security.Key. Instead, implement either
> some kind of ageing strategy or a fixed-size cache.
> ...
> This is a warning to anybody who might cache crypto keys.
> A customer has been having problems with the exhaustion of native
> memory before the Java heap is full. It was fun trying to track down
> the cause, but it's now happened several times to several customers,
> and it's a serious problem for real-world usage in app servers.
> PKCS#11 is a standard way to communicate between applications and
> crypto libraries. There is a Java crypto provider which supports
> PKCS#11. Some of our customers must use this provider in order to get
> FIPS certification.
> The problem is this:
> A crypto key is a buffer in memory, allocated by the PKCS#11 native
> library. It's accessed via a handle which is stored as an integer
> field in a Java object. This Java object is a PhantomReference, so
> when the garbage collector detects that a crypto key is no longer
> reachable it is closed and the associated native memory is freed.
> Modern garbage collectors don't much bother to process objects in the
> old generation because it's not usually worthwhile. Thus, crypto keys
> don't get recycled very quickly. They can pile up in the old
> generation. This isn't a problem for the Java heap because the
> objects containing the references to crypto keys are very small.
> Unfortunately, the native side of a crypto key is much bigger, maybe
> up to a thousand times bigger. So if we have 4000 stale crypto keys
> in the heap that's not a problem, a few kbytes. But the native memory
> may be a megabyte.
> This problem is made even worse by Tomcat because it uses
> SoftReferences to cache crypto keys. SoftReferences are processed
> lazily, and maybe not at all until the Java heap runs out of memory.
> Unfortunately it doesn't, but the machine runs out of native memory
> instead.
> We could solve this simply by making instances of PKCS#11 keys really
> big Java objects by padding with dummy fields. Then, the GC would
> collect them quickly. This does work but it seriously impacts
> performance. Also, we could tweak the garbage collectors to clear out
> stale references more enthusiastically, but this impacts performance
> even more. There are some controls with the G1 collector which
> process SoftReferences more aggressively and these help, but again at
> the cost of performance.
> Finally: the Shanandoah collector we're working on handles this
> problem much better than the older collectors, but it's some
> way off.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (JGRP-2075) SYM/ASYM_ENCRYPT: don't use WeakHashMap for old ciphers
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2075?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2075:
--------------------------------
Simple solution: use {{BoundedHashMap}} with a fixed size.
> SYM/ASYM_ENCRYPT: don't use WeakHashMap for old ciphers
> -------------------------------------------------------
>
> Key: JGRP-2075
> URL: https://issues.jboss.org/browse/JGRP-2075
> Project: JGroups
> Issue Type: Task
> Reporter: Bela Ban
> Assignee: Bela Ban
> Priority: Minor
> Fix For: 3.6.11, 4.0
>
>
> Currently we use WeakHashMap, but should not, reasons outlined below. We could replace it with a LazyRemovalCache. Andrew's email refers to SecretKeys but this probably also applies to Ciphers.
> Andrew Haley's email:
> {quote}
> TL/DR: Please don't use WeakReferences, SoftReferences, etc. to cache
> any data which might point to native memory. In particular, never do
> this with instances of java.security.Key. Instead, implement either
> some kind of ageing strategy or a fixed-size cache.
> ...
> This is a warning to anybody who might cache crypto keys.
> A customer has been having problems with the exhaustion of native
> memory before the Java heap is full. It was fun trying to track down
> the cause, but it's now happened several times to several customers,
> and it's a serious problem for real-world usage in app servers.
> PKCS#11 is a standard way to communicate between applications and
> crypto libraries. There is a Java crypto provider which supports
> PKCS#11. Some of our customers must use this provider in order to get
> FIPS certification.
> The problem is this:
> A crypto key is a buffer in memory, allocated by the PKCS#11 native
> library. It's accessed via a handle which is stored as an integer
> field in a Java object. This Java object is a PhantomReference, so
> when the garbage collector detects that a crypto key is no longer
> reachable it is closed and the associated native memory is freed.
> Modern garbage collectors don't much bother to process objects in the
> old generation because it's not usually worthwhile. Thus, crypto keys
> don't get recycled very quickly. They can pile up in the old
> generation. This isn't a problem for the Java heap because the
> objects containing the references to crypto keys are very small.
> Unfortunately, the native side of a crypto key is much bigger, maybe
> up to a thousand times bigger. So if we have 4000 stale crypto keys
> in the heap that's not a problem, a few kbytes. But the native memory
> may be a megabyte.
> This problem is made even worse by Tomcat because it uses
> SoftReferences to cache crypto keys. SoftReferences are processed
> lazily, and maybe not at all until the Java heap runs out of memory.
> Unfortunately it doesn't, but the machine runs out of native memory
> instead.
> We could solve this simply by making instances of PKCS#11 keys really
> big Java objects by padding with dummy fields. Then, the GC would
> collect them quickly. This does work but it seriously impacts
> performance. Also, we could tweak the garbage collectors to clear out
> stale references more enthusiastically, but this impacts performance
> even more. There are some controls with the G1 collector which
> process SoftReferences more aggressively and these help, but again at
> the cost of performance.
> Finally: the Shanandoah collector we're working on handles this
> problem much better than the older collectors, but it's some
> way off.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (WFLY-7052) unclear NullPointerException if the deployment-scanner element is removed from the configuration
by Wolf-Dieter Fink (JIRA)
Wolf-Dieter Fink created WFLY-7052:
--------------------------------------
Summary: unclear NullPointerException if the deployment-scanner element is removed from the configuration
Key: WFLY-7052
URL: https://issues.jboss.org/browse/WFLY-7052
Project: WildFly
Issue Type: Bug
Components: Server
Affects Versions: 11.0.0.Alpha1
Reporter: Wolf-Dieter Fink
Assignee: Jason Greene
If the deployment scanner element is removed from the configuration of the standalone server a NullPointerException is logged which is unclear and difficult to find as the stack does not show any hint.
Config:
{xml}
<subsystem xmlns="urn:jboss:domain:deployment-scanner:2.0">
<!-- deployment-scanner path="deployments" relative-to="jboss.server.base.dir" scan-interval="5000" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}"/ -->
</subsystem>
{xml}
Log message:
ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 34) WFLYCTL0403: Unexpected failure during execution of the following operation(s): []: java.lang.NullPointerException
at org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185)
at org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767)
at org.jboss.as.controller.AbstractOperationContext.executeDoneStage(AbstractOperationContext.java:753)
at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:680)
at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:359)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (WFLY-7053) unclear NullPointerException if the deployment-scanner element is removed from the configuration
by Wolf-Dieter Fink (JIRA)
Wolf-Dieter Fink created WFLY-7053:
--------------------------------------
Summary: unclear NullPointerException if the deployment-scanner element is removed from the configuration
Key: WFLY-7053
URL: https://issues.jboss.org/browse/WFLY-7053
Project: WildFly
Issue Type: Bug
Components: Server
Affects Versions: 11.0.0.Alpha1
Reporter: Wolf-Dieter Fink
Assignee: Jason Greene
If the deployment scanner element is removed from the configuration of the standalone server a NullPointerException is logged which is unclear and difficult to find as the stack does not show any hint.
Config:
{xml}
<subsystem xmlns="urn:jboss:domain:deployment-scanner:2.0">
<!-- deployment-scanner path="deployments" relative-to="jboss.server.base.dir" scan-interval="5000" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}"/ -->
</subsystem>
{xml}
Log message:
ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 34) WFLYCTL0403: Unexpected failure during execution of the following operation(s): []: java.lang.NullPointerException
at org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185)
at org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767)
at org.jboss.as.controller.AbstractOperationContext.executeDoneStage(AbstractOperationContext.java:753)
at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:680)
at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:359)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (WFLY-6127) Throw IllegalStateException if JTA tx has an unsynchronized persistence context and the target is synchronized persistence context
by Viacheslav Astapkovich (JIRA)
[ https://issues.jboss.org/browse/WFLY-6127?page=com.atlassian.jira.plugin.... ]
Viacheslav Astapkovich edited comment on WFLY-6127 at 9/5/16 12:09 PM:
-----------------------------------------------------------------------
*The obtained defect:*
A defect related to the check of synchronization type (to satisfy JPA spec 2.1 section 7.6.4.1) was found in WildFly 10.1.0.Final.
The Method getSynchronizationType of ExtendedEntityManager class ALWAYS returns SYNCHRONIZED type of synchronization.
*FIX:*
We created a fork of WildFly-jpa project at: https://github.com/argustelecom/wildfly/tree/WFLY-6127
Our Fix commit: https://github.com/wildfly/wildfly/commit/3bff5fde3cfc23f3999dc75c320029e...
Corrections: The method getSynchronizationType returns declared synchronization type.
*Effect:*
We use our own realization of Martin Fowler’s pattern "Unit of Work". We initialize Unsynchronized Extended Persistence Context and our realization controls the synchronization with transaction.
Our Beans can be controlled by UnitOfWork, as well as be used as a part of WebService call.
This requires the declaration of synchronized persistence context.
We catch IllegalStateException after we have fixed the defect of synchronization type determination, because we initialize UNSYNCHRONIZED persistence context, but we declare synchronized persistence context in our beans.
However, our UnitOfWork realization controls synchronization of persistence context and we can synchronize context before the synchronization type check.
*Our actions:*
We had to add the possibility to check synchronization type in the testForMixedSynchronizationTypes method of TransactionScopedEntityManager class by isJoinToTransaction method (i.e. the actual state of synchronization).
This ability is realized by property "jboss.as.jpa.syncasjoin" in persistence.xml file. Only if this property is set to true - we perform testForMixedSynchronizationTypes by isJoinToTransaction method. The system operates as usual if this property is not defined or set to false.
Commit: https://github.com/wildfly/wildfly/commit/195a8a65a9fae006ad603e425f6a16d...
*Commentary:*
Though our actions deviate from JPA SPEC, we have a valid reason for acting this way:
- A question arose in the development process of the JPA specification : "Should we relax this if the PC has been joined to the transaction?".
Unfortunately, no feedback was given and Linda DeMichiel decided to keep it as it was
( https://java.net/projects/jpa-spec/lists/jsr338-experts/archive/2011-08/m... )
- We found a JIRA task https://java.net/jira/browse/JPA_SPEC-6 . And it was also closed too: "No feedback in favor of changing current approach"
was (Author: veselroger):
*The obtained defect:*
A defect related to the check of synchronization type (to satisfy JPA spec 2.1 section 7.6.4.1) was found in WildFly 10.1.0.Final.
The Method getSynchronizationType of ExtendedEntityManager class ALWAYS returns SYNCHRONIZED type of synchronization.
*FIX:*
We created a fork of WildFly-jpa project at: https://github.com/argustelecom/wildfly/tree/WFLY-6127
Our Fix commit: https://github.com/wildfly/wildfly/commit/3bff5fde3cfc23f3999dc75c320029e...
Corrections: The method getSynchronizationType returns declared synchronization type.
*Effect:*
We use our own realization of Martin Fowler’s pattern "Unit of Work". We initialize Unsynchronized Extended Persistence Context and our realization controls the synchronization with transaction.
Our Beans can be controlled by UnitOfWork, as well as be used as a part of WebService call.
This requires the declaration of synchronized persistence context.
We catch IllegalStateException after we have fixed the defect of synchronization type determination, because we initialize UNSYNCHRONIZED persistence context, but we declare synchronized persistence context in our beans.
However, our UnitOfWork realization controls synchronization of persistence context and we can synchronize context before the synchronization type check.
*Our actions:*
We had to add the possibility to check synchronization type in the testForMixedSynchronizationTypes method of TransactionScopedEntityManager class by isJoinToTransaction method (i.e. the actual type of synchronization).
This ability is realized by property "jboss.as.jpa.syncasjoin" in persistence.xml file. Only if this property is set to true - we perform testForMixedSynchronizationTypes by isJoinToTransaction method. The system operates as usual if this property is not defined or set to false.
Commit: https://github.com/wildfly/wildfly/commit/195a8a65a9fae006ad603e425f6a16d...
*Commentary:*
Though our actions deviate from JPA SPEC, we have a valid reason for acting this way:
- A question arose in the development process of the JPA specification : "Should we relax this if the PC has been joined to the transaction?".
Unfortunately, no feedback was given and Linda DeMichiel decided to keep it as it was
( https://java.net/projects/jpa-spec/lists/jsr338-experts/archive/2011-08/m... )
- We found a JIRA task https://java.net/jira/browse/JPA_SPEC-6 . And it was also closed too: "No feedback in favor of changing current approach"
> Throw IllegalStateException if JTA tx has an unsynchronized persistence context and the target is synchronized persistence context
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-6127
> URL: https://issues.jboss.org/browse/WFLY-6127
> Project: WildFly
> Issue Type: Bug
> Components: JPA / Hibernate
> Affects Versions: 9.0.2.Final
> Reporter: Mazen Mahmoud
> Assignee: Scott Marlow
> Priority: Blocker
> Fix For: 10.1.0.CR1, 10.1.0.Final
>
> Attachments: server-log.txt
>
>
> SPEC: If a component is called and the JTA transaction is propagated into that component:
> If there is a persistence context of type SynchronizationType.UNSYNCHRONIZED
> associated with the JTA transaction and the target component specifies a persistence context of type SynchronizationType.SYNCHRONIZED, the IllegalStateException is thrown by the container
> We have a stateful session bean (SFB1) / PC: TRANSACTION/UNSYNCHRONIZED)
> stateful session bean (SFB2) / PC: TRANSACTION/SYNCHRONIZED)
> SFB1 method M1 (REQUIRED) calls SFB2 Method 2 (REQUIRED):
> PC is propagated from SFB1 to SFB2 without any exception.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (WFLY-6127) Throw IllegalStateException if JTA tx has an unsynchronized persistence context and the target is synchronized persistence context
by kostd kostd (JIRA)
[ https://issues.jboss.org/browse/WFLY-6127?page=com.atlassian.jira.plugin.... ]
kostd kostd commented on WFLY-6127:
-----------------------------------
[~smarlow], can you accept [~veselroger] `s commit (which adds the jboss.as.jpa.syncasjoin option and "relaxed" check against current.isJoined instead of current.getSyncType() == Synchronized)? JCP process seems stopped (https://javaee-guardians.io/lack-of-java-ee-8-progress/) and our feedback to Linda is too late :(
What we can do to you able to accept? may be some ut or something like that?
We can rework to take your comments.
> Throw IllegalStateException if JTA tx has an unsynchronized persistence context and the target is synchronized persistence context
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-6127
> URL: https://issues.jboss.org/browse/WFLY-6127
> Project: WildFly
> Issue Type: Bug
> Components: JPA / Hibernate
> Affects Versions: 9.0.2.Final
> Reporter: Mazen Mahmoud
> Assignee: Scott Marlow
> Priority: Blocker
> Fix For: 10.1.0.CR1, 10.1.0.Final
>
> Attachments: server-log.txt
>
>
> SPEC: If a component is called and the JTA transaction is propagated into that component:
> If there is a persistence context of type SynchronizationType.UNSYNCHRONIZED
> associated with the JTA transaction and the target component specifies a persistence context of type SynchronizationType.SYNCHRONIZED, the IllegalStateException is thrown by the container
> We have a stateful session bean (SFB1) / PC: TRANSACTION/UNSYNCHRONIZED)
> stateful session bean (SFB2) / PC: TRANSACTION/SYNCHRONIZED)
> SFB1 method M1 (REQUIRED) calls SFB2 Method 2 (REQUIRED):
> PC is propagated from SFB1 to SFB2 without any exception.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years