[jboss-jira] [JBoss JIRA] (JGRP-2075) SYM/ASYM_ENCRYPT: don't use WeakHashMap for old ciphers

[Bela Ban (JIRA)]

     [ https://issues.jboss.org/browse/JGRP-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bela Ban updated JGRP-2075:
---------------------------
    Fix Version/s: 3.6.11
                       (was: 3.6.10)


> SYM/ASYM_ENCRYPT: don't use WeakHashMap for old ciphers
> -------------------------------------------------------
>
>                 Key: JGRP-2075
>                 URL: https://issues.jboss.org/browse/JGRP-2075
>             Project: JGroups
>          Issue Type: Task
>            Reporter: Bela Ban
>            Assignee: Bela Ban
>            Priority: Minor
>             Fix For: 3.6.11, 4.0
>
>
> Currently we use WeakHashMap, but should not, reasons outlined below. We could replace it with a LazyRemovalCache. Andrew's email refers to SecretKeys but this probably also applies to Ciphers.
> Andrew Haley's email:
> {quote}
> TL/DR: Please don't use WeakReferences, SoftReferences, etc. to cache
> any data which might point to native memory. In particular, never do
> this with instances of java.security.Key. Instead, implement either
> some kind of ageing strategy or a fixed-size cache.
> ...
> This is a warning to anybody who might cache crypto keys.
> A customer has been having problems with the exhaustion of native
> memory before the Java heap is full.  It was fun trying to track down
> the cause, but it's now happened several times to several customers,
> and it's a serious problem for real-world usage in app servers.
> PKCS#11 is a standard way to communicate between applications and
> crypto libraries.  There is a Java crypto provider which supports
> PKCS#11.  Some of our customers must use this provider in order to get
> FIPS certification.
> The problem is this:
> A crypto key is a buffer in memory, allocated by the PKCS#11 native
> library.  It's accessed via a handle which is stored as an integer
> field in a Java object.  This Java object is a PhantomReference, so
> when the garbage collector detects that a crypto key is no longer
> reachable it is closed and the associated native memory is freed.
> Modern garbage collectors don't much bother to process objects in the
> old generation because it's not usually worthwhile.  Thus, crypto keys
> don't get recycled very quickly.  They can pile up in the old
> generation.  This isn't a problem for the Java heap because the
> objects containing the references to crypto keys are very small.
> Unfortunately, the native side of a crypto key is much bigger, maybe
> up to a thousand times bigger.  So if we have 4000 stale crypto keys
> in the heap that's not a problem, a few kbytes.  But the native memory
> may be a megabyte.
> This problem is made even worse by Tomcat because it uses
> SoftReferences to cache crypto keys.  SoftReferences are processed
> lazily, and maybe not at all until the Java heap runs out of memory.
> Unfortunately it doesn't, but the machine runs out of native memory
> instead.
> We could solve this simply by making instances of PKCS#11 keys really
> big Java objects by padding with dummy fields.  Then, the GC would
> collect them quickly.  This does work but it seriously impacts
> performance.  Also, we could tweak the garbage collectors to clear out
> stale references more enthusiastically, but this impacts performance
> even more.  There are some controls with the G1 collector which
> process SoftReferences more aggressively and these help, but again at
> the cost of performance.
> Finally: the Shanandoah collector we're working on handles this
> problem much better than the older collectors, but it's some
> way off.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)