[keycloak-dev] Race condition in SPI loading on startup

[Pedro Igor Silva]

I see. It is indeed an unexpected behavior.

But maybe also expected due to the fact that Wildfly/MSC uses different
threads during the boot process. I think that explains why you are only
getting this when using a debugger. If you set breakpoints you'll be
delaying work that normally would respect the correct order/phase of
execution.

I'm not sure if this should necessarily introduce changes to fix as we
trust how the MSC behaves.

On Wed, Sep 25, 2019 at 4:46 AM EXTERNAL Thiele Frank (TNG,
INST-CSS/BSV-OS2) <external.Frank.Thiele at bosch-si.com> wrote:

> Hi Pedro,
>
>
>
> I don’t know all the details of the deployment process. But yes, this is
> was I was observing.
>
> One can easily see this using the debugger based on the breakpoints
> provided in my last mail.
>
>
>
> Mit freundlichen Grüßen / Best regards
>
>
> *Frank Thiele *
> Open Source Services 2 - Product Group Customer Success Services
> (INST-CSS/BSV-OS2)
> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> GERMANY | www.bosch-si.com
> Tel. +49 30 726112-0 | Fax +49 30 726112-100 |
> external.Frank.Thiele at bosch-si.com
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr.
> Stefan Ferber, Michael Hahn, Dr. Aleksandar Mitrovic
>
>
>
> *Von:* Pedro Igor Silva <psilva at redhat.com>
> *Gesendet:* Dienstag, 24. September 2019 16:31
> *An:* EXTERNAL Thiele Frank (TNG, INST-CSS/BSV-OS2) <
> external.Frank.Thiele at bosch-si.com>
> *Cc:* keycloak-dev at lists.jboss.org
> *Betreff:* Re: [keycloak-dev] Race condition in SPI loading on startup
>
>
>
> Basically, what you are saying is that during startup the server
> deployment (which should cause the KSF.init to be called) is deployed prior
> to the keycloak subsystem ? So that the custom providers are not yet loaded
> thus not installed accordingly ?
>
>
>
> On Tue, Sep 24, 2019 at 11:08 AM EXTERNAL Thiele Frank (TNG,
> INST-CSS/BSV-OS2) <external.Frank.Thiele at bosch-si.com> wrote:
>
> Hi,
>
> Summary: When deploying an SPI extension for the identity-provider-mapper
> (just an example), this wasn't loaded due to that when using the debugger.
> In most cases this seems to work anyway - I could only reproduce it using a
> debugger.
>
> There seems to be a race condition between
> org.keycloak.services.DefaultKeycloakSessionFactory.init() and
> org.keycloak.subsystem.server.extension.KeycloakProviderDeploymentProcessor.deploy(DeploymentPhaseContext).
>
> The deploy() function of above is beyond others used for file system
> imports, e.g. custom extensions as jars (e.g. folder
> standalone/deployments). It interacts indirectly with the init() function /
> whole DefaultKeycloakSessionFactory class based on
> ProviderManagerRegistry.SINGLETON.
> deploy() alters the singleton to contain preBoot ProviderManager
> instances. Once these are set (properly...), the init() function takes them
> into account and loads them during startup, e.g. to load the custom jars.
> The init() function itself searches for preBoot managers and loads them
> once they are defined. If they happen to be defined later, then these are
> not loaded by init().
> In a positive case, this would then be handled by deploy() itself which
> has a handling to use the deployment process of init() (better: its class
> instance). But this is only working, if the init() function executes
> ProviderManagerRegistry.SINGLETON.setDeployer(this); before.
> One can reproduce this by debugging with certain breakpoints:
>
> -        First here:
>
> -             public void init() {
>
> -                 serverStartupTimestamp = System.currentTimeMillis();
>
> -
>
> -                 ProviderManager pm = new
> ProviderManager(KeycloakDeploymentInfo.create().services(),
> getClass().getClassLoader(), Config.scope().getArray("providers"));
>
> -                 spis.addAll(pm.loadSpis());
>
> -                 factoriesMap = loadFactories(pm);
>
> -           >>>>      for (ProviderManager manager :
> ProviderManagerRegistry.SINGLETON.getPreBoot()) {
>
> -                     Map<Class<? extends Provider>, Map<String,
> ProviderFactory>> factoryMap = loadFactories(manager);
>
> -        Second here:
>
> -             public void deploy(ProviderManager pm) {
>
> -                 ProviderManagerDeployer deployer = getDeployer();
>
> -                 if (deployer == null) {
>
> -          >>>>           preBoot.add(pm);
>
> -                 } else {
>
> -                     deployer.deploy(pm);
>
> -                 }
>
> -
>
> -            }
> By stepping through the workflow in a certain order, one can see the issue.
>
> Idea for fix: Add mutexes / semaphores / compareAndSwap based non-blocking
> synchronization means
>
> Mit freundlichen Grüßen / Best regards
>
> Frank Thiele
>
> Open Source Services 2 - Product Group Customer Success Services
> (INST-CSS/BSV-OS2)
> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> GERMANY | www.bosch-si.com<http://www.bosch-si.com>
> Tel. +49 30 726112-0 | Fax +49 30 726112-100 |
> external.Frank.Thiele at bosch-si.com<mailto:
> external.Frank.Thiele at bosch-si.com>
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr.
> Stefan Ferber, Michael Hahn, Dr. Aleksandar Mitrovic
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>