From matzew at apache.org  Tue Apr  1 05:23:23 2014
From: matzew at apache.org (Matthias Wessendorf)
Date: Tue, 1 Apr 2014 11:23:23 +0200
Subject: [keycloak-dev] Keycloak WAR inside on an EAR file ?
In-Reply-To: <CAAg5f2Qn7DDZGWQHjWc3Fhf23p7Urst2Syz0LziRiVxmjb-MTQ@mail.gmail.com>
References: <CAAg5f2RTa6EZy_qLbrK70NmoW2om2bLWkN7Zhgo7Td=P9t02jg@mail.gmail.com>
	<CAAg5f2Qn7DDZGWQHjWc3Fhf23p7Urst2Syz0LziRiVxmjb-MTQ@mail.gmail.com>
Message-ID: <CAAg5f2QU-WcoMgZniO602AudpGg9VpDqOf3hO+nnXwZ2QXNT5w@mail.gmail.com>

OK, atm I am producing a 'custom war', exactly like this project:
https://github.com/keycloak/keycloak/tree/master/server


HOWEVER, now I moved the jboss-deployment-structure.xml out of the KC.war,
into the EAR's META-INF.

Plus I had to change <deployment> to <sub-deployment
name="keycloak-auth-server.war">

Thanks Stian for help on IRC!

-M


On Mon, Mar 31, 2014 at 5:57 PM, Matthias Wessendorf <matzew at apache.org>wrote:

> NOTE: I am using Alpha-4 version
>
>
> On Mon, Mar 31, 2014 at 5:57 PM, Matthias Wessendorf <matzew at apache.org>wrote:
>
>> Hello,
>>
>> I tried to pull the auth-server.war file into an EAR. Once that is
>> deployed to the JBoss 7.1.1 (I did not try WildFly), I am getting this
>> error on the JAX-RS Application initialization:
>>
>> 17:55:00,113 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/auth]] (MSC service thread 1-14) StandardWrapper.Throwable: java.lang.RuntimeException: Failed to construct public org.keycloak.server.KeycloakServerApplication(javax.servlet.ServletContext) throws java.io.FileNotFoundException
>>     at org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:144) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.jboss.resteasy.spi.ResteasyDeployment.createFromInjectorFactory(ResteasyDeployment.java:282) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:259) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:85) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1202) [jbossweb-7.0.13.Final.jar:]
>>     at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1102) [jbossweb-7.0.13.Final.jar:]
>>     at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3655) [jbossweb-7.0.13.Final.jar:]
>>     at org.apache.catalina.core.StandardContext.start(StandardContext.java:3873) [jbossweb-7.0.13.Final.jar:]
>>     at org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:90) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
>>     at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
>>     at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
>>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_09]
>>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_09]
>>     at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09]
>> Caused by: org.jboss.resteasy.spi.LoggableFailure: Unable to find contextual data of type: javax.servlet.ServletContext
>>     at org.jboss.resteasy.core.ContextParameterInjector$GenericDelegatingProxy.invoke(ContextParameterInjector.java:53) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at $Proxy39.getContextPath(Unknown Source)  at org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:37) [keycloak-services-1.0-alpha-4.jar:]
>>     at org.keycloak.server.KeycloakServerApplication.<init>(KeycloakServerApplication.java:41) [classes:]
>>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.7.0_09]
>>     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) [rt.jar:1.7.0_09]
>>     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_09]
>>     at java.lang.reflect.Constructor.newInstance(Constructor.java:525) [rt.jar:1.7.0_09]
>>     at org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:132) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     ... 14 more
>>
>> 17:55:00,123 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/auth]] (MSC service thread 1-14) Servlet /auth threw load() exception: org.jboss.resteasy.spi.LoggableFailure: Unable to find contextual data of type: javax.servlet.ServletContext
>>     at org.jboss.resteasy.core.ContextParameterInjector$GenericDelegatingProxy.invoke(ContextParameterInjector.java:53) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at $Proxy39.getContextPath(Unknown Source)  at org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:37) [keycloak-services-1.0-alpha-4.jar:]
>>     at org.keycloak.server.KeycloakServerApplication.<init>(KeycloakServerApplication.java:41) [classes:]
>>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.7.0_09]
>>     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) [rt.jar:1.7.0_09]
>>     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_09]
>>     at java.lang.reflect.Constructor.newInstance(Constructor.java:525) [rt.jar:1.7.0_09]
>>     at org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:132) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.jboss.resteasy.spi.ResteasyDeployment.createFromInjectorFactory(ResteasyDeployment.java:282) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:259) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:85) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36) [resteasy-jaxrs-2.3.2.Final.jar:]
>>     at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1202) [jbossweb-7.0.13.Final.jar:]
>>     at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1102) [jbossweb-7.0.13.Final.jar:]
>>     at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3655) [jbossweb-7.0.13.Final.jar:]
>>     at org.apache.catalina.core.StandardContext.start(StandardContext.java:3873) [jbossweb-7.0.13.Final.jar:]
>>     at org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:90) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
>>     at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
>>     at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
>>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_09]
>>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_09]
>>     at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09]
>>
>> I wonder if there is anything wrong here, w/ the
>> jboss-deployment-structure.xml file of the actual WAR file from Keycloak?
>>
>> --
>> Matthias Wessendorf
>>
>> blog: http://matthiaswessendorf.wordpress.com/
>> sessions: http://www.slideshare.net/mwessendorf
>> twitter: http://twitter.com/mwessendorf
>>
>
>
>
> --
> Matthias Wessendorf
>
> blog: http://matthiaswessendorf.wordpress.com/
> sessions: http://www.slideshare.net/mwessendorf
> twitter: http://twitter.com/mwessendorf
>



-- 
Matthias Wessendorf

blog: http://matthiaswessendorf.wordpress.com/
sessions: http://www.slideshare.net/mwessendorf
twitter: http://twitter.com/mwessendorf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20140401/baa6545c/attachment.html 

From bburke at redhat.com  Tue Apr  1 09:12:44 2014
From: bburke at redhat.com (Bill Burke)
Date: Tue, 01 Apr 2014 09:12:44 -0400
Subject: [keycloak-dev] Keycloak WAR inside on an EAR file ?
In-Reply-To: <CAAg5f2Qn7DDZGWQHjWc3Fhf23p7Urst2Syz0LziRiVxmjb-MTQ@mail.gmail.com>
References: <CAAg5f2RTa6EZy_qLbrK70NmoW2om2bLWkN7Zhgo7Td=P9t02jg@mail.gmail.com>
	<CAAg5f2Qn7DDZGWQHjWc3Fhf23p7Urst2Syz0LziRiVxmjb-MTQ@mail.gmail.com>
Message-ID: <533ABB4C.4080103@redhat.com>

I'll take a look sometime this week.  Need to finish some Resteasy work 
and also sleep.  Got flu.

On 3/31/2014 11:57 AM, Matthias Wessendorf wrote:
> NOTE: I am using Alpha-4 version
>
>
> On Mon, Mar 31, 2014 at 5:57 PM, Matthias Wessendorf <matzew at apache.org
> <mailto:matzew at apache.org>> wrote:
>
>     Hello,
>
>     I tried to pull the |auth-server.war| file into an EAR. Once that is
>     deployed to the JBoss 7.1.1 (I did not try WildFly), I am getting
>     this error on the JAX-RS Application initialization:
>
>     |17:55:00,113 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/auth]] (MSC service thread 1-14) StandardWrapper.Throwable: java.lang.RuntimeException: Failed to construct public org.keycloak.server.KeycloakServerApplication(javax.servlet.ServletContext) throws java.io.FileNotFoundException
>          at org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:144) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.jboss.resteasy.spi.ResteasyDeployment.createFromInjectorFactory(ResteasyDeployment.java:282) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:259) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:85) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1202) [jbossweb-7.0.13.Final.jar:]
>          at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1102) [jbossweb-7.0.13.Final.jar:]
>          at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3655) [jbossweb-7.0.13.Final.jar:]
>          at org.apache.catalina.core.StandardContext.start(StandardContext.java:3873) [jbossweb-7.0.13.Final.jar:]
>          at org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:90) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
>          at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
>          at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
>          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_09]
>          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_09]
>          at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09]
>     Caused by: org.jboss.resteasy.spi.LoggableFailure: Unable to find contextual data of type: javax.servlet.ServletContext
>          at org.jboss.resteasy.core.ContextParameterInjector$GenericDelegatingProxy.invoke(ContextParameterInjector.java:53) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at $Proxy39.getContextPath(Unknown Source)  at org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:37) [keycloak-services-1.0-alpha-4.jar:]
>          at org.keycloak.server.KeycloakServerApplication.<init>(KeycloakServerApplication.java:41) [classes:]
>          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.7.0_09]
>          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) [rt.jar:1.7.0_09]
>          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_09]
>          at java.lang.reflect.Constructor.newInstance(Constructor.java:525) [rt.jar:1.7.0_09]
>          at org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:132) [resteasy-jaxrs-2.3.2.Final.jar:]
>          ... 14 more
>
>     17:55:00,123 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/auth]] (MSC service thread 1-14) Servlet /auth threw load() exception: org.jboss.resteasy.spi.LoggableFailure: Unable to find contextual data of type: javax.servlet.ServletContext
>          at org.jboss.resteasy.core.ContextParameterInjector$GenericDelegatingProxy.invoke(ContextParameterInjector.java:53) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at $Proxy39.getContextPath(Unknown Source)  at org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:37) [keycloak-services-1.0-alpha-4.jar:]
>          at org.keycloak.server.KeycloakServerApplication.<init>(KeycloakServerApplication.java:41) [classes:]
>          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.7.0_09]
>          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) [rt.jar:1.7.0_09]
>          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_09]
>          at java.lang.reflect.Constructor.newInstance(Constructor.java:525) [rt.jar:1.7.0_09]
>          at org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:132) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.jboss.resteasy.spi.ResteasyDeployment.createFromInjectorFactory(ResteasyDeployment.java:282) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:259) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:85) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36) [resteasy-jaxrs-2.3.2.Final.jar:]
>          at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1202) [jbossweb-7.0.13.Final.jar:]
>          at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1102) [jbossweb-7.0.13.Final.jar:]
>          at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3655) [jbossweb-7.0.13.Final.jar:]
>          at org.apache.catalina.core.StandardContext.start(StandardContext.java:3873) [jbossweb-7.0.13.Final.jar:]
>          at org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:90) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
>          at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
>          at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
>          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_09]
>          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_09]
>          at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09]
>     |
>
>     I wonder if there is anything wrong here, w/ the
>     |jboss-deployment-structure.xml| file of the actual WAR file from
>     Keycloak?
>
>
>     --
>     Matthias Wessendorf
>
>     blog: http://matthiaswessendorf.wordpress.com/
>     sessions: http://www.slideshare.net/mwessendorf
>     twitter: http://twitter.com/mwessendorf
>
>
>
>
> --
> Matthias Wessendorf
>
> blog: http://matthiaswessendorf.wordpress.com/
> sessions: http://www.slideshare.net/mwessendorf
> twitter: http://twitter.com/mwessendorf
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From stian at redhat.com  Tue Apr  1 11:36:01 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Tue, 1 Apr 2014 11:36:01 -0400 (EDT)
Subject: [keycloak-dev] Server alias
In-Reply-To: <1210799566.5465097.1396366138264.JavaMail.zimbra@redhat.com>
Message-ID: <1684254339.5468644.1396366561359.JavaMail.zimbra@redhat.com>

To make it simpler to manage URLs for applications and clients (redirect, management, base) I propose we add server aliases.

A server alias is an alias that resolves to a hostname and optional port.

For example valid redirect uris could be:

http://${myserver}/myapp

Which could resolve to: http://www.myserver.com/myapp or http://localhost/myapp

We could also add a special built in server alias for the Keycloak server itself (for example ${keycloak}). This server alias would be resolved depending on the URL used to contact Keycloak.

I think this should be helpful for UPS as redirect URIs for the UPS bundled with Keycloak would use the Keycloak server alias.

From matzew at apache.org  Tue Apr  1 13:33:27 2014
From: matzew at apache.org (Matthias Wessendorf)
Date: Tue, 1 Apr 2014 19:33:27 +0200
Subject: [keycloak-dev] Server alias
In-Reply-To: <1684254339.5468644.1396366561359.JavaMail.zimbra@redhat.com>
References: <1210799566.5465097.1396366138264.JavaMail.zimbra@redhat.com>
	<1684254339.5468644.1396366561359.JavaMail.zimbra@redhat.com>
Message-ID: <CAAg5f2SbRSpeFPcoJj1r6NqJ0CGniP3bXNe=jtC9=buRTt=Qeg@mail.gmail.com>

sounds like a nice improvement


On Tue, Apr 1, 2014 at 5:36 PM, Stian Thorgersen <stian at redhat.com> wrote:

> To make it simpler to manage URLs for applications and clients (redirect,
> management, base) I propose we add server aliases.
>
> A server alias is an alias that resolves to a hostname and optional port.
>
> For example valid redirect uris could be:
>
> http://${myserver}/myapp
>
> Which could resolve to: http://www.myserver.com/myapp or
> http://localhost/myapp
>
> We could also add a special built in server alias for the Keycloak server
> itself (for example ${keycloak}). This server alias would be resolved
> depending on the URL used to contact Keycloak.
>
> I think this should be helpful for UPS as redirect URIs for the UPS
> bundled with Keycloak would use the Keycloak server alias.
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>



-- 
Matthias Wessendorf

blog: http://matthiaswessendorf.wordpress.com/
sessions: http://www.slideshare.net/mwessendorf
twitter: http://twitter.com/mwessendorf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20140401/abf9be01/attachment.html 

From matzew at apache.org  Wed Apr  2 05:43:11 2014
From: matzew at apache.org (Matthias Wessendorf)
Date: Wed, 2 Apr 2014 11:43:11 +0200
Subject: [keycloak-dev] Keycloak WAR inside on an EAR file ?
In-Reply-To: <533ABB4C.4080103@redhat.com>
References: <CAAg5f2RTa6EZy_qLbrK70NmoW2om2bLWkN7Zhgo7Td=P9t02jg@mail.gmail.com>
	<CAAg5f2Qn7DDZGWQHjWc3Fhf23p7Urst2Syz0LziRiVxmjb-MTQ@mail.gmail.com>
	<533ABB4C.4080103@redhat.com>
Message-ID: <CAAg5f2Qaayfok=t2yj1zqXn_t=rRUBY0m5+B+ng3q7XVkkKt4A@mail.gmail.com>

On Tue, Apr 1, 2014 at 3:12 PM, Bill Burke <bburke at redhat.com> wrote:

> I'll take a look sometime this week.  Need to finish some Resteasy work
> and also sleep.  Got flu.
>

Oh, sorry to hear! I hope you get better soon!




regarding my EAR, I (for tests) also replaced the RestEasy bits w/
3.0.6Final inside of the UPS war file.

But I keep getting errors around the
'org.jboss.resteasy.plugins.server.servlet.Servlet3AsyncHttpRequest' class
(running on JBoss AS7.1.1).



11:39:39,254 WARN  [org.jboss.as.ee] (MSC service thread 1-14) JBAS011006:
Not installing optional component
org.jboss.resteasy.plugins.server.servlet.Servlet3AsyncHttpRequest$Servlet3ExecutionContext$Servle3AsychronousResponse
due to exception:
org.jboss.as.server.deployment.DeploymentUnitProcessingException:
JBAS011054: Could not find default constructor for class
org.jboss.resteasy.plugins.server.servlet.Servlet3AsyncHttpRequest$Servlet3ExecutionContext$Servle3AsychronousResponse

at
org.jboss.as.ee.component.ComponentDescription$DefaultComponentConfigurator.configure(ComponentDescription.java:606)

at
org.jboss.as.ee.component.deployers.EEModuleConfigurationProcessor.deploy(EEModuleConfigurationProcessor.java:81)

at
org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:113)
[jboss-as-server-7.1.1.Final.jar:7.1.1.Final]

at
org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
[jboss-msc-1.0.2.GA.jar:1.0.2.GA]

at
org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
[jboss-msc-1.0.2.GA.jar:1.0.2.GA]

at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
[rt.jar:1.7.0_09]

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[rt.jar:1.7.0_09]

at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09]












>
> On 3/31/2014 11:57 AM, Matthias Wessendorf wrote:
> > NOTE: I am using Alpha-4 version
> >
> >
> > On Mon, Mar 31, 2014 at 5:57 PM, Matthias Wessendorf <matzew at apache.org
> > <mailto:matzew at apache.org>> wrote:
> >
> >     Hello,
> >
> >     I tried to pull the |auth-server.war| file into an EAR. Once that is
> >     deployed to the JBoss 7.1.1 (I did not try WildFly), I am getting
> >     this error on the JAX-RS Application initialization:
> >
> >     |17:55:00,113 ERROR
> [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/auth]]
> (MSC service thread 1-14) StandardWrapper.Throwable:
> java.lang.RuntimeException: Failed to construct public
> org.keycloak.server.KeycloakServerApplication(javax.servlet.ServletContext)
> throws java.io.FileNotFoundException
> >          at
> org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:144)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.jboss.resteasy.spi.ResteasyDeployment.createFromInjectorFactory(ResteasyDeployment.java:282)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:259)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:85)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1202)
> [jbossweb-7.0.13.Final.jar:]
> >          at
> org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1102)
> [jbossweb-7.0.13.Final.jar:]
> >          at
> org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3655)
> [jbossweb-7.0.13.Final.jar:]
> >          at
> org.apache.catalina.core.StandardContext.start(StandardContext.java:3873)
> [jbossweb-7.0.13.Final.jar:]
> >          at
> org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:90)
> [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
> >          at
> org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
> >          at
> org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
> >          at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> [rt.jar:1.7.0_09]
> >          at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [rt.jar:1.7.0_09]
> >          at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09]
> >     Caused by: org.jboss.resteasy.spi.LoggableFailure: Unable to find
> contextual data of type: javax.servlet.ServletContext
> >          at
> org.jboss.resteasy.core.ContextParameterInjector$GenericDelegatingProxy.invoke(ContextParameterInjector.java:53)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at $Proxy39.getContextPath(Unknown Source)  at
> org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:37)
> [keycloak-services-1.0-alpha-4.jar:]
> >          at
> org.keycloak.server.KeycloakServerApplication.<init>(KeycloakServerApplication.java:41)
> [classes:]
> >          at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> [rt.jar:1.7.0_09]
> >          at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> [rt.jar:1.7.0_09]
> >          at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> [rt.jar:1.7.0_09]
> >          at
> java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> [rt.jar:1.7.0_09]
> >          at
> org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:132)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          ... 14 more
> >
> >     17:55:00,123 ERROR
> [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/auth]]
> (MSC service thread 1-14) Servlet /auth threw load() exception:
> org.jboss.resteasy.spi.LoggableFailure: Unable to find contextual data of
> type: javax.servlet.ServletContext
> >          at
> org.jboss.resteasy.core.ContextParameterInjector$GenericDelegatingProxy.invoke(ContextParameterInjector.java:53)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at $Proxy39.getContextPath(Unknown Source)  at
> org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:37)
> [keycloak-services-1.0-alpha-4.jar:]
> >          at
> org.keycloak.server.KeycloakServerApplication.<init>(KeycloakServerApplication.java:41)
> [classes:]
> >          at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> [rt.jar:1.7.0_09]
> >          at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> [rt.jar:1.7.0_09]
> >          at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> [rt.jar:1.7.0_09]
> >          at
> java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> [rt.jar:1.7.0_09]
> >          at
> org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:132)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.jboss.resteasy.spi.ResteasyDeployment.createFromInjectorFactory(ResteasyDeployment.java:282)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:259)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:85)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36)
> [resteasy-jaxrs-2.3.2.Final.jar:]
> >          at
> org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1202)
> [jbossweb-7.0.13.Final.jar:]
> >          at
> org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1102)
> [jbossweb-7.0.13.Final.jar:]
> >          at
> org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3655)
> [jbossweb-7.0.13.Final.jar:]
> >          at
> org.apache.catalina.core.StandardContext.start(StandardContext.java:3873)
> [jbossweb-7.0.13.Final.jar:]
> >          at
> org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:90)
> [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
> >          at
> org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
> >          at
> org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
> >          at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> [rt.jar:1.7.0_09]
> >          at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [rt.jar:1.7.0_09]
> >          at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09]
> >     |
> >
> >     I wonder if there is anything wrong here, w/ the
> >     |jboss-deployment-structure.xml| file of the actual WAR file from
> >     Keycloak?
> >
> >
> >     --
> >     Matthias Wessendorf
> >
> >     blog: http://matthiaswessendorf.wordpress.com/
> >     sessions: http://www.slideshare.net/mwessendorf
> >     twitter: http://twitter.com/mwessendorf
> >
> >
> >
> >
> > --
> > Matthias Wessendorf
> >
> > blog: http://matthiaswessendorf.wordpress.com/
> > sessions: http://www.slideshare.net/mwessendorf
> > twitter: http://twitter.com/mwessendorf
> >
> >
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
>
> --
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>



-- 
Matthias Wessendorf

blog: http://matthiaswessendorf.wordpress.com/
sessions: http://www.slideshare.net/mwessendorf
twitter: http://twitter.com/mwessendorf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20140402/e148f90f/attachment-0001.html 

From stian at redhat.com  Wed Apr  2 06:33:02 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Wed, 2 Apr 2014 06:33:02 -0400 (EDT)
Subject: [keycloak-dev] Keycloak server config
In-Reply-To: <320768216.6022978.1396434571496.JavaMail.zimbra@redhat.com>
Message-ID: <355175571.6024453.1396434782105.JavaMail.zimbra@redhat.com>

We're starting to get quite a lot of things that can be configured globally for a Keycloak server, with more coming soon. This includes:

* Model DB
* Audit DB
* LDAP servers (currently per-realm, but it would be better to create global config that can be selected in realm)
* SMTP servers (currently per-realm, but it would be better to create global config that can be selected in realm)
* Server Alias (proposed to list yesterday)
* Theme config (default theme, fallback theme, theme dir)

I propose that we make this configurable through a single json file. In the distribution it would be standalone/configuration/keycloak.json. Like standalone.xml it would support system property expansion.

An initial idea of the structure of the file: https://gist.github.com/stianst/9931577

From bburke at redhat.com  Wed Apr  2 08:52:23 2014
From: bburke at redhat.com (Bill Burke)
Date: Wed, 02 Apr 2014 08:52:23 -0400
Subject: [keycloak-dev] Keycloak server config
In-Reply-To: <355175571.6024453.1396434782105.JavaMail.zimbra@redhat.com>
References: <355175571.6024453.1396434782105.JavaMail.zimbra@redhat.com>
Message-ID: <533C0807.8030802@redhat.com>



On 4/2/2014 6:33 AM, Stian Thorgersen wrote:
> We're starting to get quite a lot of things that can be configured globally for a Keycloak server, with more coming soon. This includes:
>
> * Model DB
> * Audit DB
> * LDAP servers (currently per-realm, but it would be better to create global config that can be selected in realm)

Why do you think an LDAP server would be a global item?

> * SMTP servers (currently per-realm, but it would be better to create global config that can be selected in realm)

Not sure I agree with this either.  While the SMTP server IP address may 
be the same the username, password, and "From" may be different per realm.


-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From stian at redhat.com  Wed Apr  2 08:58:22 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Wed, 2 Apr 2014 08:58:22 -0400 (EDT)
Subject: [keycloak-dev] Keycloak server config
In-Reply-To: <533C0807.8030802@redhat.com>
References: <355175571.6024453.1396434782105.JavaMail.zimbra@redhat.com>
	<533C0807.8030802@redhat.com>
Message-ID: <1510444814.6170311.1396443502454.JavaMail.zimbra@redhat.com>



----- Original Message -----
> From: "Bill Burke" <bburke at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Wednesday, 2 April, 2014 1:52:23 PM
> Subject: Re: [keycloak-dev] Keycloak server config
> 
> 
> 
> On 4/2/2014 6:33 AM, Stian Thorgersen wrote:
> > We're starting to get quite a lot of things that can be configured globally
> > for a Keycloak server, with more coming soon. This includes:
> >
> > * Model DB
> > * Audit DB
> > * LDAP servers (currently per-realm, but it would be better to create
> > global config that can be selected in realm)
> 
> Why do you think an LDAP server would be a global item?

This approach lets you configure LDAP servers globally. Then refer to them in the realm config. Basic idea is that you pull the environment specific stuff (i.e. ldap server hostname) out into a global config.

You may for example have two KC servers with the same config, but they are deployed at different data-centres so hostname of ldap server could be different. Also let's you switch between dev, test and production environments while still keeping realm config the same.

> 
> > * SMTP servers (currently per-realm, but it would be better to create
> > global config that can be selected in realm)
> 
> Not sure I agree with this either.  While the SMTP server IP address may
> be the same the username, password, and "From" may be different per realm.

Same as above, you use this to configure the SMTP server. In the realm you then select which SMTP server you want, then specify username/password and from on a realm level.

> 
> 
> --
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> 

From bruno at abstractj.org  Wed Apr  2 14:30:31 2014
From: bruno at abstractj.org (Bruno Oliveira)
Date: Wed, 2 Apr 2014 15:30:31 -0300
Subject: [keycloak-dev] Realm key pair
Message-ID: <etPan.533c5747.625558ec.8d1c@animus>

Good morning guys,

I was chatting with Lucas from our team about the key pair inside the JSON file (https://github.com/keycloak/keycloak/blob/master/examples/js-console/example-realm.json#L6). I was just wondering: how bad would be if we dynamically generate the key pair on JavaScript and send the signed data to the server? I?m considering never store it.

Does it make sense? If you guys agreed on that, we can help on it.

--  
abstractj


From bburke at redhat.com  Wed Apr  2 15:27:26 2014
From: bburke at redhat.com (Bill Burke)
Date: Wed, 02 Apr 2014 15:27:26 -0400
Subject: [keycloak-dev] Realm key pair
In-Reply-To: <etPan.533c5747.625558ec.8d1c@animus>
References: <etPan.533c5747.625558ec.8d1c@animus>
Message-ID: <533C649E.8030004@redhat.com>

Not sure what you mean.  The keypair is for the realm.  When you create 
a realm this keypair is automatically generated.  The only reason it 
exists in the example imported json files is so that the example adapter 
configs can run out of the box.

On 4/2/2014 2:30 PM, Bruno Oliveira wrote:
> Good morning guys,
>
> I was chatting with Lucas from our team about the key pair inside the JSON file (https://github.com/keycloak/keycloak/blob/master/examples/js-console/example-realm.json#L6). I was just wondering: how bad would be if we dynamically generate the key pair on JavaScript and send the signed data to the server? I?m considering never store it.
>
> Does it make sense? If you guys agreed on that, we can help on it.
>
> --
> abstractj
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From bruno at abstractj.org  Thu Apr  3 10:32:43 2014
From: bruno at abstractj.org (Bruno Oliveira)
Date: Thu, 3 Apr 2014 11:32:43 -0300
Subject: [keycloak-dev] Realm key pair
Message-ID: <etPan.533d710c.6b8b4567.c454@animus>

I see. I was just wondering if is possible to avoid the key pair exposition and if the idea is valid. For our clients, establish a key agreement (ECDH for example) and use the shared key to sign JSON[1].

Does it make sense?

[1] -?http://tools.ietf.org/html/draft-ietf-jose-json-web-algorithms-25#section-4.6.1

--  
abstractj

On April 2, 2014 at 4:27:29 PM, Bill Burke (bburke at redhat.com) wrote:
> > Not sure what you mean. The keypair is for the realm. When you  
> create
> a realm this keypair is automatically generated. The only reason  
> it
> exists in the example imported json files is so that the example  
> adapter
> configs can run out of the box.



From bburke at redhat.com  Thu Apr  3 18:06:30 2014
From: bburke at redhat.com (Bill Burke)
Date: Thu, 03 Apr 2014 18:06:30 -0400
Subject: [keycloak-dev] Realm key pair
In-Reply-To: <etPan.533d710c.6b8b4567.c454@animus>
References: <etPan.533d710c.6b8b4567.c454@animus>
Message-ID: <533DDB66.30004@redhat.com>

The keypair is not someting specific to a realm-client.  It is specific 
to the realm.  The realm signs all access tokens for all clients with 
its private key.  Currently we do not support a shared secret, only PKI. 
  And we'll probably only support PKI unless there is a popular client 
which can't support it.

On 4/3/2014 10:32 AM, Bruno Oliveira wrote:
> I see. I was just wondering if is possible to avoid the key pair exposition and if the idea is valid. For our clients, establish a key agreement (ECDH for example) and use the shared key to sign JSON[1].
>
> Does it make sense?
>
> [1] - http://tools.ietf.org/html/draft-ietf-jose-json-web-algorithms-25#section-4.6.1
>
> --
> abstractj
>
> On April 2, 2014 at 4:27:29 PM, Bill Burke (bburke at redhat.com) wrote:
>>> Not sure what you mean. The keypair is for the realm. When you
>> create
>> a realm this keypair is automatically generated. The only reason
>> it
>> exists in the example imported json files is so that the example
>> adapter
>> configs can run out of the box.
>

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From matzew at apache.org  Fri Apr  4 04:59:10 2014
From: matzew at apache.org (Matthias Wessendorf)
Date: Fri, 4 Apr 2014 10:59:10 +0200
Subject: [keycloak-dev] [AS7 Adapter] KeycloakAuthenticatorValve invoked too
	early
Message-ID: <CAAg5f2SpN0YmGAMWSpST=kbDybw6Od3oQQqXgb30JDSje2Jjhg@mail.gmail.com>

Hello,

trying to integrate the Keycloak "server" into my own JAX-RS application
(for the integration with the UnifiedPush Server). I started w/ a simple
app.

Generally I am doing something similar to what the LiveOak folks do:
I create leverage the Admin realm, using Java APIs:
https://github.com/liveoak-io/liveoak/blob/master/modules/keycloak/src/main/java/io/liveoak/keycloak/KeycloakServer.java#L72-L104


Now, to get rid of the 'keycloak.json' file for the AS7 adapter, I tried
the following, inside of the constructor of my JAX-RS Application class
(which extends KeycloakApplication):

....
final AdapterConfig config = new AdapterConfig();

config.setRealm(adminRealm.getName());
config.set......
......

final ObjectMapper om = new ObjectMapper();
String json = null;
try {
    json = om.writeValueAsString(config);
} catch (IOException e) {
    e.printStackTrace();
}

servletContext.setInitParameter(AdapterConstants.AUTH_DATA_PARAM_NAME,
json);
....


However, that happens too late (a ServletContextListener is too late as
well). This line of code:
https://github.com/keycloak/keycloak/blob/master/integration/as7-eap6/adapter/src/main/java/org/keycloak/adapters/as7/KeycloakAuthenticatorValve.java#L75

is invoked before the constructor of my JAX-RS Application class (which
extends KeycloakApplication); same with a context listener.


I am wondering what's the best way to provide the "keycloak.json" content,
using Java APIs ?


Greetings,
Matthias



-- 
Matthias Wessendorf

blog: http://matthiaswessendorf.wordpress.com/
sessions: http://www.slideshare.net/mwessendorf
twitter: http://twitter.com/mwessendorf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20140404/55661906/attachment.html 

From stian at redhat.com  Tue Apr  8 08:08:15 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Tue, 8 Apr 2014 08:08:15 -0400 (EDT)
Subject: [keycloak-dev] Audit finished
In-Reply-To: <705749075.1564349.1396958362350.JavaMail.zimbra@redhat.com>
Message-ID: <1530773039.1569367.1396958895520.JavaMail.zimbra@redhat.com>

Audit has been added. Quick summary of what's provided:

* Audit Provider SPI, including implementations for JPA and Mongo (provider is configured with -Dkeycloak.audit=jpa or -Dkeycloak.audit=mongo)
* Audit Listener SPI, including implementation for jboss-logging
* Users can view events for their account through account management
* Admins can view events for realm through admin console
* Timer service that runs periodically to clear expired events (runs by default every 15 min, can be configured with -Dkeycloak.audit.expirationSchedule)

By default the JPA audit provider is used, but realms have audit disabled. To enable audit for a realm:

* Open the admin console
* Select the realm
* Click on Audit
* Click on Config
* Click on Enabled switch to enable
* If you want events to be removed after an expiration time, set expiration time

Now you can logout, login, update your users profile, etc, etc. to create some events to view ;)

From bburke at redhat.com  Tue Apr  8 23:51:16 2014
From: bburke at redhat.com (Bill Burke)
Date: Tue, 08 Apr 2014 23:51:16 -0400
Subject: [keycloak-dev] Audit finished
In-Reply-To: <1530773039.1569367.1396958895520.JavaMail.zimbra@redhat.com>
References: <1530773039.1569367.1396958895520.JavaMail.zimbra@redhat.com>
Message-ID: <5344C3B4.6010802@redhat.com>

I still think you are mixing up auditing with events.  We can't be 
writing to a database each and every request multiple times.  IMO most 
of these audits should be pushed to a text log file.  Audits include:

* login success/failure
* illegal access
* etc.

I just don't think it would be useful to view these types of audits in 
the admin console.  Once you get beyond a handful of users, this 
information will just be overbearing and will need a tool to make sense of.

Events would be different though.  These would be things that probably 
need action. i.e.

* Admin is notified of a brute force attack from an IP
* User is notified that somebody tried to log in from China

Those would be interesting to view from the admin console.


On 4/8/2014 8:08 AM, Stian Thorgersen wrote:
> Audit has been added. Quick summary of what's provided:
>
> * Audit Provider SPI, including implementations for JPA and Mongo (provider is configured with -Dkeycloak.audit=jpa or -Dkeycloak.audit=mongo)
> * Audit Listener SPI, including implementation for jboss-logging
> * Users can view events for their account through account management
> * Admins can view events for realm through admin console
> * Timer service that runs periodically to clear expired events (runs by default every 15 min, can be configured with -Dkeycloak.audit.expirationSchedule)
>
> By default the JPA audit provider is used, but realms have audit disabled. To enable audit for a realm:
>
> * Open the admin console
> * Select the realm
> * Click on Audit
> * Click on Config
> * Click on Enabled switch to enable
> * If you want events to be removed after an expiration time, set expiration time
>
> Now you can logout, login, update your users profile, etc, etc. to create some events to view ;)
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From stian at redhat.com  Wed Apr  9 01:20:46 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Wed, 9 Apr 2014 01:20:46 -0400 (EDT)
Subject: [keycloak-dev] Audit finished
In-Reply-To: <5344C3B4.6010802@redhat.com>
References: <1530773039.1569367.1396958895520.JavaMail.zimbra@redhat.com>
	<5344C3B4.6010802@redhat.com>
Message-ID: <1849138030.2253339.1397020846314.JavaMail.zimbra@redhat.com>

What events needs to be persisted, and also viewable through admin and acct mngmt, depends on the scenario. Some will be happy with disabling it all together, others will want to audit every little security related event there is. Most will probably want some middle ground. I certainly believe success and failed login attempts are important, in-fact a lot of websites let users view their login history through their account management (see attachment).

With regards to controlling the audit size, I think we need to be able to:

* Filter events prior to persisting
* Expire events after a certain time
* Aggregate events

How the events are persisted will also differ. Some will want it saved to a log file (we already have this) others will want to save it to a store where they can query it in a sensible way. The ideal store would probably be something like Cassandra, as that supports querying, as well as running map reduce jobs which are very powerful to analyze the audit as well as aggregate the data over time, and is also highly performant for this type of data.

It's not all there yet feature wise, and I wasn't expecting to add everything for beta1 as there simply isn't time. The remaining work for this includes:

* Filter - being able to filter out events
* Email - configure what events you want to be notified about through email (this would be a AuditListener)
* More events - for example there's no events for suspected brute force attacks, or users login from unexpected IP addresses
* Fine grained control on expiration of events
* Improved audit viewer in admin console - at the moment all it does is to allow filtering what events are shown based on event, client or user. As you suggested this wouldn't make it to easy to analyze huge amount of events. Some built in reports would be nice, with the ability to define your own reports.

The naming is probably a bit off, as it is more an events framework that is also used for auditing. Would it be better if we renamed AuditListener and AuditProvider to EventListener and EventProvider?

----- Original Message -----
> From: "Bill Burke" <bburke at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Wednesday, 9 April, 2014 4:51:16 AM
> Subject: Re: [keycloak-dev] Audit finished
> 
> I still think you are mixing up auditing with events.  We can't be
> writing to a database each and every request multiple times.  IMO most
> of these audits should be pushed to a text log file.  Audits include:
> 
> * login success/failure
> * illegal access
> * etc.
> 
> I just don't think it would be useful to view these types of audits in
> the admin console.  Once you get beyond a handful of users, this
> information will just be overbearing and will need a tool to make sense of.
> 
> Events would be different though.  These would be things that probably
> need action. i.e.
> 
> * Admin is notified of a brute force attack from an IP
> * User is notified that somebody tried to log in from China
> 
> Those would be interesting to view from the admin console.
> 
> 
> On 4/8/2014 8:08 AM, Stian Thorgersen wrote:
> > Audit has been added. Quick summary of what's provided:
> >
> > * Audit Provider SPI, including implementations for JPA and Mongo (provider
> > is configured with -Dkeycloak.audit=jpa or -Dkeycloak.audit=mongo)
> > * Audit Listener SPI, including implementation for jboss-logging
> > * Users can view events for their account through account management
> > * Admins can view events for realm through admin console
> > * Timer service that runs periodically to clear expired events (runs by
> > default every 15 min, can be configured with
> > -Dkeycloak.audit.expirationSchedule)
> >
> > By default the JPA audit provider is used, but realms have audit disabled.
> > To enable audit for a realm:
> >
> > * Open the admin console
> > * Select the realm
> > * Click on Audit
> > * Click on Config
> > * Click on Enabled switch to enable
> > * If you want events to be removed after an expiration time, set expiration
> > time
> >
> > Now you can logout, login, update your users profile, etc, etc. to create
> > some events to view ;)
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
> 
> --
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: google-history.png
Type: image/png
Size: 144237 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/keycloak-dev/attachments/20140409/ae45bc5d/attachment-0001.png 

From bburke at redhat.com  Wed Apr  9 21:50:11 2014
From: bburke at redhat.com (Bill Burke)
Date: Wed, 09 Apr 2014 21:50:11 -0400
Subject: [keycloak-dev] build failing again
Message-ID: <5345F8D3.9050400@redhat.com>

  AuthProvidersIntegrationTest.registerUserLdapSuccess:226 ? 
IllegalArgument No ...


-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From mposolda at redhat.com  Wed Apr  9 13:36:54 2014
From: mposolda at redhat.com (Marek Posolda)
Date: Wed, 09 Apr 2014 19:36:54 +0200
Subject: [keycloak-dev] build failing again
In-Reply-To: <5345F8D3.9050400@redhat.com>
References: <5345F8D3.9050400@redhat.com>
Message-ID: <53458536.8090402@redhat.com>

Hi,

PR is here https://github.com/keycloak/keycloak/pull/330 . I am working 
on setup CI for windows to minimalize similar issues in the future.

Marek

On 10.4.2014 03:50, Bill Burke wrote:
>    AuthProvidersIntegrationTest.registerUserLdapSuccess:226 ?
> IllegalArgument No ...
>
>


From bburke at redhat.com  Thu Apr 10 05:23:12 2014
From: bburke at redhat.com (Bill Burke)
Date: Thu, 10 Apr 2014 05:23:12 -0400
Subject: [keycloak-dev] use resteasy 2.3.7 where can
Message-ID: <53466300.6060707@redhat.com>

Server components should be using resteasy 2.3.7 as that aligns with EAP 
6.3.  I have updated our code to reflect this.  The testsuite still uses 
Resteasy 3.0.8 though.  I also still need to test the demos to make sure 
they run and also modify the WAR build so that it uses built in resteasy 
rather than bundling resteasy.
-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From bburke at redhat.com  Thu Apr 10 11:33:47 2014
From: bburke at redhat.com (Bill Burke)
Date: Thu, 10 Apr 2014 11:33:47 -0400
Subject: [keycloak-dev] removed transitive dependencies
Message-ID: <5346B9DB.2030800@redhat.com>

Everything is now scoped provided except in server/.  We can't revert if 
you don't like it.  I think this will make things easier going forward 
as we will need to add/remove artifacts or change version of artifacts 
from built WARs depending on the environment (AS7 to EAP to Wildfly, 
etc.).  Wildfly maven build also has the requirement that all artificats 
not have transitive dependencies.  It makes you exclude them explicitly 
if you have any.

I still need to test this and will have more updates tomorrow.
-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From bburke at redhat.com  Fri Apr 11 13:30:30 2014
From: bburke at redhat.com (Bill Burke)
Date: Fri, 11 Apr 2014 13:30:30 -0400
Subject: [keycloak-dev] have a bundled war example for Aerogear UPS
Message-ID: <534826B6.8060805@redhat.com>

I have implemented support and an example of a self-bootstrapping 
bundled application with Keycloak Server.  It is one war with an 
application and keycloak server all in one with the application secured 
by Keycloak too.  Its very hacked because of Resteasy 2.3.7 limitations.

See this URL for more details:

https://github.com/keycloak/keycloak/tree/master/bundled-war-example

Also, Keycloak Server now is Resteasy 2.3.7 compatible and runs within 
EAP 6.2's Resteasy implementation!

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From jbalunas at redhat.com  Fri Apr 11 13:50:36 2014
From: jbalunas at redhat.com (Jay Balunas)
Date: Fri, 11 Apr 2014 13:50:36 -0400
Subject: [keycloak-dev] have a bundled war example for Aerogear UPS
In-Reply-To: <534826B6.8060805@redhat.com>
References: <534826B6.8060805@redhat.com>
Message-ID: <A7281ED3-9F89-43B9-BBB7-F821BF99D6EB@redhat.com>


On Apr 11, 2014, at 1:30 PM, Bill Burke <bburke at redhat.com> wrote:

> I have implemented support and an example of a self-bootstrapping bundled application with Keycloak Server.  It is one war with an application and keycloak server all in one with the application secured by Keycloak too.  Its very hacked because of Resteasy 2.3.7 limitations.

Nice, thanks a lot Bill!  I'll owe you a beer or two in San Francisco!

> 
> See this URL for more details:
> 
> https://github.com/keycloak/keycloak/tree/master/bundled-war-example
> 
> Also, Keycloak Server now is Resteasy 2.3.7 compatible and runs within EAP 6.2's Resteasy implementation!
> 
> -- 
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com



From bburke at redhat.com  Tue Apr 15 11:44:31 2014
From: bburke at redhat.com (Bill Burke)
Date: Tue, 15 Apr 2014 11:44:31 -0400
Subject: [keycloak-dev] MIA until next week
Message-ID: <534D53DF.4010400@redhat.com>

Leaving for DevNation/Red Hat Summit.  Not sure what my email access 
will be.  See you guys next week.

Bill

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From stian at redhat.com  Tue Apr 15 11:56:31 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Tue, 15 Apr 2014 11:56:31 -0400 (EDT)
Subject: [keycloak-dev] MIA until next week
In-Reply-To: <534D53DF.4010400@redhat.com>
References: <534D53DF.4010400@redhat.com>
Message-ID: <1913887719.5949806.1397577391717.JavaMail.zimbra@redhat.com>

Good luck with the presentation :)

----- Original Message -----
> From: "Bill Burke" <bburke at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Tuesday, 15 April, 2014 4:44:31 PM
> Subject: [keycloak-dev] MIA until next week
> 
> Leaving for DevNation/Red Hat Summit.  Not sure what my email access
> will be.  See you guys next week.
> 
> Bill
> 
> --
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> 

From mposolda at redhat.com  Tue Apr 15 13:49:24 2014
From: mposolda at redhat.com (Marek Posolda)
Date: Tue, 15 Apr 2014 19:49:24 +0200
Subject: [keycloak-dev] MIA until next week
In-Reply-To: <1913887719.5949806.1397577391717.JavaMail.zimbra@redhat.com>
References: <534D53DF.4010400@redhat.com>
	<1913887719.5949806.1397577391717.JavaMail.zimbra@redhat.com>
Message-ID: <534D7124.8080802@redhat.com>

Good luck from me as well:-)

Marek

On 15.4.2014 17:56, Stian Thorgersen wrote:
> Good luck with the presentation :)
>
> ----- Original Message -----
>> From: "Bill Burke" <bburke at redhat.com>
>> To: keycloak-dev at lists.jboss.org
>> Sent: Tuesday, 15 April, 2014 4:44:31 PM
>> Subject: [keycloak-dev] MIA until next week
>>
>> Leaving for DevNation/Red Hat Summit.  Not sure what my email access
>> will be.  See you guys next week.
>>
>> Bill
>>
>> --
>> Bill Burke
>> JBoss, a division of Red Hat
>> http://bill.burkecentral.com
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev


From mcaspers at redhat.com  Thu Apr 17 12:39:55 2014
From: mcaspers at redhat.com (Matt Casperson)
Date: Thu, 17 Apr 2014 12:39:55 -0400 (EDT)
Subject: [keycloak-dev] KeyCloak and HornetQ REST?
In-Reply-To: <442044067.4643512.1397752633272.JavaMail.zimbra@redhat.com>
Message-ID: <1018253692.4643975.1397752795812.JavaMail.zimbra@redhat.com>

We have a need for browser based apps to interact with a message queue, and the HornetQ REST interface looks like it could serve us quite well. Since the HornetQ REST interface uses RESTEasy, is it possible to secure it like any other REST service via KeyCloak? 

Regards 

Matthew Casperson 
RHCE, RHCJA, RHCJD # 111-072-237 
Engineering Content Services 
Brisbane, Australia 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20140417/0084ea07/attachment.html 

From mposolda at redhat.com  Thu Apr 17 16:11:34 2014
From: mposolda at redhat.com (Marek Posolda)
Date: Thu, 17 Apr 2014 22:11:34 +0200
Subject: [keycloak-dev] KeyCloak and HornetQ REST?
In-Reply-To: <1018253692.4643975.1397752795812.JavaMail.zimbra@redhat.com>
References: <1018253692.4643975.1397752795812.JavaMail.zimbra@redhat.com>
Message-ID: <53503576.3010401@redhat.com>

Hi,

yes, it should be possible. The best is to look at KC examples. Here 
https://github.com/keycloak/keycloak/tree/master/examples/demo-template/database-service 
is the example of resteasy based jax-rs service secured by Bearer 
tokens. The tokens are sent to it from the frontend application (either 
servlet JEE application - see 
https://github.com/keycloak/keycloak/tree/master/examples/demo-template/customer-app 
or JS application - see 
https://github.com/keycloak/keycloak/tree/master/examples/demo-template/customer-app-js 
)

Marek

On 17.4.2014 18:39, Matt Casperson wrote:
> We have a need for browser based apps to interact with a message 
> queue, and the HornetQ REST interface looks like it could serve us 
> quite well. Since the HornetQ REST interface uses RESTEasy, is it 
> possible to secure it like any other REST service via KeyCloak?
>
> Regards
>
> Matthew Casperson
> RHCE, RHCJA, RHCJD # 111-072-237 
> <https://www.redhat.com/wapps/training/certification/verify.html?certNumber=111-072-237&isSearch=False&verify=Verify>
> Engineering Content Services
> Brisbane, Australia
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20140417/fce21dd8/attachment-0001.html 

From stian at redhat.com  Thu Apr 24 09:26:52 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Thu, 24 Apr 2014 09:26:52 -0400 (EDT)
Subject: [keycloak-dev] Theme support for admin console
In-Reply-To: <2129424013.10563211.1398345762898.JavaMail.zimbra@redhat.com>
Message-ID: <2111386515.10565960.1398346012446.JavaMail.zimbra@redhat.com>

I'm working on adding theme support to the admin console, and the plan was to use the what we have for login and account forms.

To get this to work I can either use a Servlet or a JAX-RS endpoint to deliver resources. My preference is to use JAX-RS and have everything done through RestEasy, effectively dropping Servlets (and web fragments). This would also let us drop '/rest' from the other endpoints (so it would be '/realms/keycloak-admin/tokens/login' instead of '/rest/realms/keycloak-admin/tokens/login'). 

From stian at redhat.com  Fri Apr 25 09:02:39 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Fri, 25 Apr 2014 09:02:39 -0400 (EDT)
Subject: [keycloak-dev] Admin resources moved
In-Reply-To: <1177202827.11583962.1398430821075.JavaMail.zimbra@redhat.com>
Message-ID: <556061186.11586316.1398430959676.JavaMail.zimbra@redhat.com>

I've moved resources for admin-ui to 'forms/common-themes/src/main/resources/theme/admin/base'. The admin console now has theme support. 

For example by starting the server with:

  bin/standalone.sh -Dkeycloak.theme.admin=patternfly

You'll get the standard PatternFly l&f in the console

From bburke at redhat.com  Fri Apr 25 10:04:47 2014
From: bburke at redhat.com (Bill Burke)
Date: Fri, 25 Apr 2014 10:04:47 -0400
Subject: [keycloak-dev] Admin resources moved
In-Reply-To: <556061186.11586316.1398430959676.JavaMail.zimbra@redhat.com>
References: <556061186.11586316.1398430959676.JavaMail.zimbra@redhat.com>
Message-ID: <535A6B7F.6040702@redhat.com>

awesome work stian.

On 4/25/2014 9:02 AM, Stian Thorgersen wrote:
> I've moved resources for admin-ui to 'forms/common-themes/src/main/resources/theme/admin/base'. The admin console now has theme support.
>
> For example by starting the server with:
>
>    bin/standalone.sh -Dkeycloak.theme.admin=patternfly
>
> You'll get the standard PatternFly l&f in the console
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From stian at redhat.com  Fri Apr 25 12:07:41 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Fri, 25 Apr 2014 12:07:41 -0400 (EDT)
Subject: [keycloak-dev] Broke model tests
Message-ID: <1773939019.11814794.1398442061567.JavaMail.zimbra@redhat.com>

Some idiot (me) have broken the tests. Will fix it asap!

From stian at redhat.com  Fri Apr 25 13:00:09 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Fri, 25 Apr 2014 13:00:09 -0400 (EDT)
Subject: [keycloak-dev] Broke model tests
In-Reply-To: <1773939019.11814794.1398442061567.JavaMail.zimbra@redhat.com>
References: <1773939019.11814794.1398442061567.JavaMail.zimbra@redhat.com>
Message-ID: <398910860.11846327.1398445209168.JavaMail.zimbra@redhat.com>

Fixed

----- Original Message -----
> From: "Stian Thorgersen" <stian at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Friday, 25 April, 2014 5:07:41 PM
> Subject: [keycloak-dev] Broke model tests
> 
> Some idiot (me) have broken the tests. Will fix it asap!
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> 

From mposolda at redhat.com  Mon Apr 28 03:27:45 2014
From: mposolda at redhat.com (Marek Posolda)
Date: Mon, 28 Apr 2014 09:27:45 +0200
Subject: [keycloak-dev] Export/import implementation
Message-ID: <535E02F1.4090107@redhat.com>

I am planning to start soon on export/import. If I recall correctly, one 
of the requirements is to export the content of whole DB content 
(including IDs and password hashes) to JSON file, which can then be 
later imported into other DB. This will allow to migrate between 
environments and various DB types (For example from Mongo to MySQL and 
viceversa).

I have some question though

1) I assume that DB should be cleared before full import from JSON file? 
Or do we want to update existing data without deleting the previous 
content? I assume that this is used for migration, so it's not about 
updating but completely delete and recreate existing DB, correct?

2) How to implement it. I can see two approaches

a) Use model API to retrieve content of the DB into JSON file during 
export. Similarly during import use model API to sync objects back from 
JSON into model DB.

b) Add some methods to KeycloakSession interface like:

ObjectNode export();

void import(ObjectNode node);

and implement export/import separately for each model.

Approach (b) might be better for performance as it allows to directly 
use low-level queries specific to JPA, Mongo or other model 
implementations to export/import stuff more effectively in batch, 
however it will require changes in model implementations and probably 
adding more stuff into dependencies. So I am more convinced to use (a). 
Thoughts?


3) How will be export/import triggered?

I think that for security reasons, we want to always export into local 
file with KC server and similarly always import from local file. Is it 
correct? I can see approaches like:

a) Use KC admin console. By default, just "admin" will be able to 
export/import stuff . Data will be always exported/imported into JSON 
file local to server. So it will be possible to trigger export/import 
remotely from admin console, but just use local JSON file. The import 
would be tricky though as import will recreate all data (including admin 
realm and admin user) so it would need to cancel logout sessions 
including the session of admin user who triggered import.

b) Use some script, which will trigger JVM process for export/import 
data. Script can be executed locally from CMD. I can imagine something 
like this (executed from the directory with AS7/wildfly server):

./bin/keycloak-import -Dkeycloak.model=jpa 
-Ddata-to-import=/tmp/file-with-full-kc-data-to-import-which-were-previously-exported.json

Assumption is that distribution contains deployed KC inside 
standalone/deployments/auth-server.war. Script will be able to run JVM 
which will have access to needed KC jars on classpath and access to 
persistence.xml . In AS7/Wildfly environment the persistence.xml bundled 
inside auth-server.war is using datasource though, so this JVM process 
would also need to parse datasource from 
standalone/configuration/standalone.xml as it won't be executed in 
managed AS7/Wildfly environment.

c) Use something similar to approach (b) but execute export/import from 
AS7/Wildfly CLI or admin console. The advantage is that it is triggered 
from the managed (AS7 or Wildfly) so has access to server resources like 
datasource referenced from persistence.xml

Any other possibility I am missing?

Thanks,
Marek

From stian at redhat.com  Mon Apr 28 04:39:44 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Mon, 28 Apr 2014 04:39:44 -0400 (EDT)
Subject: [keycloak-dev] Export/import implementation
In-Reply-To: <535E02F1.4090107@redhat.com>
References: <535E02F1.4090107@redhat.com>
Message-ID: <17830958.12858879.1398674384993.JavaMail.zimbra@redhat.com>



----- Original Message -----
> From: "Marek Posolda" <mposolda at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Monday, 28 April, 2014 8:27:45 AM
> Subject: [keycloak-dev] Export/import implementation
> 
> I am planning to start soon on export/import. If I recall correctly, one
> of the requirements is to export the content of whole DB content
> (including IDs and password hashes) to JSON file, which can then be
> later imported into other DB. This will allow to migrate between
> environments and various DB types (For example from Mongo to MySQL and
> viceversa).
> 
> I have some question though
> 
> 1) I assume that DB should be cleared before full import from JSON file?
> Or do we want to update existing data without deleting the previous
> content? I assume that this is used for migration, so it's not about
> updating but completely delete and recreate existing DB, correct?

Yes, I think first pass should be full DB export and import. If there is time, or at some point in the future, we can consider being able to import/export individual realms, and even some form of a merge on import

> 
> 2) How to implement it. I can see two approaches
> 
> a) Use model API to retrieve content of the DB into JSON file during
> export. Similarly during import use model API to sync objects back from
> JSON into model DB.
> 
> b) Add some methods to KeycloakSession interface like:
> 
> ObjectNode export();
> 
> void import(ObjectNode node);
> 
> and implement export/import separately for each model.
> 
> Approach (b) might be better for performance as it allows to directly
> use low-level queries specific to JPA, Mongo or other model
> implementations to export/import stuff more effectively in batch,
> however it will require changes in model implementations and probably
> adding more stuff into dependencies. So I am more convinced to use (a).
> Thoughts?

I'd say let's go with option a. If performance does become a real headache in the future we can reconsider

> 
> 
> 3) How will be export/import triggered?
> 
> I think that for security reasons, we want to always export into local
> file with KC server and similarly always import from local file. Is it
> correct? I can see approaches like:
> 
> a) Use KC admin console. By default, just "admin" will be able to
> export/import stuff . Data will be always exported/imported into JSON
> file local to server. So it will be possible to trigger export/import
> remotely from admin console, but just use local JSON file. The import
> would be tricky though as import will recreate all data (including admin
> realm and admin user) so it would need to cancel logout sessions
> including the session of admin user who triggered import.
> 
> b) Use some script, which will trigger JVM process for export/import
> data. Script can be executed locally from CMD. I can imagine something
> like this (executed from the directory with AS7/wildfly server):
> 
> ./bin/keycloak-import -Dkeycloak.model=jpa
> -Ddata-to-import=/tmp/file-with-full-kc-data-to-import-which-were-previously-exported.json
> 
> Assumption is that distribution contains deployed KC inside
> standalone/deployments/auth-server.war. Script will be able to run JVM
> which will have access to needed KC jars on classpath and access to
> persistence.xml . In AS7/Wildfly environment the persistence.xml bundled
> inside auth-server.war is using datasource though, so this JVM process
> would also need to parse datasource from
> standalone/configuration/standalone.xml as it won't be executed in
> managed AS7/Wildfly environment.
> 
> c) Use something similar to approach (b) but execute export/import from
> AS7/Wildfly CLI or admin console. The advantage is that it is triggered
> from the managed (AS7 or Wildfly) so has access to server resources like
> datasource referenced from persistence.xml

First pass would be to make users import/export with a script, or something similar. I don't think we need any support through the admin console straight up.

An alternative to script/CLI would be to just make the "server" itself do it. You'd just start the server with -Dexport=<my file> and it would export the db, then call System.exit(). While exporting we need to make sure the server doesn't allow any requests so we need a way to "pause" the server. Once you've upgraded to a new version you'd just start it with -Dimport<my file> the first time and it would import the db. Same thing on import, the server would be paused until the import is completed. We can probably import the pause with a JAX-RS filter.

It would be great if we could get the server to automatically use the export/import when an upgrade has occurred. We'd save a model "version" in the database, and whenever the server starts and detects an invalid version it would export the data to the json file, clear the database and re-import the data. We might need a prompt for users before clearing the db though. This may be complicated/time consuming to add though, but I think it would be worth considering at least.


> 
> Any other possibility I am missing?
> 
> Thanks,
> Marek
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> 

From mposolda at redhat.com  Mon Apr 28 05:28:57 2014
From: mposolda at redhat.com (Marek Posolda)
Date: Mon, 28 Apr 2014 11:28:57 +0200
Subject: [keycloak-dev] Export/import implementation
In-Reply-To: <17830958.12858879.1398674384993.JavaMail.zimbra@redhat.com>
References: <535E02F1.4090107@redhat.com>
	<17830958.12858879.1398674384993.JavaMail.zimbra@redhat.com>
Message-ID: <535E1F59.5000305@redhat.com>

On 28.4.2014 10:39, Stian Thorgersen wrote:
>
> ----- Original Message -----
>> From: "Marek Posolda" <mposolda at redhat.com>
>> To: keycloak-dev at lists.jboss.org
>> Sent: Monday, 28 April, 2014 8:27:45 AM
>> Subject: [keycloak-dev] Export/import implementation
>>
>> I am planning to start soon on export/import. If I recall correctly, one
>> of the requirements is to export the content of whole DB content
>> (including IDs and password hashes) to JSON file, which can then be
>> later imported into other DB. This will allow to migrate between
>> environments and various DB types (For example from Mongo to MySQL and
>> viceversa).
>>
>> I have some question though
>>
>> 1) I assume that DB should be cleared before full import from JSON file?
>> Or do we want to update existing data without deleting the previous
>> content? I assume that this is used for migration, so it's not about
>> updating but completely delete and recreate existing DB, correct?
> Yes, I think first pass should be full DB export and import. If there is time, or at some point in the future, we can consider being able to import/export individual realms, and even some form of a merge on import
>
>> 2) How to implement it. I can see two approaches
>>
>> a) Use model API to retrieve content of the DB into JSON file during
>> export. Similarly during import use model API to sync objects back from
>> JSON into model DB.
>>
>> b) Add some methods to KeycloakSession interface like:
>>
>> ObjectNode export();
>>
>> void import(ObjectNode node);
>>
>> and implement export/import separately for each model.
>>
>> Approach (b) might be better for performance as it allows to directly
>> use low-level queries specific to JPA, Mongo or other model
>> implementations to export/import stuff more effectively in batch,
>> however it will require changes in model implementations and probably
>> adding more stuff into dependencies. So I am more convinced to use (a).
>> Thoughts?
> I'd say let's go with option a. If performance does become a real headache in the future we can reconsider
>
>>
>> 3) How will be export/import triggered?
>>
>> I think that for security reasons, we want to always export into local
>> file with KC server and similarly always import from local file. Is it
>> correct? I can see approaches like:
>>
>> a) Use KC admin console. By default, just "admin" will be able to
>> export/import stuff . Data will be always exported/imported into JSON
>> file local to server. So it will be possible to trigger export/import
>> remotely from admin console, but just use local JSON file. The import
>> would be tricky though as import will recreate all data (including admin
>> realm and admin user) so it would need to cancel logout sessions
>> including the session of admin user who triggered import.
>>
>> b) Use some script, which will trigger JVM process for export/import
>> data. Script can be executed locally from CMD. I can imagine something
>> like this (executed from the directory with AS7/wildfly server):
>>
>> ./bin/keycloak-import -Dkeycloak.model=jpa
>> -Ddata-to-import=/tmp/file-with-full-kc-data-to-import-which-were-previously-exported.json
>>
>> Assumption is that distribution contains deployed KC inside
>> standalone/deployments/auth-server.war. Script will be able to run JVM
>> which will have access to needed KC jars on classpath and access to
>> persistence.xml . In AS7/Wildfly environment the persistence.xml bundled
>> inside auth-server.war is using datasource though, so this JVM process
>> would also need to parse datasource from
>> standalone/configuration/standalone.xml as it won't be executed in
>> managed AS7/Wildfly environment.
>>
>> c) Use something similar to approach (b) but execute export/import from
>> AS7/Wildfly CLI or admin console. The advantage is that it is triggered
>> from the managed (AS7 or Wildfly) so has access to server resources like
>> datasource referenced from persistence.xml
> First pass would be to make users import/export with a script, or something similar. I don't think we need any support through the admin console straight up.
>
> An alternative to script/CLI would be to just make the "server" itself do it. You'd just start the server with -Dexport=<my file> and it would export the db, then call System.exit(). While exporting we need to make sure the server doesn't allow any requests so we need a way to "pause" the server. Once you've upgraded to a new version you'd just start it with -Dimport<my file> the first time and it would import the db. Same thing on import, the server would be paused until the import is completed. We can probably import the pause with a JAX-RS filter.
yes, I think it will work and KC would be able to find DS as it's 
executed in managed environment and filter would take care of the pauses 
during import/export. So I will likely go this way. Thanks.

I was also thinking about adding filter if we want to support 
export/import from console, however supporting at with console would be 
more complicated as it would require pause of already running server, 
clearing all AccessCodeEntry tickets in progress, logout all existing 
sessions etc. If we want to support it just at startup of the server, 
things are much easier.
>
> It would be great if we could get the server to automatically use the export/import when an upgrade has occurred. We'd save a model "version" in the database, and whenever the server starts and detects an invalid version it would export the data to the json file, clear the database and re-import the data. We might need a prompt for users before clearing the db though. This may be complicated/time consuming to add though, but I think it would be worth considering at least.
I will look once I have all other things done. However I think that some 
manual work would be almost always needed from users during version 
update. For example there might be some new configuration options of the 
realm configuration and other configuration options might be removed. 
Some might be changed to support different type (For example some 
"boolean" property might be changed to support enum with 3 values etc). 
Maybe we can provide also tool to update previously exported JSON file 
to newer version of model? I know that in GateIn there is usage of this: 
https://windup.jboss.org/ . I don't know details and not sure if it's 
good also for our scenario though.

Marek

>
>
>> Any other possibility I am missing?
>>
>> Thanks,
>> Marek
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>


From stian at redhat.com  Mon Apr 28 06:02:43 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Mon, 28 Apr 2014 06:02:43 -0400 (EDT)
Subject: [keycloak-dev] Export/import implementation
In-Reply-To: <535E1F59.5000305@redhat.com>
References: <535E02F1.4090107@redhat.com>
	<17830958.12858879.1398674384993.JavaMail.zimbra@redhat.com>
	<535E1F59.5000305@redhat.com>
Message-ID: <458895711.12922386.1398679363099.JavaMail.zimbra@redhat.com>



----- Original Message -----
> From: "Marek Posolda" <mposolda at redhat.com>
> To: "Stian Thorgersen" <stian at redhat.com>
> Cc: keycloak-dev at lists.jboss.org
> Sent: Monday, 28 April, 2014 10:28:57 AM
> Subject: Re: [keycloak-dev] Export/import implementation
> 
> On 28.4.2014 10:39, Stian Thorgersen wrote:
> >
> > ----- Original Message -----
> >> From: "Marek Posolda" <mposolda at redhat.com>
> >> To: keycloak-dev at lists.jboss.org
> >> Sent: Monday, 28 April, 2014 8:27:45 AM
> >> Subject: [keycloak-dev] Export/import implementation
> >>
> >> I am planning to start soon on export/import. If I recall correctly, one
> >> of the requirements is to export the content of whole DB content
> >> (including IDs and password hashes) to JSON file, which can then be
> >> later imported into other DB. This will allow to migrate between
> >> environments and various DB types (For example from Mongo to MySQL and
> >> viceversa).
> >>
> >> I have some question though
> >>
> >> 1) I assume that DB should be cleared before full import from JSON file?
> >> Or do we want to update existing data without deleting the previous
> >> content? I assume that this is used for migration, so it's not about
> >> updating but completely delete and recreate existing DB, correct?
> > Yes, I think first pass should be full DB export and import. If there is
> > time, or at some point in the future, we can consider being able to
> > import/export individual realms, and even some form of a merge on import
> >
> >> 2) How to implement it. I can see two approaches
> >>
> >> a) Use model API to retrieve content of the DB into JSON file during
> >> export. Similarly during import use model API to sync objects back from
> >> JSON into model DB.
> >>
> >> b) Add some methods to KeycloakSession interface like:
> >>
> >> ObjectNode export();
> >>
> >> void import(ObjectNode node);
> >>
> >> and implement export/import separately for each model.
> >>
> >> Approach (b) might be better for performance as it allows to directly
> >> use low-level queries specific to JPA, Mongo or other model
> >> implementations to export/import stuff more effectively in batch,
> >> however it will require changes in model implementations and probably
> >> adding more stuff into dependencies. So I am more convinced to use (a).
> >> Thoughts?
> > I'd say let's go with option a. If performance does become a real headache
> > in the future we can reconsider
> >
> >>
> >> 3) How will be export/import triggered?
> >>
> >> I think that for security reasons, we want to always export into local
> >> file with KC server and similarly always import from local file. Is it
> >> correct? I can see approaches like:
> >>
> >> a) Use KC admin console. By default, just "admin" will be able to
> >> export/import stuff . Data will be always exported/imported into JSON
> >> file local to server. So it will be possible to trigger export/import
> >> remotely from admin console, but just use local JSON file. The import
> >> would be tricky though as import will recreate all data (including admin
> >> realm and admin user) so it would need to cancel logout sessions
> >> including the session of admin user who triggered import.
> >>
> >> b) Use some script, which will trigger JVM process for export/import
> >> data. Script can be executed locally from CMD. I can imagine something
> >> like this (executed from the directory with AS7/wildfly server):
> >>
> >> ./bin/keycloak-import -Dkeycloak.model=jpa
> >> -Ddata-to-import=/tmp/file-with-full-kc-data-to-import-which-were-previously-exported.json
> >>
> >> Assumption is that distribution contains deployed KC inside
> >> standalone/deployments/auth-server.war. Script will be able to run JVM
> >> which will have access to needed KC jars on classpath and access to
> >> persistence.xml . In AS7/Wildfly environment the persistence.xml bundled
> >> inside auth-server.war is using datasource though, so this JVM process
> >> would also need to parse datasource from
> >> standalone/configuration/standalone.xml as it won't be executed in
> >> managed AS7/Wildfly environment.
> >>
> >> c) Use something similar to approach (b) but execute export/import from
> >> AS7/Wildfly CLI or admin console. The advantage is that it is triggered
> >> from the managed (AS7 or Wildfly) so has access to server resources like
> >> datasource referenced from persistence.xml
> > First pass would be to make users import/export with a script, or something
> > similar. I don't think we need any support through the admin console
> > straight up.
> >
> > An alternative to script/CLI would be to just make the "server" itself do
> > it. You'd just start the server with -Dexport=<my file> and it would
> > export the db, then call System.exit(). While exporting we need to make
> > sure the server doesn't allow any requests so we need a way to "pause" the
> > server. Once you've upgraded to a new version you'd just start it with
> > -Dimport<my file> the first time and it would import the db. Same thing on
> > import, the server would be paused until the import is completed. We can
> > probably import the pause with a JAX-RS filter.
> yes, I think it will work and KC would be able to find DS as it's
> executed in managed environment and filter would take care of the pauses
> during import/export. So I will likely go this way. Thanks.
> 
> I was also thinking about adding filter if we want to support
> export/import from console, however supporting at with console would be
> more complicated as it would require pause of already running server,
> clearing all AccessCodeEntry tickets in progress, logout all existing
> sessions etc. If we want to support it just at startup of the server,
> things are much easier.
> >
> > It would be great if we could get the server to automatically use the
> > export/import when an upgrade has occurred. We'd save a model "version" in
> > the database, and whenever the server starts and detects an invalid
> > version it would export the data to the json file, clear the database and
> > re-import the data. We might need a prompt for users before clearing the
> > db though. This may be complicated/time consuming to add though, but I
> > think it would be worth considering at least.
> I will look once I have all other things done. However I think that some
> manual work would be almost always needed from users during version
> update. For example there might be some new configuration options of the
> realm configuration and other configuration options might be removed.
> Some might be changed to support different type (For example some
> "boolean" property might be changed to support enum with 3 values etc).
> Maybe we can provide also tool to update previously exported JSON file
> to newer version of model? I know that in GateIn there is usage of this:
> https://windup.jboss.org/ . I don't know details and not sure if it's
> good also for our scenario though.

I was thinking the import should be able to import an export from any previous versions. The json export should contain a version number then it goes through a chain of converters to get to the current version. That way each new version only needs to know how to convert from the previous version, but you can still import any version.

> 
> Marek
> 
> >
> >
> >> Any other possibility I am missing?
> >>
> >> Thanks,
> >> Marek
> >> _______________________________________________
> >> keycloak-dev mailing list
> >> keycloak-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>
> 
> 

From bburke at redhat.com  Mon Apr 28 08:52:16 2014
From: bburke at redhat.com (Bill Burke)
Date: Mon, 28 Apr 2014 08:52:16 -0400
Subject: [keycloak-dev] Export/import implementation
In-Reply-To: <535E02F1.4090107@redhat.com>
References: <535E02F1.4090107@redhat.com>
Message-ID: <535E4F00.7020506@redhat.com>



On 4/28/2014 3:27 AM, Marek Posolda wrote:
> I am planning to start soon on export/import. If I recall correctly, one
> of the requirements is to export the content of whole DB content
> (including IDs and password hashes) to JSON file, which can then be
> later imported into other DB. This will allow to migrate between
> environments and various DB types (For example from Mongo to MySQL and
> viceversa).
>

IMO, a full export (of credentials) should require a secret given by the 
admin that will be used to encrypt the export.  The export should only 
be saved locally to disk and not available over the network.

> I have some question though
>
> 1) I assume that DB should be cleared before full import from JSON file?
> Or do we want to update existing data without deleting the previous
> content? I assume that this is used for migration, so it's not about
> updating but completely delete and recreate existing DB, correct?
>
> 2) How to implement it. I can see two approaches
>
> a) Use model API to retrieve content of the DB into JSON file during
> export. Similarly during import use model API to sync objects back from
> JSON into model DB.
>
> b) Add some methods to KeycloakSession interface like:
>
> ObjectNode export();
>
> void import(ObjectNode node);
>
> and implement export/import separately for each model.
>
> Approach (b) might be better for performance as it allows to directly
> use low-level queries specific to JPA, Mongo or other model
> implementations to export/import stuff more effectively in batch,
> however it will require changes in model implementations and probably
> adding more stuff into dependencies. So I am more convinced to use (a).
> Thoughts?
>

"a", IMO.  Easier to maintain.

>
> 3) How will be export/import triggered?
>
> I think that for security reasons, we want to always export into local
> file with KC server and similarly always import from local file. Is it
> correct? I can see approaches like:
>

You can already import a full json description from the admin console.

> a) Use KC admin console. By default, just "admin" will be able to
> export/import stuff . Data will be always exported/imported into JSON
> file local to server. So it will be possible to trigger export/import
> remotely from admin console, but just use local JSON file. The import
> would be tricky though as import will recreate all data (including admin
> realm and admin user) so it would need to cancel logout sessions
> including the session of admin user who triggered import.
>
> b) Use some script, which will trigger JVM process for export/import
> data. Script can be executed locally from CMD. I can imagine something
> like this (executed from the directory with AS7/wildfly server):
>
> ./bin/keycloak-import -Dkeycloak.model=jpa
> -Ddata-to-import=/tmp/file-with-full-kc-data-to-import-which-were-previously-exported.json
>
> Assumption is that distribution contains deployed KC inside
> standalone/deployments/auth-server.war. Script will be able to run JVM
> which will have access to needed KC jars on classpath and access to
> persistence.xml . In AS7/Wildfly environment the persistence.xml bundled
> inside auth-server.war is using datasource though, so this JVM process
> would also need to parse datasource from
> standalone/configuration/standalone.xml as it won't be executed in
> managed AS7/Wildfly environment.
>
> c) Use something similar to approach (b) but execute export/import from
> AS7/Wildfly CLI or admin console. The advantage is that it is triggered
> from the managed (AS7 or Wildfly) so has access to server resources like
> datasource referenced from persistence.xml
>

I don't see any reason you couldn't trigger an import from the admin 
console.  Especially if we require a secret used to encrypt the export.


-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From stian at redhat.com  Mon Apr 28 09:07:34 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Mon, 28 Apr 2014 09:07:34 -0400 (EDT)
Subject: [keycloak-dev] Export/import implementation
In-Reply-To: <535E4F00.7020506@redhat.com>
References: <535E02F1.4090107@redhat.com> <535E4F00.7020506@redhat.com>
Message-ID: <1119653537.13077309.1398690454417.JavaMail.zimbra@redhat.com>



----- Original Message -----
> From: "Bill Burke" <bburke at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Monday, 28 April, 2014 1:52:16 PM
> Subject: Re: [keycloak-dev] Export/import implementation
> 
> 
> 
> On 4/28/2014 3:27 AM, Marek Posolda wrote:
> > I am planning to start soon on export/import. If I recall correctly, one
> > of the requirements is to export the content of whole DB content
> > (including IDs and password hashes) to JSON file, which can then be
> > later imported into other DB. This will allow to migrate between
> > environments and various DB types (For example from Mongo to MySQL and
> > viceversa).
> >
> 
> IMO, a full export (of credentials) should require a secret given by the
> admin that will be used to encrypt the export.  The export should only
> be saved locally to disk and not available over the network.
> 
> > I have some question though
> >
> > 1) I assume that DB should be cleared before full import from JSON file?
> > Or do we want to update existing data without deleting the previous
> > content? I assume that this is used for migration, so it's not about
> > updating but completely delete and recreate existing DB, correct?
> >
> > 2) How to implement it. I can see two approaches
> >
> > a) Use model API to retrieve content of the DB into JSON file during
> > export. Similarly during import use model API to sync objects back from
> > JSON into model DB.
> >
> > b) Add some methods to KeycloakSession interface like:
> >
> > ObjectNode export();
> >
> > void import(ObjectNode node);
> >
> > and implement export/import separately for each model.
> >
> > Approach (b) might be better for performance as it allows to directly
> > use low-level queries specific to JPA, Mongo or other model
> > implementations to export/import stuff more effectively in batch,
> > however it will require changes in model implementations and probably
> > adding more stuff into dependencies. So I am more convinced to use (a).
> > Thoughts?
> >
> 
> "a", IMO.  Easier to maintain.
> 
> >
> > 3) How will be export/import triggered?
> >
> > I think that for security reasons, we want to always export into local
> > file with KC server and similarly always import from local file. Is it
> > correct? I can see approaches like:
> >
> 
> You can already import a full json description from the admin console.
> 
> > a) Use KC admin console. By default, just "admin" will be able to
> > export/import stuff . Data will be always exported/imported into JSON
> > file local to server. So it will be possible to trigger export/import
> > remotely from admin console, but just use local JSON file. The import
> > would be tricky though as import will recreate all data (including admin
> > realm and admin user) so it would need to cancel logout sessions
> > including the session of admin user who triggered import.
> >
> > b) Use some script, which will trigger JVM process for export/import
> > data. Script can be executed locally from CMD. I can imagine something
> > like this (executed from the directory with AS7/wildfly server):
> >
> > ./bin/keycloak-import -Dkeycloak.model=jpa
> > -Ddata-to-import=/tmp/file-with-full-kc-data-to-import-which-were-previously-exported.json
> >
> > Assumption is that distribution contains deployed KC inside
> > standalone/deployments/auth-server.war. Script will be able to run JVM
> > which will have access to needed KC jars on classpath and access to
> > persistence.xml . In AS7/Wildfly environment the persistence.xml bundled
> > inside auth-server.war is using datasource though, so this JVM process
> > would also need to parse datasource from
> > standalone/configuration/standalone.xml as it won't be executed in
> > managed AS7/Wildfly environment.
> >
> > c) Use something similar to approach (b) but execute export/import from
> > AS7/Wildfly CLI or admin console. The advantage is that it is triggered
> > from the managed (AS7 or Wildfly) so has access to server resources like
> > datasource referenced from persistence.xml
> >
> 
> I don't see any reason you couldn't trigger an import from the admin
> console.  Especially if we require a secret used to encrypt the export.

The problem is that the export would also contain the admin realm itself

> 
> 
> --
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> 

From bruno at abstractj.org  Mon Apr 28 09:13:50 2014
From: bruno at abstractj.org (Bruno Oliveira)
Date: Mon, 28 Apr 2014 10:13:50 -0300
Subject: [keycloak-dev] Export/import implementation
In-Reply-To: <535E4F00.7020506@redhat.com>
References: <535E02F1.4090107@redhat.com>
 <535E4F00.7020506@redhat.com>
Message-ID: <20140428131350.GB24149@abstractj.org>

On 2014-04-28, Bill Burke wrote:
> 
> 
> On 4/28/2014 3:27 AM, Marek Posolda wrote:
> > I am planning to start soon on export/import. If I recall correctly, one
> > of the requirements is to export the content of whole DB content
> > (including IDs and password hashes) to JSON file, which can then be
> > later imported into other DB. This will allow to migrate between
> > environments and various DB types (For example from Mongo to MySQL and
> > viceversa).
> >
> 
> IMO, a full export (of credentials) should require a secret given by the 
> admin that will be used to encrypt the export.  The export should only 
> be saved locally to disk and not available over the network.

Maybe we could make use of the KDF function already on Keycloak to
encrypt file? Currently as far as I recall we already use it to validate
passwords, so based on the admin's password we generate the private key
to encrypt/decrypt this file.

> 
> > I have some question though
> >
> > 1) I assume that DB should be cleared before full import from JSON file?
> > Or do we want to update existing data without deleting the previous
> > content? I assume that this is used for migration, so it's not about
> > updating but completely delete and recreate existing DB, correct?
> >
> > 2) How to implement it. I can see two approaches
> >
> > a) Use model API to retrieve content of the DB into JSON file during
> > export. Similarly during import use model API to sync objects back from
> > JSON into model DB.
> >
> > b) Add some methods to KeycloakSession interface like:
> >
> > ObjectNode export();
> >
> > void import(ObjectNode node);
> >
> > and implement export/import separately for each model.
> >
> > Approach (b) might be better for performance as it allows to directly
> > use low-level queries specific to JPA, Mongo or other model
> > implementations to export/import stuff more effectively in batch,
> > however it will require changes in model implementations and probably
> > adding more stuff into dependencies. So I am more convinced to use (a).
> > Thoughts?
> >
> 
> "a", IMO.  Easier to maintain.

+1

> 
> -- 
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

-- 

abstractj

From mposolda at redhat.com  Mon Apr 28 11:43:24 2014
From: mposolda at redhat.com (Marek Posolda)
Date: Mon, 28 Apr 2014 17:43:24 +0200
Subject: [keycloak-dev] Export/import implementation
In-Reply-To: <535E4F00.7020506@redhat.com>
References: <535E02F1.4090107@redhat.com> <535E4F00.7020506@redhat.com>
Message-ID: <535E771C.1040309@redhat.com>

On 28.4.2014 14:52, Bill Burke wrote:
>
> On 4/28/2014 3:27 AM, Marek Posolda wrote:
>> I am planning to start soon on export/import. If I recall correctly, one
>> of the requirements is to export the content of whole DB content
>> (including IDs and password hashes) to JSON file, which can then be
>> later imported into other DB. This will allow to migrate between
>> environments and various DB types (For example from Mongo to MySQL and
>> viceversa).
>>
> IMO, a full export (of credentials) should require a secret given by the
> admin that will be used to encrypt the export.  The export should only
> be saved locally to disk and not available over the network.
Yes, so the exported JSON file will be always saved just to local 
filesystem with KC server and never sent over the network. With respect 
to this, I am not sure if securing the JSON file is big issue as the 
file is anyway available just to someone with physical access to the 
machine with KC server? However for better protection and ensuring that 
nobody can read it, I will encrypt the content of this file, likely with 
KDF as proposed by Bruno.
>
>> I have some question though
>>
>> 1) I assume that DB should be cleared before full import from JSON file?
>> Or do we want to update existing data without deleting the previous
>> content? I assume that this is used for migration, so it's not about
>> updating but completely delete and recreate existing DB, correct?
>>
>> 2) How to implement it. I can see two approaches
>>
>> a) Use model API to retrieve content of the DB into JSON file during
>> export. Similarly during import use model API to sync objects back from
>> JSON into model DB.
>>
>> b) Add some methods to KeycloakSession interface like:
>>
>> ObjectNode export();
>>
>> void import(ObjectNode node);
>>
>> and implement export/import separately for each model.
>>
>> Approach (b) might be better for performance as it allows to directly
>> use low-level queries specific to JPA, Mongo or other model
>> implementations to export/import stuff more effectively in batch,
>> however it will require changes in model implementations and probably
>> adding more stuff into dependencies. So I am more convinced to use (a).
>> Thoughts?
>>
> "a", IMO.  Easier to maintain.
oops, now thinking again that current model API doesn't allow to 
retrieve password hash or save password hash into DB? It allows to 
validate plain-text password or save plain-text password but there is no 
support for load or save directly the value of password hash. I wonder 
if it's ok to add some methods to RealmModel, which will allow to 
load/save password hashes?

If you don't want this, I will maybe need to use low-level stuff and 
stick with option (b) .
>
>> 3) How will be export/import triggered?
>>
>> I think that for security reasons, we want to always export into local
>> file with KC server and similarly always import from local file. Is it
>> correct? I can see approaches like:
>>
> You can already import a full json description from the admin console.
>
>> a) Use KC admin console. By default, just "admin" will be able to
>> export/import stuff . Data will be always exported/imported into JSON
>> file local to server. So it will be possible to trigger export/import
>> remotely from admin console, but just use local JSON file. The import
>> would be tricky though as import will recreate all data (including admin
>> realm and admin user) so it would need to cancel logout sessions
>> including the session of admin user who triggered import.
>>
>> b) Use some script, which will trigger JVM process for export/import
>> data. Script can be executed locally from CMD. I can imagine something
>> like this (executed from the directory with AS7/wildfly server):
>>
>> ./bin/keycloak-import -Dkeycloak.model=jpa
>> -Ddata-to-import=/tmp/file-with-full-kc-data-to-import-which-were-previously-exported.json
>>
>> Assumption is that distribution contains deployed KC inside
>> standalone/deployments/auth-server.war. Script will be able to run JVM
>> which will have access to needed KC jars on classpath and access to
>> persistence.xml . In AS7/Wildfly environment the persistence.xml bundled
>> inside auth-server.war is using datasource though, so this JVM process
>> would also need to parse datasource from
>> standalone/configuration/standalone.xml as it won't be executed in
>> managed AS7/Wildfly environment.
>>
>> c) Use something similar to approach (b) but execute export/import from
>> AS7/Wildfly CLI or admin console. The advantage is that it is triggered
>> from the managed (AS7 or Wildfly) so has access to server resources like
>> datasource referenced from persistence.xml
>>
> I don't see any reason you couldn't trigger an import from the admin
> console.  Especially if we require a secret used to encrypt the export.
>
>
Sure, I can. However as already mentioned, all the KC data (including 
admin realm and current admin user who is triggering import) are going 
to be deleted and re-imported from JSON file, so there are many things, 
which need to be addressed if we want to support it on the "online" 
system like:
- Filter, which will pause all requests until export/import is done
- All existing states in memory (TokenManager, SocialRequestManager) 
needs to be cleared after import
- All existing sessions should be logged out (including the session of 
current admin user) after import

There might be more issues related to this and user experience might not 
be good. So it seems to be much easier to support importing data either 
on KC startup as proposed by Stian or at "offline" time when KC server 
is not running at all, which can be done with script?

Exporting data from KC DB into JSON file seems to be safer, but there is 
still a need to pause the server with HTTP or JAX-RS filter, as if users 
are allowed to change data during export (like changing password or 
changing their emails etc), it might result in broken exported data.

I don't know exactly how are production systems deal with 
backup/migration of data, don't have much experience with this. But I 
assume that administrators are usually not doing it on the "online" data.

Marek

From bburke at redhat.com  Tue Apr 29 12:15:27 2014
From: bburke at redhat.com (Bill Burke)
Date: Tue, 29 Apr 2014 12:15:27 -0400
Subject: [keycloak-dev] isolate picketlink dependency please
Message-ID: <535FD01F.7080308@redhat.com>

Mongo model project seems to have picketlink dependencies:

org.picketlink.common

These need to be isolated and removed as a dependency.  Since we may be 
introducing Keycloak into EAP (via Aerogear) we want to be sure we can 
remove any version conflicting picketlink dependencies.  So, anything 
picketlink related has to be behind a plugglable and removable SPI.
-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From bruno at abstractj.org  Tue Apr 29 14:49:23 2014
From: bruno at abstractj.org (Bruno Oliveira)
Date: Tue, 29 Apr 2014 15:49:23 -0300
Subject: [keycloak-dev] isolate picketlink dependency please
In-Reply-To: <535FD01F.7080308@redhat.com>
References: <535FD01F.7080308@redhat.com>
Message-ID: <20140429184923.GA33874@abstractj.org>

+1

On 2014-04-29, Bill Burke wrote:
> Mongo model project seems to have picketlink dependencies:
> 
> org.picketlink.common
> 
> These need to be isolated and removed as a dependency.  Since we may be 
> introducing Keycloak into EAP (via Aerogear) we want to be sure we can 
> remove any version conflicting picketlink dependencies.  So, anything 
> picketlink related has to be behind a plugglable and removable SPI.
> -- 
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

-- 

abstractj

From mposolda at redhat.com  Tue Apr 29 17:59:23 2014
From: mposolda at redhat.com (Marek Posolda)
Date: Tue, 29 Apr 2014 23:59:23 +0200
Subject: [keycloak-dev] isolate picketlink dependency please
In-Reply-To: <535FD01F.7080308@redhat.com>
References: <535FD01F.7080308@redhat.com>
Message-ID: <536020BB.5030807@redhat.com>

Mongo model is using just some helper reflection classes from 
org.picketlink.common. It should be easy to fork some functionality and 
completely remove dependency on org.picketlink.common from mongo model.

However picketlink is also used for Ldap integration and here it's more 
complicated...

So what exactly is the requirement for picketlink integration? Am I 
understand correctly that all picketlink dependencies must be removed 
from auth-server.war/WEB-INF/lib/ and added as deps to 
auth-server.war/WEB-INF/jboss-deployment-structure.xml instead?

If I understand correctly, this means that Keycloak must use same 
Picketlink version, which is bundled with EAP. Do you know what is our 
target EAP version and which version of Picketlink is in it?

Today I've upgraded Keycloak to newly released Picketlink 2.6.0.CR2, 
which contains some nice LDAP improvements and fixes (like support for 
RHDS and connection pooling). So it seems that I will need to revert 
this and use some older picketlink version bundled in EAP instead:-(

Marek

On 29.4.2014 18:15, Bill Burke wrote:
> Mongo model project seems to have picketlink dependencies:
>
> org.picketlink.common
>
> These need to be isolated and removed as a dependency.  Since we may be
> introducing Keycloak into EAP (via Aerogear) we want to be sure we can
> remove any version conflicting picketlink dependencies.  So, anything
> picketlink related has to be behind a plugglable and removable SPI.


From stian at redhat.com  Wed Apr 30 03:43:44 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Wed, 30 Apr 2014 03:43:44 -0400 (EDT)
Subject: [keycloak-dev] isolate picketlink dependency please
In-Reply-To: <536020BB.5030807@redhat.com>
References: <535FD01F.7080308@redhat.com> <536020BB.5030807@redhat.com>
Message-ID: <1352845995.14705733.1398843824867.JavaMail.zimbra@redhat.com>

AeroGear will use a stripped-down version of Keycloak WAR, without mongo, ldap, social, etc. so this won't be an issue for them, but I agree that we should remove this dependency from the Mongo model though.

I don't see a problem with us using the latest version of PicketLink as long as only authentication-picketlink depends on it.

----- Original Message -----
> From: "Marek Posolda" <mposolda at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Tuesday, 29 April, 2014 10:59:23 PM
> Subject: Re: [keycloak-dev] isolate picketlink dependency please
> 
> Mongo model is using just some helper reflection classes from
> org.picketlink.common. It should be easy to fork some functionality and
> completely remove dependency on org.picketlink.common from mongo model.
> 
> However picketlink is also used for Ldap integration and here it's more
> complicated...
> 
> So what exactly is the requirement for picketlink integration? Am I
> understand correctly that all picketlink dependencies must be removed
> from auth-server.war/WEB-INF/lib/ and added as deps to
> auth-server.war/WEB-INF/jboss-deployment-structure.xml instead?
> 
> If I understand correctly, this means that Keycloak must use same
> Picketlink version, which is bundled with EAP. Do you know what is our
> target EAP version and which version of Picketlink is in it?
> 
> Today I've upgraded Keycloak to newly released Picketlink 2.6.0.CR2,
> which contains some nice LDAP improvements and fixes (like support for
> RHDS and connection pooling). So it seems that I will need to revert
> this and use some older picketlink version bundled in EAP instead:-(
> 
> Marek
> 
> On 29.4.2014 18:15, Bill Burke wrote:
> > Mongo model project seems to have picketlink dependencies:
> >
> > org.picketlink.common
> >
> > These need to be isolated and removed as a dependency.  Since we may be
> > introducing Keycloak into EAP (via Aerogear) we want to be sure we can
> > remove any version conflicting picketlink dependencies.  So, anything
> > picketlink related has to be behind a plugglable and removable SPI.
> 
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> 

From stian at redhat.com  Wed Apr 30 04:44:31 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Wed, 30 Apr 2014 04:44:31 -0400 (EDT)
Subject: [keycloak-dev] isolate picketlink dependency please
In-Reply-To: <5360B496.8070102@redhat.com>
References: <535FD01F.7080308@redhat.com> <536020BB.5030807@redhat.com>
	<1352845995.14705733.1398843824867.JavaMail.zimbra@redhat.com>
	<5360B496.8070102@redhat.com>
Message-ID: <1534140431.14738896.1398847471640.JavaMail.zimbra@redhat.com>

It may be in the future, if we want to support all/most features on EAP, but I don't think we do now.

Bill: wdyt?

----- Original Message -----
> From: "Marek Posolda" <mposolda at redhat.com>
> To: "Stian Thorgersen" <stian at redhat.com>
> Cc: keycloak-dev at lists.jboss.org
> Sent: Wednesday, 30 April, 2014 9:30:14 AM
> Subject: Re: [keycloak-dev] isolate picketlink dependency please
> 
> Ok, I will remove the dependency from the mongo model, that's an easy
> part though.
> 
> So the fact that we actually bundle latest picketlink jars inside
> Keycloak WAR in auth-server.war/WEB-INF/lib/ is not an issue?
> 
> Marek
> 
> On 30.4.2014 09:43, Stian Thorgersen wrote:
> > AeroGear will use a stripped-down version of Keycloak WAR, without mongo,
> > ldap, social, etc. so this won't be an issue for them, but I agree that we
> > should remove this dependency from the Mongo model though.
> >
> > I don't see a problem with us using the latest version of PicketLink as
> > long as only authentication-picketlink depends on it.
> >
> > ----- Original Message -----
> >> From: "Marek Posolda" <mposolda at redhat.com>
> >> To: keycloak-dev at lists.jboss.org
> >> Sent: Tuesday, 29 April, 2014 10:59:23 PM
> >> Subject: Re: [keycloak-dev] isolate picketlink dependency please
> >>
> >> Mongo model is using just some helper reflection classes from
> >> org.picketlink.common. It should be easy to fork some functionality and
> >> completely remove dependency on org.picketlink.common from mongo model.
> >>
> >> However picketlink is also used for Ldap integration and here it's more
> >> complicated...
> >>
> >> So what exactly is the requirement for picketlink integration? Am I
> >> understand correctly that all picketlink dependencies must be removed
> >> from auth-server.war/WEB-INF/lib/ and added as deps to
> >> auth-server.war/WEB-INF/jboss-deployment-structure.xml instead?
> >>
> >> If I understand correctly, this means that Keycloak must use same
> >> Picketlink version, which is bundled with EAP. Do you know what is our
> >> target EAP version and which version of Picketlink is in it?
> >>
> >> Today I've upgraded Keycloak to newly released Picketlink 2.6.0.CR2,
> >> which contains some nice LDAP improvements and fixes (like support for
> >> RHDS and connection pooling). So it seems that I will need to revert
> >> this and use some older picketlink version bundled in EAP instead:-(
> >>
> >> Marek
> >>
> >> On 29.4.2014 18:15, Bill Burke wrote:
> >>> Mongo model project seems to have picketlink dependencies:
> >>>
> >>> org.picketlink.common
> >>>
> >>> These need to be isolated and removed as a dependency.  Since we may be
> >>> introducing Keycloak into EAP (via Aerogear) we want to be sure we can
> >>> remove any version conflicting picketlink dependencies.  So, anything
> >>> picketlink related has to be behind a plugglable and removable SPI.
> >> _______________________________________________
> >> keycloak-dev mailing list
> >> keycloak-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>
> 
> 

From mposolda at redhat.com  Wed Apr 30 04:30:14 2014
From: mposolda at redhat.com (Marek Posolda)
Date: Wed, 30 Apr 2014 10:30:14 +0200
Subject: [keycloak-dev] isolate picketlink dependency please
In-Reply-To: <1352845995.14705733.1398843824867.JavaMail.zimbra@redhat.com>
References: <535FD01F.7080308@redhat.com> <536020BB.5030807@redhat.com>
	<1352845995.14705733.1398843824867.JavaMail.zimbra@redhat.com>
Message-ID: <5360B496.8070102@redhat.com>

Ok, I will remove the dependency from the mongo model, that's an easy 
part though.

So the fact that we actually bundle latest picketlink jars inside 
Keycloak WAR in auth-server.war/WEB-INF/lib/ is not an issue?

Marek

On 30.4.2014 09:43, Stian Thorgersen wrote:
> AeroGear will use a stripped-down version of Keycloak WAR, without mongo, ldap, social, etc. so this won't be an issue for them, but I agree that we should remove this dependency from the Mongo model though.
>
> I don't see a problem with us using the latest version of PicketLink as long as only authentication-picketlink depends on it.
>
> ----- Original Message -----
>> From: "Marek Posolda" <mposolda at redhat.com>
>> To: keycloak-dev at lists.jboss.org
>> Sent: Tuesday, 29 April, 2014 10:59:23 PM
>> Subject: Re: [keycloak-dev] isolate picketlink dependency please
>>
>> Mongo model is using just some helper reflection classes from
>> org.picketlink.common. It should be easy to fork some functionality and
>> completely remove dependency on org.picketlink.common from mongo model.
>>
>> However picketlink is also used for Ldap integration and here it's more
>> complicated...
>>
>> So what exactly is the requirement for picketlink integration? Am I
>> understand correctly that all picketlink dependencies must be removed
>> from auth-server.war/WEB-INF/lib/ and added as deps to
>> auth-server.war/WEB-INF/jboss-deployment-structure.xml instead?
>>
>> If I understand correctly, this means that Keycloak must use same
>> Picketlink version, which is bundled with EAP. Do you know what is our
>> target EAP version and which version of Picketlink is in it?
>>
>> Today I've upgraded Keycloak to newly released Picketlink 2.6.0.CR2,
>> which contains some nice LDAP improvements and fixes (like support for
>> RHDS and connection pooling). So it seems that I will need to revert
>> this and use some older picketlink version bundled in EAP instead:-(
>>
>> Marek
>>
>> On 29.4.2014 18:15, Bill Burke wrote:
>>> Mongo model project seems to have picketlink dependencies:
>>>
>>> org.picketlink.common
>>>
>>> These need to be isolated and removed as a dependency.  Since we may be
>>> introducing Keycloak into EAP (via Aerogear) we want to be sure we can
>>> remove any version conflicting picketlink dependencies.  So, anything
>>> picketlink related has to be behind a plugglable and removable SPI.
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>


From bburke at redhat.com  Wed Apr 30 09:48:35 2014
From: bburke at redhat.com (Bill Burke)
Date: Wed, 30 Apr 2014 09:48:35 -0400
Subject: [keycloak-dev] isolate picketlink dependency please
In-Reply-To: <1534140431.14738896.1398847471640.JavaMail.zimbra@redhat.com>
References: <535FD01F.7080308@redhat.com>
	<536020BB.5030807@redhat.com>	<1352845995.14705733.1398843824867.JavaMail.zimbra@redhat.com>	<5360B496.8070102@redhat.com>
	<1534140431.14738896.1398847471640.JavaMail.zimbra@redhat.com>
Message-ID: <5360FF33.10604@redhat.com>

Primary Keycloak code should not depend on Picketlink.  Picketlink 
should always be hidden by SPIs.  So, if we need to provide LDAP support 
on EAP using an older version of Picketlink, then we write a separate 
maven module using that older version of Picketlink and plug it in.

Following me?

Right now, it looks that only the Mongo data model has a PL dependency. 
  Correct?

On 4/30/2014 4:44 AM, Stian Thorgersen wrote:
> It may be in the future, if we want to support all/most features on EAP, but I don't think we do now.
>
> Bill: wdyt?
>
> ----- Original Message -----
>> From: "Marek Posolda" <mposolda at redhat.com>
>> To: "Stian Thorgersen" <stian at redhat.com>
>> Cc: keycloak-dev at lists.jboss.org
>> Sent: Wednesday, 30 April, 2014 9:30:14 AM
>> Subject: Re: [keycloak-dev] isolate picketlink dependency please
>>
>> Ok, I will remove the dependency from the mongo model, that's an easy
>> part though.
>>
>> So the fact that we actually bundle latest picketlink jars inside
>> Keycloak WAR in auth-server.war/WEB-INF/lib/ is not an issue?
>>
>> Marek
>>
>> On 30.4.2014 09:43, Stian Thorgersen wrote:
>>> AeroGear will use a stripped-down version of Keycloak WAR, without mongo,
>>> ldap, social, etc. so this won't be an issue for them, but I agree that we
>>> should remove this dependency from the Mongo model though.
>>>
>>> I don't see a problem with us using the latest version of PicketLink as
>>> long as only authentication-picketlink depends on it.
>>>
>>> ----- Original Message -----
>>>> From: "Marek Posolda" <mposolda at redhat.com>
>>>> To: keycloak-dev at lists.jboss.org
>>>> Sent: Tuesday, 29 April, 2014 10:59:23 PM
>>>> Subject: Re: [keycloak-dev] isolate picketlink dependency please
>>>>
>>>> Mongo model is using just some helper reflection classes from
>>>> org.picketlink.common. It should be easy to fork some functionality and
>>>> completely remove dependency on org.picketlink.common from mongo model.
>>>>
>>>> However picketlink is also used for Ldap integration and here it's more
>>>> complicated...
>>>>
>>>> So what exactly is the requirement for picketlink integration? Am I
>>>> understand correctly that all picketlink dependencies must be removed
>>>> from auth-server.war/WEB-INF/lib/ and added as deps to
>>>> auth-server.war/WEB-INF/jboss-deployment-structure.xml instead?
>>>>
>>>> If I understand correctly, this means that Keycloak must use same
>>>> Picketlink version, which is bundled with EAP. Do you know what is our
>>>> target EAP version and which version of Picketlink is in it?
>>>>
>>>> Today I've upgraded Keycloak to newly released Picketlink 2.6.0.CR2,
>>>> which contains some nice LDAP improvements and fixes (like support for
>>>> RHDS and connection pooling). So it seems that I will need to revert
>>>> this and use some older picketlink version bundled in EAP instead:-(
>>>>
>>>> Marek
>>>>
>>>> On 29.4.2014 18:15, Bill Burke wrote:
>>>>> Mongo model project seems to have picketlink dependencies:
>>>>>
>>>>> org.picketlink.common
>>>>>
>>>>> These need to be isolated and removed as a dependency.  Since we may be
>>>>> introducing Keycloak into EAP (via Aerogear) we want to be sure we can
>>>>> remove any version conflicting picketlink dependencies.  So, anything
>>>>> picketlink related has to be behind a plugglable and removable SPI.
>>>> _______________________________________________
>>>> keycloak-dev mailing list
>>>> keycloak-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>
>>
>>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From stian at redhat.com  Wed Apr 30 10:14:17 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Wed, 30 Apr 2014 10:14:17 -0400 (EDT)
Subject: [keycloak-dev] isolate picketlink dependency please
In-Reply-To: <5360FF33.10604@redhat.com>
References: <535FD01F.7080308@redhat.com> <536020BB.5030807@redhat.com>
	<1352845995.14705733.1398843824867.JavaMail.zimbra@redhat.com>
	<5360B496.8070102@redhat.com>
	<1534140431.14738896.1398847471640.JavaMail.zimbra@redhat.com>
	<5360FF33.10604@redhat.com>
Message-ID: <1169670447.15096483.1398867257802.JavaMail.zimbra@redhat.com>



----- Original Message -----
> From: "Bill Burke" <bburke at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Wednesday, 30 April, 2014 2:48:35 PM
> Subject: Re: [keycloak-dev] isolate picketlink dependency please
> 
> Primary Keycloak code should not depend on Picketlink.  Picketlink
> should always be hidden by SPIs.  So, if we need to provide LDAP support
> on EAP using an older version of Picketlink, then we write a separate
> maven module using that older version of Picketlink and plug it in.
> 
> Following me?

Yep

> 
> Right now, it looks that only the Mongo data model has a PL dependency.
>   Correct?

Yes (except authentication/authentication-picketlink of course)

> 
> On 4/30/2014 4:44 AM, Stian Thorgersen wrote:
> > It may be in the future, if we want to support all/most features on EAP,
> > but I don't think we do now.
> >
> > Bill: wdyt?
> >
> > ----- Original Message -----
> >> From: "Marek Posolda" <mposolda at redhat.com>
> >> To: "Stian Thorgersen" <stian at redhat.com>
> >> Cc: keycloak-dev at lists.jboss.org
> >> Sent: Wednesday, 30 April, 2014 9:30:14 AM
> >> Subject: Re: [keycloak-dev] isolate picketlink dependency please
> >>
> >> Ok, I will remove the dependency from the mongo model, that's an easy
> >> part though.
> >>
> >> So the fact that we actually bundle latest picketlink jars inside
> >> Keycloak WAR in auth-server.war/WEB-INF/lib/ is not an issue?
> >>
> >> Marek
> >>
> >> On 30.4.2014 09:43, Stian Thorgersen wrote:
> >>> AeroGear will use a stripped-down version of Keycloak WAR, without mongo,
> >>> ldap, social, etc. so this won't be an issue for them, but I agree that
> >>> we
> >>> should remove this dependency from the Mongo model though.
> >>>
> >>> I don't see a problem with us using the latest version of PicketLink as
> >>> long as only authentication-picketlink depends on it.
> >>>
> >>> ----- Original Message -----
> >>>> From: "Marek Posolda" <mposolda at redhat.com>
> >>>> To: keycloak-dev at lists.jboss.org
> >>>> Sent: Tuesday, 29 April, 2014 10:59:23 PM
> >>>> Subject: Re: [keycloak-dev] isolate picketlink dependency please
> >>>>
> >>>> Mongo model is using just some helper reflection classes from
> >>>> org.picketlink.common. It should be easy to fork some functionality and
> >>>> completely remove dependency on org.picketlink.common from mongo model.
> >>>>
> >>>> However picketlink is also used for Ldap integration and here it's more
> >>>> complicated...
> >>>>
> >>>> So what exactly is the requirement for picketlink integration? Am I
> >>>> understand correctly that all picketlink dependencies must be removed
> >>>> from auth-server.war/WEB-INF/lib/ and added as deps to
> >>>> auth-server.war/WEB-INF/jboss-deployment-structure.xml instead?
> >>>>
> >>>> If I understand correctly, this means that Keycloak must use same
> >>>> Picketlink version, which is bundled with EAP. Do you know what is our
> >>>> target EAP version and which version of Picketlink is in it?
> >>>>
> >>>> Today I've upgraded Keycloak to newly released Picketlink 2.6.0.CR2,
> >>>> which contains some nice LDAP improvements and fixes (like support for
> >>>> RHDS and connection pooling). So it seems that I will need to revert
> >>>> this and use some older picketlink version bundled in EAP instead:-(
> >>>>
> >>>> Marek
> >>>>
> >>>> On 29.4.2014 18:15, Bill Burke wrote:
> >>>>> Mongo model project seems to have picketlink dependencies:
> >>>>>
> >>>>> org.picketlink.common
> >>>>>
> >>>>> These need to be isolated and removed as a dependency.  Since we may be
> >>>>> introducing Keycloak into EAP (via Aerogear) we want to be sure we can
> >>>>> remove any version conflicting picketlink dependencies.  So, anything
> >>>>> picketlink related has to be behind a plugglable and removable SPI.
> >>>> _______________________________________________
> >>>> keycloak-dev mailing list
> >>>> keycloak-dev at lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>>>
> >>
> >>
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
> 
> --
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> 

From mposolda at redhat.com  Wed Apr 30 11:52:33 2014
From: mposolda at redhat.com (Marek Posolda)
Date: Wed, 30 Apr 2014 17:52:33 +0200
Subject: [keycloak-dev] isolate picketlink dependency please
In-Reply-To: <1169670447.15096483.1398867257802.JavaMail.zimbra@redhat.com>
References: <535FD01F.7080308@redhat.com>
	<536020BB.5030807@redhat.com>	<1352845995.14705733.1398843824867.JavaMail.zimbra@redhat.com>	<5360B496.8070102@redhat.com>	<1534140431.14738896.1398847471640.JavaMail.zimbra@redhat.com>	<5360FF33.10604@redhat.com>
	<1169670447.15096483.1398867257802.JavaMail.zimbra@redhat.com>
Message-ID: <53611C41.6050908@redhat.com>

On 30.4.2014 16:14, Stian Thorgersen wrote:
>
> ----- Original Message -----
>> From: "Bill Burke" <bburke at redhat.com>
>> To: keycloak-dev at lists.jboss.org
>> Sent: Wednesday, 30 April, 2014 2:48:35 PM
>> Subject: Re: [keycloak-dev] isolate picketlink dependency please
>>
>> Primary Keycloak code should not depend on Picketlink.  Picketlink
>> should always be hidden by SPIs.  So, if we need to provide LDAP support
>> on EAP using an older version of Picketlink, then we write a separate
>> maven module using that older version of Picketlink and plug it in.
>>
>> Following me?
> Yep
>
>> Right now, it looks that only the Mongo data model has a PL dependency.
>>    Correct?
> Yes (except authentication/authentication-picketlink of course)
Ok, I did not know that using picketlink is so big headache. Generally 
said, it seems that if your project want to run on EAP, it's much easier 
to depend on some 3rd party library, which can be bundled directly in 
your WAR instead on "jboss" projects, which are available as modules in 
EAP...:-(

My idea was that picketlink IDM will be used for LDAP integration and 
leveraged by both "authentication" and "sync" SPIs. So I've also added 
"keycloak-picketlink-api", which adds IdentityManagerProvider interface 
and is itself the SPI module. It's used by authentication-picketlink and 
the plan was to use it also in sync-picketlink .

So IdentityManagerProvider, which has direct dependency on 
picketlink-idm-api is itself the SPI module and it's referenced from 
KeycloakApplication: 
https://github.com/keycloak/keycloak/blob/master/services/src/main/java/org/keycloak/services/resources/KeycloakApplication.java#L135 
hence removing picketlink idm JARS is causing NoClassDefFoundErrors now. 
So it seems that I will also need to refactor this, so that there is no 
dependency on picketlink from some SPI modules, but just from "SPI 
implementation" modules. Correct?

Marek
>
>> On 4/30/2014 4:44 AM, Stian Thorgersen wrote:
>>> It may be in the future, if we want to support all/most features on EAP,
>>> but I don't think we do now.
>>>
>>> Bill: wdyt?
>>>
>>> ----- Original Message -----
>>>> From: "Marek Posolda" <mposolda at redhat.com>
>>>> To: "Stian Thorgersen" <stian at redhat.com>
>>>> Cc: keycloak-dev at lists.jboss.org
>>>> Sent: Wednesday, 30 April, 2014 9:30:14 AM
>>>> Subject: Re: [keycloak-dev] isolate picketlink dependency please
>>>>
>>>> Ok, I will remove the dependency from the mongo model, that's an easy
>>>> part though.
>>>>
>>>> So the fact that we actually bundle latest picketlink jars inside
>>>> Keycloak WAR in auth-server.war/WEB-INF/lib/ is not an issue?
>>>>
>>>> Marek
>>>>
>>>> On 30.4.2014 09:43, Stian Thorgersen wrote:
>>>>> AeroGear will use a stripped-down version of Keycloak WAR, without mongo,
>>>>> ldap, social, etc. so this won't be an issue for them, but I agree that
>>>>> we
>>>>> should remove this dependency from the Mongo model though.
>>>>>
>>>>> I don't see a problem with us using the latest version of PicketLink as
>>>>> long as only authentication-picketlink depends on it.
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Marek Posolda" <mposolda at redhat.com>
>>>>>> To: keycloak-dev at lists.jboss.org
>>>>>> Sent: Tuesday, 29 April, 2014 10:59:23 PM
>>>>>> Subject: Re: [keycloak-dev] isolate picketlink dependency please
>>>>>>
>>>>>> Mongo model is using just some helper reflection classes from
>>>>>> org.picketlink.common. It should be easy to fork some functionality and
>>>>>> completely remove dependency on org.picketlink.common from mongo model.
>>>>>>
>>>>>> However picketlink is also used for Ldap integration and here it's more
>>>>>> complicated...
>>>>>>
>>>>>> So what exactly is the requirement for picketlink integration? Am I
>>>>>> understand correctly that all picketlink dependencies must be removed
>>>>>> from auth-server.war/WEB-INF/lib/ and added as deps to
>>>>>> auth-server.war/WEB-INF/jboss-deployment-structure.xml instead?
>>>>>>
>>>>>> If I understand correctly, this means that Keycloak must use same
>>>>>> Picketlink version, which is bundled with EAP. Do you know what is our
>>>>>> target EAP version and which version of Picketlink is in it?
>>>>>>
>>>>>> Today I've upgraded Keycloak to newly released Picketlink 2.6.0.CR2,
>>>>>> which contains some nice LDAP improvements and fixes (like support for
>>>>>> RHDS and connection pooling). So it seems that I will need to revert
>>>>>> this and use some older picketlink version bundled in EAP instead:-(
>>>>>>
>>>>>> Marek
>>>>>>
>>>>>> On 29.4.2014 18:15, Bill Burke wrote:
>>>>>>> Mongo model project seems to have picketlink dependencies:
>>>>>>>
>>>>>>> org.picketlink.common
>>>>>>>
>>>>>>> These need to be isolated and removed as a dependency.  Since we may be
>>>>>>> introducing Keycloak into EAP (via Aerogear) we want to be sure we can
>>>>>>> remove any version conflicting picketlink dependencies.  So, anything
>>>>>>> picketlink related has to be behind a plugglable and removable SPI.
>>>>>> _______________________________________________
>>>>>> keycloak-dev mailing list
>>>>>> keycloak-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>
>>>>
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>> --
>> Bill Burke
>> JBoss, a division of Red Hat
>> http://bill.burkecentral.com
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev


From bburke at redhat.com  Wed Apr 30 12:15:43 2014
From: bburke at redhat.com (Bill Burke)
Date: Wed, 30 Apr 2014 12:15:43 -0400
Subject: [keycloak-dev] isolate picketlink dependency please
In-Reply-To: <53611C41.6050908@redhat.com>
References: <535FD01F.7080308@redhat.com>	<536020BB.5030807@redhat.com>	<1352845995.14705733.1398843824867.JavaMail.zimbra@redhat.com>	<5360B496.8070102@redhat.com>	<1534140431.14738896.1398847471640.JavaMail.zimbra@redhat.com>	<5360FF33.10604@redhat.com>	<1169670447.15096483.1398867257802.JavaMail.zimbra@redhat.com>
	<53611C41.6050908@redhat.com>
Message-ID: <536121AF.9070703@redhat.com>



On 4/30/2014 11:52 AM, Marek Posolda wrote:
> On 30.4.2014 16:14, Stian Thorgersen wrote:
>>
>> ----- Original Message -----
>>> From: "Bill Burke" <bburke at redhat.com>
>>> To: keycloak-dev at lists.jboss.org
>>> Sent: Wednesday, 30 April, 2014 2:48:35 PM
>>> Subject: Re: [keycloak-dev] isolate picketlink dependency please
>>>
>>> Primary Keycloak code should not depend on Picketlink.  Picketlink
>>> should always be hidden by SPIs.  So, if we need to provide LDAP support
>>> on EAP using an older version of Picketlink, then we write a separate
>>> maven module using that older version of Picketlink and plug it in.
>>>
>>> Following me?
>> Yep
>>
>>> Right now, it looks that only the Mongo data model has a PL dependency.
>>>     Correct?
>> Yes (except authentication/authentication-picketlink of course)
> Ok, I did not know that using picketlink is so big headache. Generally
> said, it seems that if your project want to run on EAP, it's much easier
> to depend on some 3rd party library, which can be bundled directly in
> your WAR instead on "jboss" projects, which are available as modules in
> EAP...:-(
>

It's not that.  Its the whole brew, we-must-build-everything-we-ship, 
way of doing things.



-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From stian at redhat.com  Wed Apr 30 12:17:47 2014
From: stian at redhat.com (Stian Thorgersen)
Date: Wed, 30 Apr 2014 12:17:47 -0400 (EDT)
Subject: [keycloak-dev] Account management requirements for beta1
In-Reply-To: <1235278278.15187461.1398874119862.JavaMail.zimbra@redhat.com>
Message-ID: <556989743.15195729.1398874667210.JavaMail.zimbra@redhat.com>

With regards to account management what additional requirements do we have for beta1?

Features I can think off to add now or in the future includes:

* Manage refresh tokens - view applications and clients that have refresh tokens, and the ability to invalidate specific tokens
* Manage devices - view browsers and devices that have access (remember me cookie?), and the ability to invalidate specific cookies
* Manage devices that can bypass totp - it seems to be quite common that it's possible to not require asking for totp again for a specific device, I assume this is done by setting a cookie, if we enable this it should be possible to view what devices have this option, as well as invalidate them
* Manage applications - view all applications, be able to navigate to an application, and the ability to invalidate access to specific application
* Manage clients - view all clients and what grants they have, and the ability to revoke access to specific client

I think listing client grants, invalidate specific client grants, and a logout everything option would be sufficient. The logout everything option would invalidate any refresh tokens, remember me cookies, 'skip' totp cookies and do a sso-logout.

From bburke at redhat.com  Wed Apr 30 13:25:26 2014
From: bburke at redhat.com (Bill Burke)
Date: Wed, 30 Apr 2014 13:25:26 -0400
Subject: [keycloak-dev] unable to view admin console
Message-ID: <53613206.3080304@redhat.com>

firefox is giving me a download window.  Looking into it after I eat 
unless you have a quick fix.
-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From bburke at redhat.com  Wed Apr 30 14:24:01 2014
From: bburke at redhat.com (Bill Burke)
Date: Wed, 30 Apr 2014 14:24:01 -0400
Subject: [keycloak-dev] Account management requirements for beta1
In-Reply-To: <556989743.15195729.1398874667210.JavaMail.zimbra@redhat.com>
References: <556989743.15195729.1398874667210.JavaMail.zimbra@redhat.com>
Message-ID: <53613FC1.3050006@redhat.com>

We have most of this via a not-before policy you can set at the realm 
level, application, client, or user level.  No ability yet to view 
tokens that have been given out though and which may still be valid. 
Only an admin can set the not-before policy right now.

Tasks:

* Make sure all not before policies are checked before login or refresh
* Set UserModel.notBefore when a user does a logout.
* Allow user to invalidate all grants (sets a UserModel.notBefore(now) 
policy)

Not a priority:
* Allow a user to view and invalidate specific oauth grants.  We can 
just make it all or nothing.  I just think there's higher priority 
things to do.

On 4/30/2014 12:17 PM, Stian Thorgersen wrote:
> With regards to account management what additional requirements do we have for beta1?
>
> Features I can think off to add now or in the future includes:
>
> * Manage refresh tokens - view applications and clients that have refresh tokens, and the ability to invalidate specific tokens
> * Manage devices - view browsers and devices that have access (remember me cookie?), and the ability to invalidate specific cookies
> * Manage devices that can bypass totp - it seems to be quite common that it's possible to not require asking for totp again for a specific device, I assume this is done by setting a cookie, if we enable this it should be possible to view what devices have this option, as well as invalidate them
> * Manage applications - view all applications, be able to navigate to an application, and the ability to invalidate access to specific application
> * Manage clients - view all clients and what grants they have, and the ability to revoke access to specific client
>
> I think listing client grants, invalidate specific client grants, and a logout everything option would be sufficient. The logout everything option would invalidate any refresh tokens, remember me cookies, 'skip' totp cookies and do a sso-logout.
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From bburke at redhat.com  Wed Apr 30 14:28:11 2014
From: bburke at redhat.com (Bill Burke)
Date: Wed, 30 Apr 2014 14:28:11 -0400
Subject: [keycloak-dev] unable to view admin console
In-Reply-To: <53613206.3080304@redhat.com>
References: <53613206.3080304@redhat.com>
Message-ID: <536140BB.40400@redhat.com>

Fixed.  Just had to add text/html to mime.types

On 4/30/2014 1:25 PM, Bill Burke wrote:
> firefox is giving me a download window.  Looking into it after I eat
> unless you have a quick fix.
>

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

From bburke at redhat.com  Wed Apr 30 22:37:46 2014
From: bburke at redhat.com (Bill Burke)
Date: Wed, 30 Apr 2014 22:37:46 -0400
Subject: [keycloak-dev] management problems
Message-ID: <5361B37A.7080203@redhat.com>

Our current "master realm" structure/design is deficient.  Consider an 
application like UPS that wants to use Keycloak to manage users.  This 
application would also have its own management console whose security is 
also managed by keycloak.

My first thought is that you could define the application's management 
console as an application in the "master" keycloak realm.  This solution 
isn't a great one if the keycloak server is managing multiple realms. 
So, IMO not something we should recommend.

Another option is to define admin roles within the application's realm 
itself.  These roles are assignable to users within the realm.  This 
would require rethinking of the Angular JS admin console and how things 
are authenticated and how people log-in.  We should probably treat this 
as SSO and have individual applications within the application realm, 
for example:

UPS Realm registered applications:

realm-management (keycloak admin console)
aerogear-ups-management (ups admin console)





-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com