Re: [keycloak-user] Authentication failed: org.jvnet.libpam.PAMException

Hi, We are currently evaluating keycloak as a possible authentication mechanism deployed to our facility. We use kerberos for user authentication with FreeIPA and configured sssd for user federation in keycloak (follow the official document both from keycloak and freeipa.org) One of the requirement we desire is to enable kerboros password for SSH login and enabled 'otp' for HTTP based applications. To do so, 1. We enabled both user-auth-types for the user: - password - password + otp 2. Created HBAC rules in IPA, allowing keycloak server access for following services: (I purposely did not enable 'otp' at this point as I want to verify both 'password' and 'otp' shall work) - keycloak - sshd 3. Confimred sshd worked with both 'password' and 'otp' types via PAM/SSSD, then I went ahead and accessed URL that is protected by keycloak, 'password' works but 'otp' won't, the following ERRORs were seen in keycloak's server.log: ----------- 019-03-04 17:01:20,246 WARN [org.keycloak.events] (default task-22) type=LOGIN_ERROR, realmId=SDCC, clientId=vproxytest03, userId=9900928d-efee-4192-bbc8-7e29cf512d2b, ipAddress=130.199.6.120, error=invalid_user_credentials, auth_method=openid-connect, auth_type=code, redirect_uri=https://www.example.com/secure/ <https://vproxytest03.racf.bnl.gov/secure/>*, code_id=d6c83411-4ca8-4d2b-b942-afd0006e98d2, username=mmstestu 2019-03-04 17:01:43,033 ERROR [org.keycloak.federation.sssd.impl.PAMAuthenticator] (default task-22) Authentication failed: org.jvnet.libpam.PAMException: pam_authenticate failed : Permission denied at org.jvnet.libpam.PAM.check(PAM.java:113) at org.jvnet.libpam.PAM.authenticate(PAM.java:129) at org.keycloak.federation.sssd.impl.PAMAuthenticator.authenticate(PAMAuthenticator.java:53) at org.keycloak.federation.sssd.SSSDFederationProvider.isValid(SSSDFederationProvider.java:180) at org.keycloak.credential.UserCredentialStoreManager.validate(UserCredentialStoreManager.java:143) at org.keycloak.credential.UserCredentialStoreManager.isValid(UserCredentialStoreManager.java:124) at org.keycloak.authentication.authenticators.browser.AbstractUsernameFormAuthenticator.validatePassword(AbstractUsernameFormAuthenticator.java:193) at org.keycloak.authentication.authenticators.browser.AbstractUsernameFormAuthenticator.validateUserAndPassword(AbstractUsernameFormAuthenticator.java:166) at org.keycloak.authentication.authenticators.browser.UsernamePasswordForm.validateForm(UsernamePasswordForm.java:55) at org.keycloak.authentication.authenticators.browser.UsernamePasswordForm.action(UsernamePasswordForm.java:48) at org.keycloak.authentication.DefaultAuthenticationFlow.processAction(DefaultAuthenticationFlow.java:113) at org.keycloak.authentication.DefaultAuthenticationFlow.processAction(DefaultAuthenticationFlow.java:97) at org.keycloak.authentication.AuthenticationProcessor.authenticationAction(AuthenticationProcessor.java:873) at org.keycloak.services.resources.LoginActionsService.processFlow(LoginActionsService.java:292) at org.keycloak.services.resources.LoginActionsService.processAuthentication(LoginActionsService.java:263) at org.keycloak.services.resources.LoginActionsService.authenticate(LoginActionsService.java:259) at org.keycloak.services.resources.LoginActionsService.authenticateForm(LoginActionsService.java:320) at sun.reflect.GeneratedMethodAccessor719.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140) at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:509) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:399) at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:363) at org.jboss.resteasy.core.ResourceMethodInvoker$$Lambda$849.00000000BB8BBB40.get(Unknown Source) at org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:358) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:365) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:337) at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:137) at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:100) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:443) at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:233) at org.jboss.resteasy.core.SynchronousDispatcher$$Lambda$847.00000000BE026450.run(Unknown Source) at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:139) at org.jboss.resteasy.core.SynchronousDispatcher$$Lambda$848.00000000BDC48A90.get(Unknown Source) at org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:358) at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:142) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:219) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:791) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) at org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:90) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction$$Lambda$693.00000000BCF725B0.call(Unknown Source) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$694.00000000BD0FBEE0.call(Unknown Source) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$694.00000000BD0FBEE0.call(Unknown Source) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$694.00000000BD0FBEE0.call(Unknown Source) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$694.00000000BD0FBEE0.call(Unknown Source) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) at java.lang.Thread.run(Thread.java:812) ------------------ Interesting thing is keycloak handles OTP just fine if I have 'password+otp' only checked on, then we won't be able to log onto the machines via SSH using password, that defeats our purposes. I tested different version of JAVA and the latest keycloak (4.8.3) version (on REHL 7), all got the same results. I'm wondering if this is more likely a bug or I missed something. I'd appreciate if someone can advice what the approach is. Thank you very much. Mizuki _______________________________________________ keycloak-user mailing list keycloak-user(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-user