[keycloak-user] db deadlock in concurrence environment
Thomas Darimont
thomas.darimont at googlemail.com
Mon Mar 7 16:37:44 EST 2016
Hello Alex,
looks like your database connection pool is exhausted - how is your
connection pool configured in wildfly?
Cheers,
Thomas
2016-03-07 22:32 GMT+01:00 alex orl <alex_orl1079 at yahoo.it>:
> Hi,
> i wrote my custom federation provider.
> All the synchronize operations (roles and user-roles) are done inside the
> getUserByUsername and the isValid method.
> This way every time a user or a new-user logins into my system, all the
> roles and role-mapping are kept up to date.
> All seems to work well with one user, but now i'm experiencing a lot of
> exceptions in a concurrence environment.
> These are the tests i'm launching:
> 1) 20 concurrent threads, each of which tries to login, to send a REST
> request (to backend) and finally to logout. All 20 threads login with the
> same test username.
> 2) 200 concurrent threads, doing the same as at the previous point, with
> 20 differents username.
>
> Everytime each federation provider instance tries to synchronize all realm
> roles...adding or removing roles in concurrence, i catch this error:
>
>
> 16:10:08,228 ERROR [io.undertow.request] (default task-96) UT005023:
> Exception handling request to
> /auth/realms/MyRealm/protocol/openid-connect/token:
> java.lang.RuntimeException: reques
> t path: /auth/realms/MyRealm/protocol/openid-connect/token
> at
> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:75)
> at
> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
> at
> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)
> at
> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> at
> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> at
> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> at
> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> at
> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> at
> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)
> at
> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> at
> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> at
> io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> at
> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> at
> io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> at
> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.jboss.resteasy.spi.UnhandledException:
> javax.persistence.PersistenceException:
> org.hibernate.exception.GenericJDBCException: Could not open connection
> at
> org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:76)
> at
> org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:212)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:149)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:372)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
> at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at
> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130)
> at
> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:61)
> ... 29 more
> Caused by: javax.persistence.PersistenceException:
> org.hibernate.exception.GenericJDBCException: Could not open connection
> at
> org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)
> at
> org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677)
> at
> org.hibernate.jpa.spi.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1771)
> at
> org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:64)
> at
> org.keycloak.connections.jpa.JpaKeycloakTransaction.begin(JpaKeycloakTransaction.java:22)
> at
> org.keycloak.services.DefaultKeycloakTransactionManager.enlist(DefaultKeycloakTransactionManager.java:25)
> at
> org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.create(DefaultJpaConnectionProviderFactory.java:46)
> at
> org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.create(DefaultJpaConnectionProviderFactory.java:30)
> at
> org.keycloak.services.DefaultKeycloakSession.getProvider(DefaultKeycloakSession.java:103)
> at
> org.keycloak.models.jpa.JpaUserProviderFactory.create(JpaUserProviderFactory.java:34)
> at
> org.keycloak.models.jpa.JpaUserProviderFactory.create(JpaUserProviderFactory.java:16)
> at
> org.keycloak.services.DefaultKeycloakSession.getProvider(DefaultKeycloakSession.java:103)
> at
> org.keycloak.models.cache.infinispan.DefaultCacheUserProvider.getDelegate(DefaultCacheUserProvider.java:50)
> at
> org.keycloak.models.cache.infinispan.DefaultCacheUserProvider.getUserByUsername(DefaultCacheUserProvider.java:147)
> at
> org.keycloak.models.UserFederationManager.getUserByUsername(UserFederationManager.java:180)
> at
> org.keycloak.models.utils.KeycloakModelUtils.findUserByNameOrEmail(KeycloakModelUtils.java:246)
> at
> org.keycloak.authentication.authenticators.directgrant.ValidateUsername.authenticate(ValidateUsername.java:47)
> at
> org.keycloak.authentication.DefaultAuthenticationFlow.processFlow(DefaultAuthenticationFlow.java:155)
> at
> org.keycloak.authentication.AuthenticationProcessor.authenticateOnly(AuthenticationProcessor.java:776)
> at
> org.keycloak.protocol.oidc.endpoints.TokenEndpoint.buildResourceOwnerPasswordCredentialsGrant(TokenEndpoint.java:369)
> at
> org.keycloak.protocol.oidc.endpoints.TokenEndpoint.build(TokenEndpoint.java:110)
> at sun.reflect.GeneratedMethodAccessor279.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
> at
> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:296)
> at
> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:250)
> at
> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:140)
> at
> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:109)
> at
> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:135)
> at
> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:103)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
> ... 37 more
> Caused by: org.hibernate.exception.GenericJDBCException: Could not open
> connection
> at
> org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
> at
> org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126)
> at
> org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112)
> at
> org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:235)
> at
> org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:171)
> at
> org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:67)
> at
> org.hibernate.engine.transaction.spi.AbstractTransactionImpl.begin(AbstractTransactionImpl.java:162)
> at
> org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1471)
> at
> org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:61)
> ... 65 more
> Caused by: java.sql.SQLException: javax.resource.ResourceException:
> IJ000453: Unable to get managed connection for
> java:jboss/datasources/KeycloakOracleDS
> at
> org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:146)
> at
> org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:67)
> at
> org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:139)
> at
> org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:380)
> at
> org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:228)
> ... 70 more
> Caused by: javax.resource.ResourceException: IJ000453: Unable to get
> managed connection for java:jboss/datasources/KeycloakOracleDS
> at
> org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:646)
> at
> org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:430)
> at
> org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:737)
> at
> org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:138)
> ... 74 more
> Caused by: javax.resource.ResourceException: IJ000655: No managed
> connections available within configured blocking timeout (30000 [ms])
> at
> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:569)
> at
> org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:627)
> at
> org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:599)
> at
> org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:579)
> ... 77 more
>
> 16:10:08,260 ERROR [stderr] (default task-41)
> javax.persistence.PersistenceException:
> org.hibernate.exception.GenericJDBCException: Could not open connection
> 16:10:08,333 ERROR [stderr] (default task-41) at
> org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)
> 16:10:08,333 ERROR [stderr] (default task-41) at
> org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677)
>
> What's the solution to correctly handle a concurrence environment?
> What am doing wrong? Is there a way to make synchronized keycloak jpa
> transactions?
> Thanks
>
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160307/84beaa5c/attachment-0001.html
More information about the keycloak-user
mailing list