Hello Alex, looks like your database connection pool is exhausted - how is your connection pool configured in wildfly? Cheers, Thomas 2016-03-07 22:32 GMT+01:00 alex orl <alex_orl1079(a)yahoo.it&gt;:

I will try to make my use case clearer:Every time a user logins into my system, so when keycloak invokes "getUserByUsername()" method or the "isValid()"  method,  i do the realm roles synchronization: i.e.RoleModel keycloakRole;  public UserModel getUserByUsername(RealmModel realm, String username) {     ...     for ( MyRole role : MyRoleList) {            keycloakRole = realm.addRole(myNewRole.getName());        keycloakRole.setDescription(myNewRole.getDescription());    } public boolean isValid(RealmModel realm, UserModel local) {      ...     for ( MyRole role : MyRoleList) {            keycloakRole = realm.addRole(myNewRole.getName());        keycloakRole.setDescription(myNewRole.getDescription());    } This does work well just with a single thread user.When the environment is configured with 20 concurrent threads/users trying to login,  i get "deadlock exception". I can say the one i printed in my first email is a direct consequence of the deadlock.After:"optministiLock exception" i get also:09:51:12,677 WARN  [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-2) SQL Error: 1, SQLState: 2300009:51:12,679 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-102) ORA-00001: unique constraint (KEYCLOAK17.UK_J3RWUVD56ONTGSUHOGM184WW2-2) violated 09:51:12,679 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-105) ORA-00001: unique constraint (KEYCLOAK17.UK_J3RWUVD56ONTGSUHOGM184WW2-2) violated Now my question is:- I never found a point inside the keycloak authentication flow from which triggering the synchronization operations so i guessed to make this operation inside the LOGIN phase. Is it right?- doing like this, is the best way or is there a way to hook the login jpa transaction? the way that all the sync operations are flushed after the whole login transaction is committed?- Or am i totally wrong and there is a specific pattern to follow for all the roles and attributes synchronize operations? thanks Il Martedì 8 Marzo 2016 7:34, Stian Thorgersen <sthorger(a)redhat.com&gt; ha scritto: As it times out after 30 seconds to obtain a connection it looks more like connections/transactions are not being closed. Is your federation provider using the same datasource? Any chance it's not closing the connections properly? On 7 March 2016 at 22:37, Thomas Darimont <thomas.darimont(a)googlemail.com&gt; wrote: Hello Alex, looks like your database connection pool is exhausted - how is your connection pool configured in wildfly? Cheers,Thomas 2016-03-07 22:32 GMT+01:00 alex orl <alex_orl1079(a)yahoo.it&gt;: Hi,i wrote my custom federation provider.All the synchronize operations (roles and user-roles) are done inside the getUserByUsername and the isValid method.This way every time a user or a new-user logins into my system, all the roles and role-mapping are kept up to date.All seems to work well with one user, but now i'm experiencing a lot of exceptions in a concurrence environment.These are the tests i'm launching:1) 20 concurrent threads, each of which tries to login, to send a REST request (to backend) and finally to logout. All 20 threads login with the same test username.2) 200 concurrent threads, doing the same as at the previous point, with 20 differents username. Everytime each federation provider instance tries to synchronize all realm roles...adding or removing roles in concurrence, i catch this error: 16:10:08,228 ERROR [io.undertow.request] (default task-96) UT005023: Exception handling request to /auth/realms/MyRealm/protocol/openid-connect/token: java.lang.RuntimeException: request path: /auth/realms/MyRealm/protocol/openid-connect/token        at org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:75)        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)        at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)        at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)        at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)        at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)        at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)        at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)        at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)        at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)        at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)        at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)        at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)        at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)        at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)        at java.lang.Thread.run(Thread.java:745)Caused by: org.jboss.resteasy.spi.UnhandledException: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection        at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:76)        at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:212)        at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:149)        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:372)        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)        at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)        at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)        at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)        at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130)        at org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:61)        ... 29 moreCaused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677)        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1771)        at org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:64)        at org.keycloak.connections.jpa.JpaKeycloakTransaction.begin(JpaKeycloakTransaction.java:22)        at org.keycloak.services.DefaultKeycloakTransactionManager.enlist(DefaultKeycloakTransactionManager.java:25)        at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.create(DefaultJpaConnectionProviderFactory.java:46)        at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.create(DefaultJpaConnectionProviderFactory.java:30)        at org.keycloak.services.DefaultKeycloakSession.getProvider(DefaultKeycloakSession.java:103)        at org.keycloak.models.jpa.JpaUserProviderFactory.create(JpaUserProviderFactory.java:34)        at org.keycloak.models.jpa.JpaUserProviderFactory.create(JpaUserProviderFactory.java:16)        at org.keycloak.services.DefaultKeycloakSession.getProvider(DefaultKeycloakSession.java:103)        at org.keycloak.models.cache.infinispan.DefaultCacheUserProvider.getDelegate(DefaultCacheUserProvider.java:50)        at org.keycloak.models.cache.infinispan.DefaultCacheUserProvider.getUserByUsername(DefaultCacheUserProvider.java:147)        at org.keycloak.models.UserFederationManager.getUserByUsername(UserFederationManager.java:180)        at org.keycloak.models.utils.KeycloakModelUtils.findUserByNameOrEmail(KeycloakModelUtils.java:246)        at org.keycloak.authentication.authenticators.directgrant.ValidateUsername.authenticate(ValidateUsername.java:47)        at org.keycloak.authentication.DefaultAuthenticationFlow.processFlow(DefaultAuthenticationFlow.java:155)        at org.keycloak.authentication.AuthenticationProcessor.authenticateOnly(AuthenticationProcessor.java:776)        at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.buildResourceOwnerPasswordCredentialsGrant(TokenEndpoint.java:369)        at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.build(TokenEndpoint.java:110)        at sun.reflect.GeneratedMethodAccessor279.invoke(Unknown Source)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:497)        at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)        at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:296)        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:250)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:140)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:109)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:135)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:103)        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)        ... 37 moreCaused by: org.hibernate.exception.GenericJDBCException: Could not open connection        at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126)        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112)        at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:235)        at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:171)        at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:67)        at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.begin(AbstractTransactionImpl.java:162)        at org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1471)        at org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:61)        ... 65 moreCaused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/KeycloakOracleDS        at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:146)        at org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:67)        at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:139)        at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:380)        at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:228)        ... 70 moreCaused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/KeycloakOracleDS        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:646)        at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:430)        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:737)        at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:138)        ... 74 moreCaused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])        at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:569)        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:627)        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:599)        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:579)        ... 77 more 16:10:08,260 ERROR [stderr] (default task-41) javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection16:10:08,333 ERROR [stderr] (default task-41)   at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)16:10:08,333 ERROR [stderr] (default task-41)   at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677) What's the solution to correctly handle a concurrence environment?What am doing wrong? Is there a way to make synchronized keycloak jpa transactions?Thanks _______________________________________________ keycloak-user mailing list keycloak-user(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-user _______________________________________________ keycloak-user mailing list keycloak-user(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-user

Sorry again...another important information is that i'm working on keycloak 1.7.0 finalAny help?is there any bug fixed on the next releases? Il Martedì 8 Marzo 2016 13:01, alex orl <alex_orl1079(a)yahoo.it&gt; ha scritto: in addition to my previous email i attach the deadlock exception stactrace: 2:53:20,188 WARN  [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-118) SQL Error: 60, SQLState: 6100012:53:20,189 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-118) ORA-00060: deadlock detected while waiting for resource 12:53:20,189 INFO  [org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl] (default task-118) HHH000010: On release of batch it still contained JDBC statements12:53:20,192 ERROR [org.keycloak.services.resources.ModelExceptionMapper] (default task-118) javax.persistence.PersistenceException: org.hibernate.exception.LockAcquisitionException: could not execute statement: org.keycloak.models.ModelException: javax.persistence.PersistenceException: org.hibernate.exception.LockAcquisitionException: could not execute statement        at org.keycloak.connections.jpa.PersistenceExceptionConverter.convert(PersistenceExceptionConverter.java:44)        at org.keycloak.connections.jpa.PersistenceExceptionConverter.invoke(PersistenceExceptionConverter.java:34)        at com.sun.proxy.$Proxy147.flush(Unknown Source)        at org.keycloak.models.jpa.JpaUserProvider.removeUser(JpaUserProvider.java:117)        at org.keycloak.models.jpa.JpaUserProvider.removeUser(JpaUserProvider.java:97)        at org.keycloak.models.cache.infinispan.DefaultCacheUserProvider.removeUser(DefaultCacheUserProvider.java:283)        at org.keycloak.models.UserFederationManager.deleteInvalidUser(UserFederationManager.java:113)        at org.keycloak.models.UserFederationManager.validateAndProxyUser(UserFederationManager.java:135)        at org.keycloak.models.UserFederationManager.getUserByUsername(UserFederationManager.java:182)        at org.keycloak.models.utils.KeycloakModelUtils.findUserByNameOrEmail(KeycloakModelUtils.java:246)        at org.keycloak.authentication.authenticators.directgrant.ValidateUsername.authenticate(ValidateUsername.java:47)        at org.keycloak.authentication.DefaultAuthenticationFlow.processFlow(DefaultAuthenticationFlow.java:155)        at org.keycloak.authentication.AuthenticationProcessor.authenticateOnly(AuthenticationProcessor.java:776)        at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.buildResourceOwnerPasswordCredentialsGrant(TokenEndpoint.java:369)        at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.build(TokenEndpoint.java:110)        at sun.reflect.GeneratedMethodAccessor284.invoke(Unknown Source)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:497)        at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)        at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:296)        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:250)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:140)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:109)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:135)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:103)        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)        at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)        at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)        at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)        at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130)        at org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:61)        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)        at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)        at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)        at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)        at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)        at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)        at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)        at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)        at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)        at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)        at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)        at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)        at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)        at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)        at java.lang.Thread.run(Thread.java:745)Caused by: javax.persistence.PersistenceException: org.hibernate.exception.LockAcquisitionException: could not execute statement        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677)        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1683)        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.flush(AbstractEntityManagerImpl.java:1338)        at sun.reflect.GeneratedMethodAccessor280.invoke(Unknown Source)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:497)        at org.keycloak.connections.jpa.PersistenceExceptionConverter.invoke(PersistenceExceptionConverter.java:32)        ... 61 moreCaused by: org.hibernate.exception.LockAcquisitionException: could not execute statement        at org.hibernate.dialect.Oracle8iDialect$2.convert(Oracle8iDialect.java:473)        at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126)        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112)        at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:211)        at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:62)        at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3400)        at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3630)        at org.hibernate.action.internal.EntityDeleteAction.execute(EntityDeleteAction.java:114)        at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:465)        at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:351)        at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:350)        at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56)        at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1258)        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.flush(AbstractEntityManagerImpl.java:1335)        ... 65 moreCaused by: java.sql.SQLException: ORA-00060: deadlock detected while waiting for resource Il Martedì 8 Marzo 2016 10:46, alex orl <alex_orl1079(a)yahoo.it&gt; ha scritto: I will try to make my use case clearer:Every time a user logins into my system, so when keycloak invokes "getUserByUsername()" method or the "isValid()"  method,  i do the realm roles synchronization: i.e.RoleModel keycloakRole;  public UserModel getUserByUsername(RealmModel realm, String username) {     ...     for ( MyRole role : MyRoleList) {            keycloakRole = realm.addRole(myNewRole.getName());        keycloakRole.setDescription(myNewRole.getDescription());    } public boolean isValid(RealmModel realm, UserModel local) {      ...     for ( MyRole role : MyRoleList) {            keycloakRole = realm.addRole(myNewRole.getName());        keycloakRole.setDescription(myNewRole.getDescription());    } This does work well just with a single thread user.When the environment is configured with 20 concurrent threads/users trying to login,  i get "deadlock exception". I can say the one i printed in my first email is a direct consequence of the deadlock.After:"optministiLock exception" i get also:09:51:12,677 WARN  [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-2) SQL Error: 1, SQLState: 2300009:51:12,679 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-102) ORA-00001: unique constraint (KEYCLOAK17.UK_J3RWUVD56ONTGSUHOGM184WW2-2) violated 09:51:12,679 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-105) ORA-00001: unique constraint (KEYCLOAK17.UK_J3RWUVD56ONTGSUHOGM184WW2-2) violated Now my question is:- I never found a point inside the keycloak authentication flow from which triggering the synchronization operations so i guessed to make this operation inside the LOGIN phase. Is it right?- doing like this, is the best way or is there a way to hook the login jpa transaction? the way that all the sync operations are flushed after the whole login transaction is committed?- Or am i totally wrong and there is a specific pattern to follow for all the roles and attributes synchronize operations? thanks Il Martedì 8 Marzo 2016 7:34, Stian Thorgersen <sthorger(a)redhat.com&gt; ha scritto: As it times out after 30 seconds to obtain a connection it looks more like connections/transactions are not being closed. Is your federation provider using the same datasource? Any chance it's not closing the connections properly? On 7 March 2016 at 22:37, Thomas Darimont <thomas.darimont(a)googlemail.com&gt; wrote: Hello Alex, looks like your database connection pool is exhausted - how is your connection pool configured in wildfly? Cheers,Thomas 2016-03-07 22:32 GMT+01:00 alex orl <alex_orl1079(a)yahoo.it&gt;: Hi,i wrote my custom federation provider.All the synchronize operations (roles and user-roles) are done inside the getUserByUsername and the isValid method.This way every time a user or a new-user logins into my system, all the roles and role-mapping are kept up to date.All seems to work well with one user, but now i'm experiencing a lot of exceptions in a concurrence environment.These are the tests i'm launching:1) 20 concurrent threads, each of which tries to login, to send a REST request (to backend) and finally to logout. All 20 threads login with the same test username.2) 200 concurrent threads, doing the same as at the previous point, with 20 differents username. Everytime each federation provider instance tries to synchronize all realm roles...adding or removing roles in concurrence, i catch this error: 16:10:08,228 ERROR [io.undertow.request] (default task-96) UT005023: Exception handling request to /auth/realms/MyRealm/protocol/openid-connect/token: java.lang.RuntimeException: request path: /auth/realms/MyRealm/protocol/openid-connect/token        at org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:75)        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)        at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)        at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)        at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)        at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)        at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)        at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)        at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)        at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)        at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)        at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)        at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)        at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)        at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)        at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)        at java.lang.Thread.run(Thread.java:745)Caused by: org.jboss.resteasy.spi.UnhandledException: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection        at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:76)        at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:212)        at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:149)        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:372)        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)        at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)        at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)        at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)        at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130)        at org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:61)        ... 29 moreCaused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677)        at org.hibernate.jpa.spi.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1771)        at org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:64)        at org.keycloak.connections.jpa.JpaKeycloakTransaction.begin(JpaKeycloakTransaction.java:22)        at org.keycloak.services.DefaultKeycloakTransactionManager.enlist(DefaultKeycloakTransactionManager.java:25)        at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.create(DefaultJpaConnectionProviderFactory.java:46)        at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.create(DefaultJpaConnectionProviderFactory.java:30)        at org.keycloak.services.DefaultKeycloakSession.getProvider(DefaultKeycloakSession.java:103)        at org.keycloak.models.jpa.JpaUserProviderFactory.create(JpaUserProviderFactory.java:34)        at org.keycloak.models.jpa.JpaUserProviderFactory.create(JpaUserProviderFactory.java:16)        at org.keycloak.services.DefaultKeycloakSession.getProvider(DefaultKeycloakSession.java:103)        at org.keycloak.models.cache.infinispan.DefaultCacheUserProvider.getDelegate(DefaultCacheUserProvider.java:50)        at org.keycloak.models.cache.infinispan.DefaultCacheUserProvider.getUserByUsername(DefaultCacheUserProvider.java:147)        at org.keycloak.models.UserFederationManager.getUserByUsername(UserFederationManager.java:180)        at org.keycloak.models.utils.KeycloakModelUtils.findUserByNameOrEmail(KeycloakModelUtils.java:246)        at org.keycloak.authentication.authenticators.directgrant.ValidateUsername.authenticate(ValidateUsername.java:47)        at org.keycloak.authentication.DefaultAuthenticationFlow.processFlow(DefaultAuthenticationFlow.java:155)        at org.keycloak.authentication.AuthenticationProcessor.authenticateOnly(AuthenticationProcessor.java:776)        at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.buildResourceOwnerPasswordCredentialsGrant(TokenEndpoint.java:369)        at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.build(TokenEndpoint.java:110)        at sun.reflect.GeneratedMethodAccessor279.invoke(Unknown Source)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:497)        at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)        at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:296)        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:250)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:140)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:109)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:135)        at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:103)        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)        ... 37 moreCaused by: org.hibernate.exception.GenericJDBCException: Could not open connection        at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126)        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112)        at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:235)        at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:171)        at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:67)        at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.begin(AbstractTransactionImpl.java:162)        at org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1471)        at org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:61)        ... 65 moreCaused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/KeycloakOracleDS        at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:146)        at org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:67)        at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:139)        at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:380)        at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:228)        ... 70 moreCaused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/KeycloakOracleDS        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:646)        at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:430)        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:737)        at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:138)        ... 74 moreCaused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])        at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:569)        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:627)        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:599)        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:579)        ... 77 more 16:10:08,260 ERROR [stderr] (default task-41) javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection16:10:08,333 ERROR [stderr] (default task-41)   at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)16:10:08,333 ERROR [stderr] (default task-41)   at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677) What's the solution to correctly handle a concurrence environment?What am doing wrong? Is there a way to make synchronized keycloak jpa transactions?Thanks _______________________________________________ keycloak-user mailing list keycloak-user(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-user _______________________________________________ keycloak-user mailing list keycloak-user(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-user