in addition to my previous email i attach the deadlock exception stactrace:
2:53:20,188 WARN [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-118) SQL Error: 60, SQLState: 61000
12:53:20,189 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-118) ORA-00060: deadlock detected while waiting for resource
12:53:20,189 INFO [org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl] (default task-118) HHH000010: On release of batch it still contained JDBC statements
12:53:20,192 ERROR [org.keycloak.services.resources.ModelExceptionMapper] (default task-118) javax.persistence.PersistenceException: org.hibernate.exception.LockAcquisitionException: cou
ld not execute statement: org.keycloak.models.ModelException: javax.persistence.PersistenceException: org.hibernate.exception.LockAcquisitionException: could not execute statement
at org.keycloak.connections.jpa.PersistenceExceptionConverter.convert(PersistenceExceptionConverter.java:44)
at org.keycloak.connections.jpa.PersistenceExceptionConverter.invoke(PersistenceExceptionConverter.java:34)
at com.sun.proxy.$Proxy147.flush(Unknown Source)
at org.keycloak.models.jpa.JpaUserProvider.removeUser(JpaUserProvider.java:117)
at org.keycloak.models.jpa.JpaUserProvider.removeUser(JpaUserProvider.java:97)
at org.keycloak.models.cache.infinispan.DefaultCacheUserProvider.removeUser(DefaultCacheUserProvider.java:283)
at org.keycloak.models.UserFederationManager.deleteInvalidUser(UserFederationManager.java:113)
at org.keycloak.models.UserFederationManager.validateAndProxyUser(UserFederationManager.java:135)
at org.keycloak.models.UserFederationManager.getUserByUsername(UserFederationManager.java:182)
at org.keycloak.models.utils.KeycloakModelUtils.findUserByNameOrEmail(KeycloakModelUtils.java:246)
at org.keycloak.authentication.authenticators.directgrant.ValidateUsername.authenticate(ValidateUsername.java:47)
at org.keycloak.authentication.DefaultAuthenticationFlow.processFlow(DefaultAuthenticationFlow.java:155)
at org.keycloak.authentication.AuthenticationProcessor.authenticateOnly(AuthenticationProcessor.java:776)
at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.buildResourceOwnerPasswordCredentialsGrant(TokenEndpoint.java:369)
at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.build(TokenEndpoint.java:110)
at sun.reflect.GeneratedMethodAccessor284.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:296)
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:250)
at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:140)
at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:109)
at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:135)
at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:103)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130)
at org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:61)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.LockAcquisitionException: could not execute statement
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1683)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.flush(AbstractEntityManagerImpl.java:1338)
at sun.reflect.GeneratedMethodAccessor280.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.keycloak.connections.jpa.PersistenceExceptionConverter.invoke(PersistenceExceptionConverter.java:32)
... 61 more
Caused by: org.hibernate.exception.LockAcquisitionException: could not execute statement
at org.hibernate.dialect.Oracle8iDialect$2.convert(Oracle8iDialect.java:473)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:211)
at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:62)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3400)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3630)
at org.hibernate.action.internal.EntityDeleteAction.execute(EntityDeleteAction.java:114)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:465)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:351)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:350)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1258)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.flush(AbstractEntityManagerImpl.java:1335)
... 65 more
Caused by: java.sql.SQLException: ORA-00060: deadlock detected while waiting for resource
Il Martedì 8 Marzo 2016 10:46, alex orl <alex_orl1079@yahoo.it> ha scritto:
I will try to make my use case clearer:
Every time a user logins into my system, so when keycloak invokes "getUserByUsername()" method or the "isValid()" method, i do the realm roles synchronization:
i.e.
RoleModel keycloakRole;
public UserModel getUserByUsername(RealmModel realm, String username) {
...
for ( MyRole role : MyRoleList) {
keycloakRole = realm.addRole(myNewRole.getName());
keycloakRole.setDescription(myNewRole.getDescription());
}
public boolean isValid(RealmModel realm, UserModel local) {
...
for ( MyRole role : MyRoleList) {
keycloakRole = realm.addRole(myNewRole.getName());
keycloakRole.setDescription(myNewRole.getDescription());
}
This does work well just with a single thread user.
When the environment is configured with 20 concurrent threads/users trying to login, i get "deadlock exception". I can say the one i printed in my first email is a direct consequence of the deadlock.
After:
"optministiLock exception"
i get also:
09:51:12,677 WARN [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-2) SQL Error: 1, SQLState: 23000
09:51:12,679 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-102) ORA-00001: unique constraint (KEYCLOAK17.UK_J3RWUVD56ONTGSUHOGM184WW2-2) violated
09:51:12,679 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-105) ORA-00001: unique constraint (KEYCLOAK17.UK_J3RWUVD56ONTGSUHOGM184WW2-2) violated
Now my question is:
- I never found a point inside the keycloak authentication flow from which triggering the synchronization operations so i guessed to make this operation inside the LOGIN phase. Is it right?
- doing like this, is the best way or is there a way to hook the login jpa transaction? the way that all the sync operations are flushed after the whole login transaction is committed?
- Or am i totally wrong and there is a specific pattern to follow for all the roles and attributes synchronize operations?
thanks
Il Martedì 8 Marzo 2016 7:34, Stian Thorgersen <sthorger@redhat.com> ha scritto:
As it times out after 30 seconds to obtain a connection it looks more like connections/transactions are not being closed. Is your federation provider using the same datasource? Any chance it's not closing the connections properly?