[keycloak-dev] [keycloak-user] Remove realm in HA environment throw org.keycloak.models.ModelException: javax.persistence.OptimisticLockException
Sebastian Laskawiec
slaskawi at redhat.com
Tue Feb 12 08:02:55 EST 2019
+madhura nishshanka <madhura.nishshanka at gmail.com>
I totally agree with you Marek.
Madhura, would that be ok with you for now? Once we revisit our storage
layer, we should have better option on the table.
On Mon, Feb 11, 2019 at 6:08 PM Marek Posolda <mposolda at redhat.com> wrote:
> Hello,
>
> am I understand correctly that you're creating realms (and then doing some
> other updates on those realms) and the other thread is trying to delete the
> realm?
>
> TBH it looks to me like quite corner case scenario as deleting the realm
> is not very often operation. Considering that we may want to do some more
> refactoring of our RDBMS storage layer in near future and considering that
> deleting realm is quite rare operation, I am not sure if we want to
> prioritize fixing this particular issue?
>
> Maybe others have different opinion, but ATM my vote is to deffer this.
> However if you send PR with the fix, it may be accepted (hard to promise as
> it depends on the amount of refactoring needed and the risk of introducing
> regressions with various databses etc.)
>
> Marek
>
> On 11/02/2019 14:54, Sebastian Laskawiec wrote:
>
> Dropping the Keycloak User mailing list.
>
> @Marek - so do we want to do something with it? Have you heard any more
> complains about this?
>
> On Tue, Feb 5, 2019 at 2:31 PM Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
>
>> So perhaps you can slightly modify your performance test to do steps 1..3
>> multiple times and then just wipe out all the realms that were created?
>>
>> On Mon, Feb 4, 2019 at 2:47 PM madhura nishshanka <
>> madhura.nishshanka at gmail.com> wrote:
>>
>>> I was invoking following 1,2,and 3 steps sequentially in one thread and
>>> then the 4th step in a seperate thread. The whole test was done with
>>> multiple theads in parallel.
>>>
>>> 1) Create realm with a user
>>> 2) Create another user on the same realm
>>> 3) Delete orginal user
>>> 4) Delete the new realm.
>>>
>>> On Mon, Feb 4, 2019, 6:10 PM Sebastian Laskawiec <slaskawi at redhat.com
>>> wrote:
>>>
>>>> Let me add +Marek Posolda <mposolda at redhat.com>, maybe he'll have
>>>> better idea, what might be causing this...
>>>>
>>>> The error happened here [1]. Hibernate wanted to remove a
>>>> given RoleEntity object but between `em.remove(roleEntity)` and
>>>> `em.flush()`, some other transaction had removed that object from the
>>>> database.
>>>>
>>>> One of the things that could result in such a behavior is deleting
>>>> multiple realms at the same time. Could you please tell us more about your
>>>> test? How it works, does it perform operations in sequential order or in
>>>> parallel?
>>>>
>>>> One improvement we could do on our side is to swap flushing the
>>>> EntityManager and publishing events. That could also potentially solve your
>>>> problem. Marek, what do you think about this?
>>>>
>>>> Thanks,
>>>> Sebastian
>>>>
>>>> [1]
>>>> https://github.com/keycloak/keycloak/blob/7d85ce93bbf33eb11981a6c118abc48cab39742d/model/jpa/src/main/java/org/keycloak/models/jpa/JpaRealmProvider.java#L320
>>>>
>>>> On Fri, Feb 1, 2019 at 5:12 AM madhura nishshanka <
>>>> madhura.nishshanka at gmail.com> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> I am getting "org.keycloak.models.ModelException:
>>>>> javax.persistence.OptimisticLockException: Batch update returned
>>>>> unexpected
>>>>> row count from update [0]; actual row count: 0; expected: 1" When a
>>>>> realm
>>>>> is delte from keycloak java admin client. This occurs in a HA
>>>>> environment
>>>>> when we do a performance test. Can someone please help me on this?
>>>>>
>>>>> I am using keycloak 4.8.1 final.
>>>>>
>>>>> Full exception
>>>>> 11:56:25,452 ERROR [org.keycloak.services.error.KeycloakErrorHandler]
>>>>> (default task-2) Uncaught server error:
>>>>> org.keycloak.models.ModelException:
>>>>> javax.persistence.OptimisticLockException: Batch update returned
>>>>> unexpected
>>>>> row count from update [0]; actual row count: 0; expected: 1
>>>>> at
>>>>>
>>>>> org.keycloak.connections.jpa.PersistenceExceptionConverter.convert(PersistenceExceptionConverter.java:61)
>>>>> at
>>>>>
>>>>> org.keycloak.connections.jpa.PersistenceExceptionConverter.invoke(PersistenceExceptionConverter.java:51)
>>>>> at com.sun.proxy.$Proxy99.flush(Unknown Source)
>>>>> at
>>>>>
>>>>> org.keycloak.models.jpa.JpaRealmProvider.removeRole(JpaRealmProvider.java:320)
>>>>> at
>>>>>
>>>>> org.keycloak.models.jpa.JpaRealmProvider.removeClient(JpaRealmProvider.java:567)
>>>>> at
>>>>>
>>>>> *org.keycloak.models.jpa.JpaRealmProvider.removeRealm(JpaRealmProvider.java:153)*
>>>>> at
>>>>>
>>>>> org.keycloak.models.cache.infinispan.RealmCacheSession.removeRealm(RealmCacheSession.java:486)
>>>>> at
>>>>>
>>>>> org.keycloak.services.managers.RealmManager.removeRealm(RealmManager.java:248)
>>>>> at
>>>>>
>>>>> org.keycloak.services.resources.admin.RealmAdminResource.deleteRealm(RealmAdminResource.java:453)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>>
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>> at
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:509)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:399)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:363)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:358)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:365)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:337)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:137)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:106)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:132)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:100)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:443)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:233)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:139)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:358)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:142)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:219)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
>>>>> at
>>>>>
>>>>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:791)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
>>>>> at
>>>>>
>>>>> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:90)
>>>>> at
>>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
>>>>> at
>>>>>
>>>>> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
>>>>> at
>>>>>
>>>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
>>>>> at
>>>>>
>>>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>>>> at
>>>>>
>>>>> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
>>>>> at
>>>>>
>>>>> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
>>>>> at
>>>>>
>>>>> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
>>>>> at
>>>>>
>>>>> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
>>>>> at
>>>>>
>>>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>>>> at
>>>>>
>>>>> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
>>>>> at
>>>>>
>>>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>>>> at
>>>>>
>>>>> org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68)
>>>>> at
>>>>>
>>>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
>>>>> at
>>>>>
>>>>> org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
>>>>> at
>>>>>
>>>>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
>>>>> at
>>>>>
>>>>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
>>>>> at
>>>>>
>>>>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
>>>>> at
>>>>>
>>>>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
>>>>> at
>>>>>
>>>>> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
>>>>> at
>>>>> io.undertow.server.Connectors.executeRootHandler(Connectors.java:360)
>>>>> at
>>>>>
>>>>> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
>>>>> at
>>>>>
>>>>> org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
>>>>> at
>>>>>
>>>>> org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
>>>>> at
>>>>>
>>>>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
>>>>> at
>>>>>
>>>>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
>>>>> at java.lang.Thread.run(Thread.java:748)
>>>>> Caused by: javax.persistence.OptimisticLockException: Batch update
>>>>> returned
>>>>> unexpected row count from update [0]; actual row count: 0; expected: 1
>>>>> at
>>>>>
>>>>> org.hibernate.internal.ExceptionConverterImpl.wrapStaleStateException(ExceptionConverterImpl.java:238)
>>>>> at
>>>>>
>>>>> org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:93)
>>>>> at
>>>>>
>>>>> org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:181)
>>>>> at
>>>>>
>>>>> org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:188)
>>>>> at
>>>>> org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1460)
>>>>> at
>>>>> org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1440)
>>>>> at sun.reflect.GeneratedMethodAccessor483.invoke(Unknown
>>>>> Source)
>>>>> at
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>> at
>>>>>
>>>>> org.keycloak.connections.jpa.PersistenceExceptionConverter.invoke(PersistenceExceptionConverter.java:49)
>>>>> ... 78 more
>>>>> Caused by: org.hibernate.StaleStateException: Batch update returned
>>>>> unexpected row count from update [0]; actual row count: 0; expected: 1
>>>>> at
>>>>>
>>>>> org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched(Expectations.java:67)
>>>>> at
>>>>>
>>>>> org.hibernate.jdbc.Expectations$BasicExpectation.verifyOutcome(Expectations.java:54)
>>>>> at
>>>>>
>>>>> org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:46)
>>>>> at
>>>>>
>>>>> org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3478)
>>>>> at
>>>>>
>>>>> org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3735)
>>>>> at
>>>>>
>>>>> org.hibernate.action.internal.EntityDeleteAction.execute(EntityDeleteAction.java:99)
>>>>> at
>>>>>
>>>>> org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:604)
>>>>> at
>>>>>
>>>>> org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:478)
>>>>> at
>>>>>
>>>>> org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:356)
>>>>> at
>>>>>
>>>>> org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
>>>>> at
>>>>> org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1454)
>>>>> ... 83 more
>>>>>
>>>>> Thanks
>>>>> Madhura
>>>>> _______________________________________________
>>>>> keycloak-user mailing list
>>>>> keycloak-user at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>>>>
>>>>
>
More information about the keycloak-dev
mailing list