[keycloak-user] Lost session when removing an instance off cluster
Stian Thorgersen
sthorger at redhat.com
Tue Jul 5 02:41:25 EDT 2016
The sessions cache is a distributed-cache, but by default it only has 1
owner of each segment. This means that sessions are not replicated to
multiple nodes. So it's expected that you loose sessions when a node is
removed. If you want sessions to survive the removal of a node you should
increase the number of owners for the cache. For example:
<distributed-cache name="sessions" mode="SYNC" owners="2"/>
We've not actually tried with enabling transactions on the sessions, but I
would say performance is more important. Worst case scenario is that a
session is lost and a user has to re-authenticate.
BTW the config you have for offlineSessions and loginFailures are wrong
they both need to be distributed caches as well in clustered mode:
<distributed-cache name="offlineSessions" mode="SYNC" owners="1"/>
<distributed-cache name="loginFailures" mode="SYNC" owners="1"/>
On 4 July 2016 at 15:45, Sarp Kaya <akaya at expedia.com> wrote:
> Just to add on this.
>
> I think I misunderstood something or Keycloak does not really implement
> the infinispan caching lifetime correctly.
>
> So first thing I did is to TRACE the infinispan logs. So I added this
> logger in standalone.xml:
>
> <logger category="org.infinispan">
> <level name="TRACE"/>
> </logger>
>
>
> The next thing I did is to do the timing. Basically my test is that what
> happens when I log in. I also enabled events and printing the events as
> well.
>
> What I found is:
>
>
> 1. Login event is triggered at 13:18:35,193 in Keycloak.
> 2. The browser got the full response at 13:18:35,273
> 3. Now the surprising part is the fact that there are whole a lot of
> infinispan logging out occurring for “sessions” cache after the browser got
> the response. The last log message occurring is like this:
>
> 2016-07-04 13:18:35,345 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (default task-4) Response(s) to TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=19, gtx=GlobalTransaction:<kc1>:239:local, cacheName=sessions} is {}
>
>
> To see the all the logs after event trigger please check here:
> <http://pastebin.com/pkvYP07T> <http://pastebin.com/pkvYP07T>
> <http://pastebin.com/pkvYP07T> <http://pastebin.com/pkvYP07T>
> http://pastebin.com/p4K2Ghff
>
> So my understanding was that I marked sessions as SYNC, which meant to me
> that it is supposed to do all of the synchronization before it sends the
> response back to the browser. If it does the other way around then it
> should be ASYNC mode. Could you please clarify the intended/expected
> behavior?
>
> Kind Regards,
> Sarp Kaya
>
>
> From: Abdullah Sarp Kaya <akaya at expedia.com>
> Date: Monday, July 4, 2016 at 9:59 PM
> To: "stian at redhat.com" <stian at redhat.com>
> Cc: "keycloak-user at lists.jboss.org" <keycloak-user at lists.jboss.org>
> Subject: Re: [keycloak-user] Lost session when removing an instance off
> cluster
>
> Hello,
>
> Below is exactly how Infinispan is configured:
>
> <subsystem xmlns="urn:jboss:domain:infinispan:4.0">
> <cache-container name="keycloak" jndi-name="infinispan/Keycloak">
> <transport lock-timeout="60000"/>
> <invalidation-cache name="realms" mode="SYNC"/>
> <invalidation-cache name="users" mode="SYNC">
> <eviction max-entries="10000" strategy="LRU"/>
> </invalidation-cache>
> <distributed-cache xmlns="" name="sessions" mode="SYNC">
> <transaction mode="NON_DURABLE_XA"/>
> </distributed-cache>
> <local-cache xmlns="" name="offlineSessions"/>
> <local-cache xmlns="" name="loginFailures"/>
> <replicated-cache name="work" mode="SYNC"/>
> <local-cache name="realmVersions">
> <transaction mode="BATCH" locking="PESSIMISTIC"/>
> </local-cache>
> </cache-container>
> <cache-container name="server" aliases="singleton cluster" default-cache="default" module="org.wildfly.clustering.server">
> <transport lock-timeout="60000"/>
> <replicated-cache name="default" mode="SYNC">
> <transaction mode="BATCH"/>
> </replicated-cache>
> </cache-container>
> <cache-container name="web" default-cache="dist" module="org.wildfly.clustering.web.infinispan">
> <transport lock-timeout="60000"/>
> <distributed-cache name="dist" mode="ASYNC" l1-lifespan="0" owners="2">
> <locking isolation="REPEATABLE_READ"/>
> <transaction mode="BATCH"/>
> <file-store/>
> </distributed-cache>
> </cache-container>
> <cache-container name="ejb" aliases="sfsb" default-cache="dist" module="org.wildfly.clustering.ejb.infinispan">
> <transport lock-timeout="60000"/>
> <distributed-cache name="dist" mode="ASYNC" l1-lifespan="0" owners="2">
> <locking isolation="REPEATABLE_READ"/>
> <transaction mode="BATCH"/>
> <file-store/>
> </distributed-cache>
> </cache-container>
> <cache-container name="hibernate" default-cache="local-query" module="org.hibernate.infinispan">
> <transport lock-timeout="60000"/>
> <invalidation-cache name="entity" mode="SYNC">
> <transaction mode="NON_XA"/>
> <eviction strategy="LRU" max-entries="10000"/>
> <expiration max-idle="100000"/>
> </invalidation-cache>
> <local-cache name="local-query">
> <eviction strategy="LRU" max-entries="10000"/>
> <expiration max-idle="100000"/>
> </local-cache>
> <replicated-cache name="timestamps" mode="ASYNC"/>
> </cache-container>
> </subsystem>
>
>
>
>
> This is the complete exception:
>
> 03:49:40,241 ERROR
> [org.infinispan.interceptors.InvocationContextInterceptor] (default
> task-12) ISPN000136: Error executing command PutKeyValueCommand, writing
> keys [ba74e523-3299-42b3-801e-9f8cb5a0d81d]:
> org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> 03:49:40,250 ERROR [org.jboss.resteasy.resteasy_jaxrs.i18n] (default
> task-12) RESTEASY002025: Unknown exception while executing GET
> /realms/partner-dmz/protocol/openid-connect/auth:
> org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> 03:49:40,257 ERROR [io.undertow.request] (default task-12) UT005023:
> Exception handling request to
> /auth/realms/partner-dmz/protocol/openid-connect/auth:
> org.jboss.resteasy.spi.UnhandledException:
> org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
> at
> org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
> at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at
> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
> at
> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
> at
> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
> at
> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
> at
> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> at
> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> at
> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> at
> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> at
> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> at
> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> at
> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
> at
> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> at
> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
> at
> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
> at
> io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
> at
> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.infinispan.util.concurrent.TimeoutException: Replication
> timeout for kc2
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> ... 3 more
>
> 03:49:41,251 ERROR
> [org.infinispan.interceptors.InvocationContextInterceptor] (default task-3)
> ISPN000136: Error executing command PrepareCommand, writing keys
> [02cff8bb-0ab9-4ab9-905f-17ffb3226050]:
> org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> 03:49:41,258 ERROR
> [org.infinispan.transaction.impl.TransactionCoordinator] (default task-3)
> ISPN000097: Error while processing a prepare in a single-phase transaction:
> org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> 03:49:45,504 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel
> server: [kc1|4] (1) [kc1]
> 03:49:45,507 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel
> keycloak: [kc1|4] (1) [kc1]
> 03:49:45,534 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel web:
> [kc1|4] (1) [kc1]
> 03:49:45,538 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel ejb:
> [kc1|4] (1) [kc1]
> 03:49:45,548 WARN [com.arjuna.ats.jta] (default task-3) ARJUNA016039:
> onePhaseCommit on < formatId=131077, gtrid_length=29, bqual_length=36,
> tx_uid=0:ffffac110003:-7fcaa87f:5779dc60:f01, node_name=1,
> branch_uid=0:ffffac110003:-7fcaa87f:5779dc60:f02, subordinatenodename=null,
> eis_name=unknown eis name >
> (TransactionXaAdapter{localTransaction=LocalXaTransaction{xid=<
> formatId=131077, gtrid_length=29, bqual_length=36,
> tx_uid=0:ffffac110003:-7fcaa87f:5779dc60:f01, node_name=1,
> branch_uid=0:ffffac110003:-7fcaa87f:5779dc60:f02, subordinatenodename=null,
> eis_name=unknown eis name >} LocalTransaction{remoteLockedNodes=null,
> isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null,
> topologyId=6, stateTransferFlag=null}
> org.infinispan.transaction.xa.LocalXaTransaction at 3ef}) failed with
> exception XAException.XA_HEURRB: javax.transaction.xa.XAException
> at
> org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
> at
> org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
> at
> org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
> at
> com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
> at
> com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
> at
> com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
> at
> com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at
> com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at
> org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
> at
> org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
> at
> org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
> at
> org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
> at
> org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
> at
> org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
> at
> org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
> at
> org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
> at
> org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
> at
> org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
> at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at
> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
> at
> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
> at
> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
> at
> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
> at
> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> at
> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> at
> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> at
> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> at
> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> at
> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> at
> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
> at
> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> at
> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
> at
> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
> at
> io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
> at
> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.infinispan.util.concurrent.TimeoutException: Replication
> timeout for kc2
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> ... 3 more
>
> 03:49:45,563 WARN [org.infinispan.cache.impl.CacheImpl] (default task-3)
> ISPN000160: Could not complete injected transaction.:
> javax.transaction.RollbackException: ARJUNA016053: Could not commit
> transaction.
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at
> com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at
> org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
> at
> org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
> at
> org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
> at
> org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
> at
> org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
> at
> org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
> at
> org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
> at
> org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
> at
> org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
> at
> org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
> at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at
> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
> at
> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
> at
> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
> at
> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
> at
> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> at
> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> at
> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> at
> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> at
> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> at
> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> at
> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
> at
> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> at
> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
> at
> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
> at
> io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
> at
> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Suppressed: javax.transaction.xa.XAException
> at
> org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
> at
> org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
> at
> org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
> at
> com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
> at
> com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
> at
> com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
> at
> com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
> at
> com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
> ... 54 more
> Caused by: org.infinispan.util.concurrent.TimeoutException:
> Replication timeout for kc2
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> ... 3 more
>
> 03:49:45,575 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel
> hibernate: [kc1|4] (1) [kc1]
> 03:49:45,575 ERROR [org.jboss.resteasy.resteasy_jaxrs.i18n] (default
> task-3) RESTEASY002025: Unknown exception while executing GET
> /realms/partner-dmz/protocol/openid-connect/auth:
> org.infinispan.commons.CacheException: Could not commit implicit transaction
> at
> org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1725)
> at
> org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
> at
> org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
> at
> org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
> at
> org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
> at
> org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
> at
> org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
> at
> org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
> at
> org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
> at
> org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
> at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at
> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
> at
> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
> at
> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
> at
> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
> at
> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> at
> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> at
> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> at
> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> at
> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> at
> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> at
> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
> at
> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> at
> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
> at
> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
> at
> io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
> at
> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not
> commit transaction.
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at
> com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at
> org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
> ... 51 more
> Suppressed: javax.transaction.xa.XAException
> at
> org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
> at
> org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
> at
> org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
> at
> com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
> at
> com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
> at
> com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
> at
> com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
> at
> com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
> ... 54 more
> Caused by: org.infinispan.util.concurrent.TimeoutException:
> Replication timeout for kc2
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> ... 3 more
>
> 03:49:45,596 ERROR [io.undertow.request] (default task-3) UT005023:
> Exception handling request to
> /auth/realms/partner-dmz/protocol/openid-connect/auth:
> org.jboss.resteasy.spi.UnhandledException:
> org.infinispan.commons.CacheException: Could not commit implicit transaction
> at
> org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
> at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at
> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
> at
> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
> at
> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
> at
> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
> at
> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
> at
> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> at
> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> at
> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> at
> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> at
> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> at
> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> at
> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
> at
> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> at
> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
> at
> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
> at
> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
> at
> io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
> at
> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.infinispan.commons.CacheException: Could not commit
> implicit transaction
> at
> org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1725)
> at
> org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
> at
> org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
> at
> org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
> at
> org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
> at
> org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
> at
> org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
> at
> org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
> at
> org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
> at
> org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
> ... 38 more
> Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not
> commit transaction.
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at
> com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at
> org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
> ... 51 more
> Suppressed: javax.transaction.xa.XAException
> at
> org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
> at
> org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
> at
> org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
> at
> com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
> at
> com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
> at
> com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
> at
> com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
> at
> com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at
> com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
> ... 54 more
> Caused by: org.infinispan.util.concurrent.TimeoutException:
> Replication timeout for kc2
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> ... 3 more
>
>
>
> Here kc2 is the removed instance.
>
> Kind Regards,
> Sarp Kaya
>
>
> From: Stian Thorgersen <sthorger at redhat.com>
> Reply-To: "stian at redhat.com" <stian at redhat.com>
> Date: Friday, July 1, 2016 at 7:17 PM
> To: Abdullah Sarp Kaya <akaya at expedia.com>
> Cc: "keycloak-user at lists.jboss.org" <keycloak-user at lists.jboss.org>
> Subject: Re: [keycloak-user] Lost session when removing an instance off
> cluster
>
> Can you please include more details from the log if there is any, at least
> a full stack trace and not just the bit you've included. We also need to
> know details around how you've configured the caches and Infinispan.
>
> On 1 July 2016 at 10:16, Sarp Kaya <akaya at expedia.com> wrote:
>
>> Hello,
>>
>> I have tried various ways of configuring infinispan but it just seems
>> like if I deploy a new instance to the cluster and remove one, then some
>> sessions are lost and an exception is thrown saying that it was not
>> handled. This is the Infinispan exception:
>>
>> Exceptionhandlingrequestto/auth/realms/realmname/protocol/openid-connect/
>> auth:org.jboss.resteasy.spi.UnhandledException:org.infinispan.util.
>> concurrent.TimeoutException:Replicationtimeoutfor79a0757ecab3atorg.jboss.
>> resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
>> atorg.jboss.resteasy.core.SynchronousDispatcher.writeException(
>> SynchronousDispatcher.java:168) atorg.jboss.resteasy.core.
>> SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471) atorg
>> .jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.
>> java:415) atorg.jboss.resteasy.core.SynchronousDispatcher.invoke(
>> SynchronousDispatcher.java:202) atorg.jboss.resteasy.plugins.server.
>> servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.
>> java:221) atorg.jboss.resteasy.plugins.server.servlet.
>> HttpServletDispatcher.service(HttpServletDispatcher.java:56) atorg.jboss.
>> resteasy.plugins.server.servlet.HttpServletDispatcher.service(
>> HttpServletDispatcher.java:51) atjavax.servlet.http.HttpServlet.service(
>> HttpServlet.java:790) atio.undertow.servlet.handlers.ServletHandler.
>> handleRequest(ServletHandler.java:85) atio.undertow.servlet.handlers.
>> FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) atorg.
>> keycloak.services.filters.KeycloakSessionServletFilter.doFilter(
>> KeycloakSessionServletFilter.java:88) atio.undertow.servlet.core.
>> ManagedFilter.doFilter(ManagedFilter.java:60) atio.undertow.servlet.
>> handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>> atio.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.
>> java:84) atio.undertow.servlet.handlers.security.
>> ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:
>> 62) atio.undertow.servlet.handlers.ServletDispatchingHandler.
>> handleRequest(ServletDispatchingHandler.java:36) atorg.wildfly.extension.
>> undertow.security.SecurityContextAssociationHandler.handleRequest(
>> SecurityContextAssociationHandler.java:78) atio.undertow.server.handlers.
>> PredicateHandler.handleRequest(PredicateHandler.java:43) atio.undertow.
>> servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(
>> SSLInformationAssociationHandler.java:131) atio.undertow.servlet.handlers
>> .security.ServletAuthenticationCallHandler.handleRequest(
>> ServletAuthenticationCallHandler.java:57) atio.undertow.server.handlers.
>> PredicateHandler.handleRequest(PredicateHandler.java:43) atio.undertow.
>> security.handlers.AbstractConfidentialityHandler.handleRequest(
>> AbstractConfidentialityHandler.java:46) atio.undertow.servlet.handlers.
>> security.ServletConfidentialityConstraintHandler.handleRequest(
>> ServletConfidentialityConstraintHandler.java:64) atio.undertow.security.
>> handlers.AuthenticationMechanismsHandler.handleRequest(
>> AuthenticationMechanismsHandler.java:60) atio.undertow.servlet.handlers.
>> security.CachedAuthenticatedSessionHandler.handleRequest(
>> CachedAuthenticatedSessionHandler.java:77) atio.undertow.security.
>> handlers.NotificationReceiverHandler.handleRequest(
>> NotificationReceiverHandler.java:50) atio.undertow.security.handlers.
>> AbstractSecurityContextAssociationHandler.handleRequest(
>> AbstractSecurityContextAssociationHandler.java:43) atio.undertow.server.
>> handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) atorg.
>> wildfly.extension.undertow.security.jacc.JACCContextIdHandler.
>> handleRequest(JACCContextIdHandler.java:61) atio.undertow.server.handlers
>> .PredicateHandler.handleRequest(PredicateHandler.java:43) atio.undertow.
>> server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>> atio.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(
>> ServletInitialHandler.java:284) atio.undertow.servlet.handlers.
>> ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263) at
>> io.undertow.servlet.handlers.ServletInitialHandler.access$000(
>> ServletInitialHandler.java:81) atio.undertow.servlet.handlers.
>> ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174) at
>> io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at
>> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
>> atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
>> java:1142) atjava.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:617) atjava.lang.Thread.run(Thread.java:745)
>> Causedby:org.infinispan.util.concurrent.TimeoutException:Replication
>> timeoutfor79a0757ecab3atorg.infinispan.remoting.transport.jgroups.
>> JGroupsTransport.checkRsp(JGroupsTransport.java:765) atorg.infinispan.
>> remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$72
>> (JGroupsTransport.java:599) atjava.util.concurrent.CompletableFuture.
>> uniApply(CompletableFuture.java:602) atjava.util.concurrent.
>> CompletableFuture$UniApply.tryFire(CompletableFuture.java:577) atjava.
>> util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
>> atjava.util.concurrent.CompletableFuture.complete(CompletableFuture.java:
>> 1962) atorg.infinispan.remoting.transport.jgroups.SingleResponseFuture.
>> call(SingleResponseFuture.java:46) atorg.infinispan.remoting.transport.
>> jgroups.SingleResponseFuture.call(SingleResponseFuture.java:17) atjava.
>> util.concurrent.FutureTask.run(FutureTask.java:266) atjava.util.
>> concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(
>> ScheduledThreadPoolExecutor.java:180)
>>
>> This causes browsers to see Internal Server Error. Shouldn’t that be
>> handled in Keycloak as lost session, therefore KC should try to handle it
>> rather than showing that it’s an Internal Server Error?
>>
>> My current infinispan configuration looks like this:
>>
>> <distributed-cache name="sessions" mode="SYNC">
>> <transaction mode="NON_DURABLE_XA"/>
>> </distributed-cache>
>>
>>
>> I use Keycloak version 1.9.5. My question is am I doing something wrong
>> with my configuration? I tried both replicated-cache and distributed-cache
>> and tried all transaction mode on both of them. None of them seems to solve
>> the error that I’ve had above.
>>
>> Kind Regards,
>> Sarp Kaya
>>
>>
>>
>>
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160705/629b6321/attachment-0001.html
More information about the keycloak-user
mailing list