The sessions cache is a distributed-cache, but by default it only has 1
owner of each segment. This means that sessions are not replicated to
multiple nodes. So it's expected that you loose sessions when a node is
removed. If you want sessions to survive the removal of a node you should
increase the number of owners for the cache. For example:
<distributed-cache name="sessions" mode="SYNC"
owners="2"/>
We've not actually tried with enabling transactions on the sessions, but I
would say performance is more important. Worst case scenario is that a
session is lost and a user has to re-authenticate.
BTW the config you have for offlineSessions and loginFailures are wrong
they both need to be distributed caches as well in clustered mode:
<distributed-cache name="offlineSessions" mode="SYNC"
owners="1"/>
<distributed-cache name="loginFailures" mode="SYNC"
owners="1"/>
On 4 July 2016 at 15:45, Sarp Kaya <akaya(a)expedia.com> wrote:
Just to add on this.
I think I misunderstood something or Keycloak does not really implement
the infinispan caching lifetime correctly.
So first thing I did is to TRACE the infinispan logs. So I added this
logger in standalone.xml:
<logger category="org.infinispan">
<level name="TRACE"/>
</logger>
The next thing I did is to do the timing. Basically my test is that what
happens when I log in. I also enabled events and printing the events as
well.
What I found is:
1. Login event is triggered at 13:18:35,193 in Keycloak.
2. The browser got the full response at 13:18:35,273
3. Now the surprising part is the fact that there are whole a lot of
infinispan logging out occurring for “sessions” cache after the browser got
the response. The last log message occurring is like this:
2016-07-04 13:18:35,345 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (default
task-4) Response(s) to TxCompletionNotificationCommand{ xid=null, internalId=0,
topologyId=19, gtx=GlobalTransaction:<kc1>:239:local, cacheName=sessions} is {}
To see the all the logs after event trigger please check here:
<
http://pastebin.com/pkvYP07T> <
http://pastebin.com/pkvYP07T>
<
http://pastebin.com/pkvYP07T> <
http://pastebin.com/pkvYP07T>
http://pastebin.com/p4K2Ghff
So my understanding was that I marked sessions as SYNC, which meant to me
that it is supposed to do all of the synchronization before it sends the
response back to the browser. If it does the other way around then it
should be ASYNC mode. Could you please clarify the intended/expected
behavior?
Kind Regards,
Sarp Kaya
From: Abdullah Sarp Kaya <akaya(a)expedia.com>
Date: Monday, July 4, 2016 at 9:59 PM
To: "stian(a)redhat.com" <stian(a)redhat.com>
Cc: "keycloak-user(a)lists.jboss.org" <keycloak-user(a)lists.jboss.org>
Subject: Re: [keycloak-user] Lost session when removing an instance off
cluster
Hello,
Below is exactly how Infinispan is configured:
<subsystem xmlns="urn:jboss:domain:infinispan:4.0">
<cache-container name="keycloak"
jndi-name="infinispan/Keycloak">
<transport lock-timeout="60000"/>
<invalidation-cache name="realms" mode="SYNC"/>
<invalidation-cache name="users" mode="SYNC">
<eviction max-entries="10000" strategy="LRU"/>
</invalidation-cache>
<distributed-cache xmlns="" name="sessions"
mode="SYNC">
<transaction mode="NON_DURABLE_XA"/>
</distributed-cache>
<local-cache xmlns="" name="offlineSessions"/>
<local-cache xmlns="" name="loginFailures"/>
<replicated-cache name="work" mode="SYNC"/>
<local-cache name="realmVersions">
<transaction mode="BATCH" locking="PESSIMISTIC"/>
</local-cache>
</cache-container>
<cache-container name="server" aliases="singleton cluster"
default-cache="default" module="org.wildfly.clustering.server">
<transport lock-timeout="60000"/>
<replicated-cache name="default" mode="SYNC">
<transaction mode="BATCH"/>
</replicated-cache>
</cache-container>
<cache-container name="web" default-cache="dist"
module="org.wildfly.clustering.web.infinispan">
<transport lock-timeout="60000"/>
<distributed-cache name="dist" mode="ASYNC"
l1-lifespan="0" owners="2">
<locking isolation="REPEATABLE_READ"/>
<transaction mode="BATCH"/>
<file-store/>
</distributed-cache>
</cache-container>
<cache-container name="ejb" aliases="sfsb"
default-cache="dist"
module="org.wildfly.clustering.ejb.infinispan">
<transport lock-timeout="60000"/>
<distributed-cache name="dist" mode="ASYNC"
l1-lifespan="0" owners="2">
<locking isolation="REPEATABLE_READ"/>
<transaction mode="BATCH"/>
<file-store/>
</distributed-cache>
</cache-container>
<cache-container name="hibernate" default-cache="local-query"
module="org.hibernate.infinispan">
<transport lock-timeout="60000"/>
<invalidation-cache name="entity" mode="SYNC">
<transaction mode="NON_XA"/>
<eviction strategy="LRU" max-entries="10000"/>
<expiration max-idle="100000"/>
</invalidation-cache>
<local-cache name="local-query">
<eviction strategy="LRU" max-entries="10000"/>
<expiration max-idle="100000"/>
</local-cache>
<replicated-cache name="timestamps" mode="ASYNC"/>
</cache-container>
</subsystem>
This is the complete exception:
03:49:40,241 ERROR
[org.infinispan.interceptors.InvocationContextInterceptor] (default
task-12) ISPN000136: Error executing command PutKeyValueCommand, writing
keys [ba74e523-3299-42b3-801e-9f8cb5a0d81d]:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
03:49:40,250 ERROR [org.jboss.resteasy.resteasy_jaxrs.i18n] (default
task-12) RESTEASY002025: Unknown exception while executing GET
/realms/partner-dmz/protocol/openid-connect/auth:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
03:49:40,257 ERROR [io.undertow.request] (default task-12) UT005023:
Exception handling request to
/auth/realms/partner-dmz/protocol/openid-connect/auth:
org.jboss.resteasy.spi.UnhandledException:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at
io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at
io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at
io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.util.concurrent.TimeoutException: Replication
timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
03:49:41,251 ERROR
[org.infinispan.interceptors.InvocationContextInterceptor] (default task-3)
ISPN000136: Error executing command PrepareCommand, writing keys
[02cff8bb-0ab9-4ab9-905f-17ffb3226050]:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
03:49:41,258 ERROR
[org.infinispan.transaction.impl.TransactionCoordinator] (default task-3)
ISPN000097: Error while processing a prepare in a single-phase transaction:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
03:49:45,504 INFO
[org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel
server: [kc1|4] (1) [kc1]
03:49:45,507 INFO
[org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel
keycloak: [kc1|4] (1) [kc1]
03:49:45,534 INFO
[org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel web:
[kc1|4] (1) [kc1]
03:49:45,538 INFO
[org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel ejb:
[kc1|4] (1) [kc1]
03:49:45,548 WARN [com.arjuna.ats.jta] (default task-3) ARJUNA016039:
onePhaseCommit on < formatId=131077, gtrid_length=29, bqual_length=36,
tx_uid=0:ffffac110003:-7fcaa87f:5779dc60:f01, node_name=1,
branch_uid=0:ffffac110003:-7fcaa87f:5779dc60:f02, subordinatenodename=null,
eis_name=unknown eis name >
(TransactionXaAdapter{localTransaction=LocalXaTransaction{xid=<
formatId=131077, gtrid_length=29, bqual_length=36,
tx_uid=0:ffffac110003:-7fcaa87f:5779dc60:f01, node_name=1,
branch_uid=0:ffffac110003:-7fcaa87f:5779dc60:f02, subordinatenodename=null,
eis_name=unknown eis name >} LocalTransaction{remoteLockedNodes=null,
isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null,
topologyId=6, stateTransferFlag=null}
org.infinispan.transaction.xa.LocalXaTransaction@3ef}) failed with
exception XAException.XA_HEURRB: javax.transaction.xa.XAException
at
org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
at
org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
at
org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
at
org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
at
org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
at
org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
at
org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
at
org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
at
org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
at
org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
at
org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at
io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at
io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at
io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.util.concurrent.TimeoutException: Replication
timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
03:49:45,563 WARN [org.infinispan.cache.impl.CacheImpl] (default task-3)
ISPN000160: Could not complete injected transaction.:
javax.transaction.RollbackException: ARJUNA016053: Could not commit
transaction.
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
at
org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
at
org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
at
org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
at
org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
at
org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
at
org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
at
org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
at
org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at
io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at
io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at
io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Suppressed: javax.transaction.xa.XAException
at
org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
at
org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
at
org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
at
com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
... 54 more
Caused by: org.infinispan.util.concurrent.TimeoutException:
Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
03:49:45,575 INFO
[org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel
hibernate: [kc1|4] (1) [kc1]
03:49:45,575 ERROR [org.jboss.resteasy.resteasy_jaxrs.i18n] (default
task-3) RESTEASY002025: Unknown exception while executing GET
/realms/partner-dmz/protocol/openid-connect/auth:
org.infinispan.commons.CacheException: Could not commit implicit transaction
at
org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1725)
at
org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
at
org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
at
org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
at
org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
at
org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
at
org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
at
org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at
io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at
io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at
io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not
commit transaction.
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
at
org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
... 51 more
Suppressed: javax.transaction.xa.XAException
at
org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
at
org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
at
org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
at
com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
... 54 more
Caused by: org.infinispan.util.concurrent.TimeoutException:
Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
03:49:45,596 ERROR [io.undertow.request] (default task-3) UT005023:
Exception handling request to
/auth/realms/partner-dmz/protocol/openid-connect/auth:
org.jboss.resteasy.spi.UnhandledException:
org.infinispan.commons.CacheException: Could not commit implicit transaction
at
org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at
io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at
io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at
io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.commons.CacheException: Could not commit
implicit transaction
at
org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1725)
at
org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
at
org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
at
org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
at
org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
at
org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
at
org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
at
org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
... 38 more
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not
commit transaction.
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
at
org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
... 51 more
Suppressed: javax.transaction.xa.XAException
at
org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
at
org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
at
org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
at
com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
... 54 more
Caused by: org.infinispan.util.concurrent.TimeoutException:
Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
Here kc2 is the removed instance.
Kind Regards,
Sarp Kaya
From: Stian Thorgersen <sthorger(a)redhat.com>
Reply-To: "stian(a)redhat.com" <stian(a)redhat.com>
Date: Friday, July 1, 2016 at 7:17 PM
To: Abdullah Sarp Kaya <akaya(a)expedia.com>
Cc: "keycloak-user(a)lists.jboss.org" <keycloak-user(a)lists.jboss.org>
Subject: Re: [keycloak-user] Lost session when removing an instance off
cluster
Can you please include more details from the log if there is any, at least
a full stack trace and not just the bit you've included. We also need to
know details around how you've configured the caches and Infinispan.
On 1 July 2016 at 10:16, Sarp Kaya <akaya(a)expedia.com> wrote:
> Hello,
>
> I have tried various ways of configuring infinispan but it just seems
> like if I deploy a new instance to the cluster and remove one, then some
> sessions are lost and an exception is thrown saying that it was not
> handled. This is the Infinispan exception:
>
> Exceptionhandlingrequestto/auth/realms/realmname/protocol/openid-connect/
> auth:org.jboss.resteasy.spi.UnhandledException:org.infinispan.util.
> concurrent.TimeoutException:Replicationtimeoutfor79a0757ecab3atorg.jboss.
> resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
> atorg.jboss.resteasy.core.SynchronousDispatcher.writeException(
> SynchronousDispatcher.java:168) atorg.jboss.resteasy.core.
> SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471) atorg
> .jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.
> java:415) atorg.jboss.resteasy.core.SynchronousDispatcher.invoke(
> SynchronousDispatcher.java:202) atorg.jboss.resteasy.plugins.server.
> servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.
> java:221) atorg.jboss.resteasy.plugins.server.servlet.
> HttpServletDispatcher.service(HttpServletDispatcher.java:56) atorg.jboss.
> resteasy.plugins.server.servlet.HttpServletDispatcher.service(
> HttpServletDispatcher.java:51) atjavax.servlet.http.HttpServlet.service(
> HttpServlet.java:790) atio.undertow.servlet.handlers.ServletHandler.
> handleRequest(ServletHandler.java:85) atio.undertow.servlet.handlers.
> FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) atorg.
> keycloak.services.filters.KeycloakSessionServletFilter.doFilter(
> KeycloakSessionServletFilter.java:88) atio.undertow.servlet.core.
> ManagedFilter.doFilter(ManagedFilter.java:60) atio.undertow.servlet.
> handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
> atio.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.
> java:84) atio.undertow.servlet.handlers.security.
> ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:
> 62) atio.undertow.servlet.handlers.ServletDispatchingHandler.
> handleRequest(ServletDispatchingHandler.java:36) atorg.wildfly.extension.
> undertow.security.SecurityContextAssociationHandler.handleRequest(
> SecurityContextAssociationHandler.java:78) atio.undertow.server.handlers.
> PredicateHandler.handleRequest(PredicateHandler.java:43) atio.undertow.
> servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(
> SSLInformationAssociationHandler.java:131) atio.undertow.servlet.handlers
> .security.ServletAuthenticationCallHandler.handleRequest(
> ServletAuthenticationCallHandler.java:57) atio.undertow.server.handlers.
> PredicateHandler.handleRequest(PredicateHandler.java:43) atio.undertow.
> security.handlers.AbstractConfidentialityHandler.handleRequest(
> AbstractConfidentialityHandler.java:46) atio.undertow.servlet.handlers.
> security.ServletConfidentialityConstraintHandler.handleRequest(
> ServletConfidentialityConstraintHandler.java:64) atio.undertow.security.
> handlers.AuthenticationMechanismsHandler.handleRequest(
> AuthenticationMechanismsHandler.java:60) atio.undertow.servlet.handlers.
> security.CachedAuthenticatedSessionHandler.handleRequest(
> CachedAuthenticatedSessionHandler.java:77) atio.undertow.security.
> handlers.NotificationReceiverHandler.handleRequest(
> NotificationReceiverHandler.java:50) atio.undertow.security.handlers.
> AbstractSecurityContextAssociationHandler.handleRequest(
> AbstractSecurityContextAssociationHandler.java:43) atio.undertow.server.
> handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) atorg.
> wildfly.extension.undertow.security.jacc.JACCContextIdHandler.
> handleRequest(JACCContextIdHandler.java:61) atio.undertow.server.handlers
> .PredicateHandler.handleRequest(PredicateHandler.java:43) atio.undertow.
> server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> atio.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(
> ServletInitialHandler.java:284) atio.undertow.servlet.handlers.
> ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263) at
> io.undertow.servlet.handlers.ServletInitialHandler.access$000(
> ServletInitialHandler.java:81) atio.undertow.servlet.handlers.
> ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174) at
> io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at
> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
> atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> java:1142) atjava.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617) atjava.lang.Thread.run(Thread.java:745)
> Causedby:org.infinispan.util.concurrent.TimeoutException:Replication
> timeoutfor79a0757ecab3atorg.infinispan.remoting.transport.jgroups.
> JGroupsTransport.checkRsp(JGroupsTransport.java:765) atorg.infinispan.
> remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$72
> (JGroupsTransport.java:599) atjava.util.concurrent.CompletableFuture.
> uniApply(CompletableFuture.java:602) atjava.util.concurrent.
> CompletableFuture$UniApply.tryFire(CompletableFuture.java:577) atjava.
> util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> atjava.util.concurrent.CompletableFuture.complete(CompletableFuture.java:
> 1962) atorg.infinispan.remoting.transport.jgroups.SingleResponseFuture.
> call(SingleResponseFuture.java:46) atorg.infinispan.remoting.transport.
> jgroups.SingleResponseFuture.call(SingleResponseFuture.java:17) atjava.
> util.concurrent.FutureTask.run(FutureTask.java:266) atjava.util.
> concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(
> ScheduledThreadPoolExecutor.java:180)
>
> This causes browsers to see Internal Server Error. Shouldn’t that be
> handled in Keycloak as lost session, therefore KC should try to handle it
> rather than showing that it’s an Internal Server Error?
>
> My current infinispan configuration looks like this:
>
> <distributed-cache name="sessions" mode="SYNC">
> <transaction mode="NON_DURABLE_XA"/>
> </distributed-cache>
>
>
> I use Keycloak version 1.9.5. My question is am I doing something wrong
> with my configuration? I tried both replicated-cache and distributed-cache
> and tried all transaction mode on both of them. None of them seems to solve
> the error that I’ve had above.
>
> Kind Regards,
> Sarp Kaya
>
>
>
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/keycloak-user
>