Hi Stian,
I have tested with owners being 4 and had 4 instances, removed 1 off and still had the
issue. I’ll try it with version 2.0.0 with 4 owners in sessions and the rest with 1
owners. I don’t have any network issues because it’s all tested locally.
Kind Regards,
Sarp Kaya
From: Stian Thorgersen <sthorger@redhat.com<mailto:sthorger@redhat.com>>
Reply-To: "stian@redhat.com<mailto:stian@redhat.com>"
<stian@redhat.com<mailto:stian@redhat.com>>
Date: Tuesday, July 5, 2016 at 4:58 PM
To: Abdullah Sarp Kaya <akaya@expedia.com<mailto:akaya@expedia.com>>
Cc: "keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org>"
<keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org>>
Subject: Re: [keycloak-user] Lost session when removing an instance off cluster
There probably will be some errors from Infinispan when you remove a node at least if
there's ongoing replication/messages being sent to the cluster.
Can you try with the latest Keycloak and with the default clustering config and only
increasing owners? We have tested session failover ourselves and it was working fine. So
could be down to the way you've configured it or could be an issue due to your network
setup.
On 5 July 2016 at 08:50, Sarp Kaya
<akaya@expedia.com<mailto:akaya@expedia.com>> wrote:
Hi Stian,
You are right, it needs to have owner. By default the configuration had owners but as I
experimented I forgot to add it. However adding owners does not fix the problem.
Initially I also experimented this with offlineSessions and loginFailures being
distributed-cache so logins were already failing with that (same issue, Internal Server
Error being presented).
Anyway, if worse case user needs to re-authenticate, then Internal Server Error problem
should be fixed right? (it’s the below exception I have pasted) Because whenever there is
org.jboss.resteasy.spi.UnhandledException then browsers will see 500, instead of logging
in again (there could be something added as such as handling this exception and asking
user to log in again).
Kind Regards,
Sarp Kaya
From: Stian Thorgersen <sthorger@redhat.com<mailto:sthorger@redhat.com>>
Reply-To: "stian@redhat.com<mailto:stian@redhat.com>"
<stian@redhat.com<mailto:stian@redhat.com>>
Date: Tuesday, July 5, 2016 at 4:41 PM
To: Abdullah Sarp Kaya <akaya@expedia.com<mailto:akaya@expedia.com>>
Cc: "keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org>"
<keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org>>
Subject: Re: [keycloak-user] Lost session when removing an instance off cluster
The sessions cache is a distributed-cache, but by default it only has 1 owner of each
segment. This means that sessions are not replicated to multiple nodes. So it's
expected that you loose sessions when a node is removed. If you want sessions to survive
the removal of a node you should increase the number of owners for the cache. For
example:
<distributed-cache name="sessions" mode="SYNC"
owners="2"/>
We've not actually tried with enabling transactions on the sessions, but I would say
performance is more important. Worst case scenario is that a session is lost and a user
has to re-authenticate.
BTW the config you have for offlineSessions and loginFailures are wrong they both need to
be distributed caches as well in clustered mode:
<distributed-cache name="offlineSessions" mode="SYNC"
owners="1"/>
<distributed-cache name="loginFailures" mode="SYNC"
owners="1"/>
On 4 July 2016 at 15:45, Sarp Kaya
<akaya@expedia.com<mailto:akaya@expedia.com>> wrote:
Just to add on this.
I think I misunderstood something or Keycloak does not really implement the infinispan
caching lifetime correctly.
So first thing I did is to TRACE the infinispan logs. So I added this logger in
standalone.xml:
<logger category="org.infinispan">
<level name="TRACE"/>
</logger>
The next thing I did is to do the timing. Basically my test is that what happens when I
log in. I also enabled events and printing the events as well.
What I found is:
1. Login event is triggered at 13:18:35,193 in Keycloak.
2. The browser got the full response at 13:18:35,273
3. Now the surprising part is the fact that there are whole a lot of infinispan logging
out occurring for “sessions” cache after the browser got the response. The last log
message occurring is like this:
2016-07-04 13:18:35,345 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (default
task-4) Response(s) to TxCompletionNotificationCommand{ xid=null, internalId=0,
topologyId=19, gtx=GlobalTransaction:<kc1>:239:local, cacheName=sessions} is {}
To see the all the logs after event trigger please check here:
<
http://pastebin.com/pkvYP07T><http://pastebin.com/pkvYP07T><h...
So my understanding was that I marked sessions as SYNC, which meant to me that it is
supposed to do all of the synchronization before it sends the response back to the
browser. If it does the other way around then it should be ASYNC mode. Could you please
clarify the intended/expected behavior?
Kind Regards,
Sarp Kaya
From: Abdullah Sarp Kaya <akaya@expedia.com<mailto:akaya@expedia.com>>
Date: Monday, July 4, 2016 at 9:59 PM
To: "stian@redhat.com<mailto:stian@redhat.com>"
<stian@redhat.com<mailto:stian@redhat.com>>
Cc: "keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org>"
<keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org>>
Subject: Re: [keycloak-user] Lost session when removing an instance off cluster
Hello,
Below is exactly how Infinispan is configured:
<subsystem xmlns="urn:jboss:domain:infinispan:4.0">
<cache-container name="keycloak"
jndi-name="infinispan/Keycloak">
<transport lock-timeout="60000"/>
<invalidation-cache name="realms" mode="SYNC"/>
<invalidation-cache name="users" mode="SYNC">
<eviction max-entries="10000" strategy="LRU"/>
</invalidation-cache>
<distributed-cache xmlns="" name="sessions"
mode="SYNC">
<transaction mode="NON_DURABLE_XA"/>
</distributed-cache>
<local-cache xmlns="" name="offlineSessions"/>
<local-cache xmlns="" name="loginFailures"/>
<replicated-cache name="work" mode="SYNC"/>
<local-cache name="realmVersions">
<transaction mode="BATCH" locking="PESSIMISTIC"/>
</local-cache>
</cache-container>
<cache-container name="server" aliases="singleton cluster"
default-cache="default" module="org.wildfly.clustering.server">
<transport lock-timeout="60000"/>
<replicated-cache name="default" mode="SYNC">
<transaction mode="BATCH"/>
</replicated-cache>
</cache-container>
<cache-container name="web" default-cache="dist"
module="org.wildfly.clustering.web.infinispan">
<transport lock-timeout="60000"/>
<distributed-cache name="dist" mode="ASYNC"
l1-lifespan="0" owners="2">
<locking isolation="REPEATABLE_READ"/>
<transaction mode="BATCH"/>
<file-store/>
</distributed-cache>
</cache-container>
<cache-container name="ejb" aliases="sfsb"
default-cache="dist"
module="org.wildfly.clustering.ejb.infinispan">
<transport lock-timeout="60000"/>
<distributed-cache name="dist" mode="ASYNC"
l1-lifespan="0" owners="2">
<locking isolation="REPEATABLE_READ"/>
<transaction mode="BATCH"/>
<file-store/>
</distributed-cache>
</cache-container>
<cache-container name="hibernate" default-cache="local-query"
module="org.hibernate.infinispan">
<transport lock-timeout="60000"/>
<invalidation-cache name="entity" mode="SYNC">
<transaction mode="NON_XA"/>
<eviction strategy="LRU" max-entries="10000"/>
<expiration max-idle="100000"/>
</invalidation-cache>
<local-cache name="local-query">
<eviction strategy="LRU" max-entries="10000"/>
<expiration max-idle="100000"/>
</local-cache>
<replicated-cache name="timestamps" mode="ASYNC"/>
</cache-container>
</subsystem>
This is the complete exception:
03:49:40,241 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default
task-12) ISPN000136: Error executing command PutKeyValueCommand, writing keys
[ba74e523-3299-42b3-801e-9f8cb5a0d81d]: org.infinispan.util.concurrent.TimeoutException:
Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
03:49:40,250 ERROR [org.jboss.resteasy.resteasy_jaxrs.i18n] (default task-12)
RESTEASY002025: Unknown exception while executing GET
/realms/partner-dmz/protocol/openid-connect/auth:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
03:49:40,257 ERROR [io.undertow.request] (default task-12) UT005023: Exception handling
request to /auth/realms/partner-dmz/protocol/openid-connect/auth:
org.jboss.resteasy.spi.UnhandledException:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
03:49:41,251 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default
task-3) ISPN000136: Error executing command PrepareCommand, writing keys
[02cff8bb-0ab9-4ab9-905f-17ffb3226050]: org.infinispan.util.concurrent.TimeoutException:
Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
03:49:41,258 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default
task-3) ISPN000097: Error while processing a prepare in a single-phase transaction:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
03:49:45,504 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel server: [kc1|4] (1)
[kc1]
03:49:45,507 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel keycloak: [kc1|4]
(1) [kc1]
03:49:45,534 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel web: [kc1|4] (1)
[kc1]
03:49:45,538 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel ejb: [kc1|4] (1)
[kc1]
03:49:45,548 WARN [com.arjuna.ats.jta] (default task-3) ARJUNA016039: onePhaseCommit on
< formatId=131077, gtrid_length=29, bqual_length=36,
tx_uid=0:ffffac110003:-7fcaa87f:5779dc60:f01, node_name=1,
branch_uid=0:ffffac110003:-7fcaa87f:5779dc60:f02, subordinatenodename=null,
eis_name=unknown eis name >
(TransactionXaAdapter{localTransaction=LocalXaTransaction{xid=< formatId=131077,
gtrid_length=29, bqual_length=36, tx_uid=0:ffffac110003:-7fcaa87f:5779dc60:f01,
node_name=1, branch_uid=0:ffffac110003:-7fcaa87f:5779dc60:f02, subordinatenodename=null,
eis_name=unknown eis name >} LocalTransaction{remoteLockedNodes=null,
isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, topologyId=6,
stateTransferFlag=null} org.infinispan.transaction.xa.LocalXaTransaction@3ef}) failed with
exception XAException.XA_HEURRB: javax.transaction.xa.XAException
at
org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
at
org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
at
org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
at org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
at
org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
at
org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
at
org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
at
org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
at
org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
at
org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.util.concurrent.TimeoutException: Replication timeout for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
03:49:45,563 WARN [org.infinispan.cache.impl.CacheImpl] (default task-3) ISPN000160:
Could not complete injected transaction.: javax.transaction.RollbackException:
ARJUNA016053: Could not commit transaction.
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
at org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
at
org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
at
org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
at
org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
at
org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
at
org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
at
org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Suppressed: javax.transaction.xa.XAException
at
org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
at
org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
at
org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
... 54 more
Caused by: org.infinispan.util.concurrent.TimeoutException: Replication timeout
for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
03:49:45,575 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-3,dev,kc1) ISPN000094: Received new cluster view for channel hibernate: [kc1|4]
(1) [kc1]
03:49:45,575 ERROR [org.jboss.resteasy.resteasy_jaxrs.i18n] (default task-3)
RESTEASY002025: Unknown exception while executing GET
/realms/partner-dmz/protocol/openid-connect/auth: org.infinispan.commons.CacheException:
Could not commit implicit transaction
at org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1725)
at
org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
at
org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
at
org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
at
org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
at
org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
at
org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit
transaction.
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
at org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
... 51 more
Suppressed: javax.transaction.xa.XAException
at
org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
at
org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
at
org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
... 54 more
Caused by: org.infinispan.util.concurrent.TimeoutException: Replication timeout
for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
03:49:45,596 ERROR [io.undertow.request] (default task-3) UT005023: Exception handling
request to /auth/realms/partner-dmz/protocol/openid-connect/auth:
org.jboss.resteasy.spi.UnhandledException: org.infinispan.commons.CacheException: Could
not commit implicit transaction
at
org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at
org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.commons.CacheException: Could not commit implicit transaction
at org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1725)
at
org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1121)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1111)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1742)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:248)
at
org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:291)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction$CacheTask.execute(InfinispanUserSessionProvider.java:854)
at
org.keycloak.models.sessions.infinispan.InfinispanUserSessionProvider$InfinispanKeycloakTransaction.commit(InfinispanUserSessionProvider.java:750)
at
org.keycloak.services.DefaultKeycloakTransactionManager.commit(DefaultKeycloakTransactionManager.java:104)
at
org.keycloak.services.filters.KeycloakTransactionCommitter.filter(KeycloakTransactionCommitter.java:43)
at
org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:121)
at
org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:48)
at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
... 38 more
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit
transaction.
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
at org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1722)
... 51 more
Suppressed: javax.transaction.xa.XAException
at
org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
at
org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
at
org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:112)
at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
at
com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
... 54 more
Caused by: org.infinispan.util.concurrent.TimeoutException: Replication timeout
for kc2
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
Here kc2 is the removed instance.
Kind Regards,
Sarp Kaya
From: Stian Thorgersen <sthorger@redhat.com<mailto:sthorger@redhat.com>>
Reply-To: "stian@redhat.com<mailto:stian@redhat.com>"
<stian@redhat.com<mailto:stian@redhat.com>>
Date: Friday, July 1, 2016 at 7:17 PM
To: Abdullah Sarp Kaya <akaya@expedia.com<mailto:akaya@expedia.com>>
Cc: "keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org>"
<keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org>>
Subject: Re: [keycloak-user] Lost session when removing an instance off cluster
Can you please include more details from the log if there is any, at least a full stack
trace and not just the bit you've included. We also need to know details around how
you've configured the caches and Infinispan.
On 1 July 2016 at 10:16, Sarp Kaya
<akaya@expedia.com<mailto:akaya@expedia.com>> wrote:
Hello,
I have tried various ways of configuring infinispan but it just seems like if I deploy a
new instance to the cluster and remove one, then some sessions are lost and an exception
is thrown saying that it was not handled. This is the Infinispan exception:
Exceptionhandlingrequestto/auth/realms/realmname/protocol/openid-connect/auth:org.jboss.resteasy.spi.UnhandledException:org.infinispan.util.concurrent.TimeoutException:Replicationtimeoutfor79a0757ecab3atorg.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
atorg.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)
atorg.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)
atorg.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
atorg.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
atorg.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
atorg.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
atorg.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
atjavax.servlet.http.HttpServlet.service(HttpServlet.java:790)
atio.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
atio.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
atorg.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:88)
atio.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
atio.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
atio.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
atio.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
atio.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
atorg.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
atio.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
atio.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
atio.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
atio.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
atio.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
atio.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
atio.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
atio.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
atio.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
atio.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
atio.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
atorg.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
atio.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
atio.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
atio.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
atio.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
atio.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
atio.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
atio.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
atio.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
atjava.lang.Thread.run(Thread.java:745)
Causedby:org.infinispan.util.concurrent.TimeoutException:Replicationtimeoutfor79a0757ecab3atorg.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
atorg.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$72(JGroupsTransport.java:599)
atjava.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
atjava.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
atjava.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
atjava.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
atorg.infinispan.remoting.transport.jgroups.SingleResponseFuture.call(SingleResponseFuture.java:46)
atorg.infinispan.remoting.transport.jgroups.SingleResponseFuture.call(SingleResponseFuture.java:17)
atjava.util.concurrent.FutureTask.run(FutureTask.java:266)
atjava.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
This causes browsers to see Internal Server Error. Shouldn’t that be handled in Keycloak
as lost session, therefore KC should try to handle it rather than showing that it’s an
Internal Server Error?
My current infinispan configuration looks like this:
<distributed-cache name="sessions" mode="SYNC">
<transaction mode="NON_DURABLE_XA"/>
</distributed-cache>
I use Keycloak version 1.9.5. My question is am I doing something wrong with my
configuration? I tried both replicated-cache and distributed-cache and tried all
transaction mode on both of them. None of them seems to solve the error that I’ve had
above.
Kind Regards,
Sarp Kaya
_______________________________________________
keycloak-user mailing list
keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-user