[keycloak-user] Handling SuspectExceptions in Keycloak
Marek Posolda
mposolda at redhat.com
Thu Aug 4 05:50:16 EDT 2016
Hmm... so according to
https://docs.jboss.org/author/display/WFLY10/Infinispan+Subsystem it
seems that you're right. It's not easily possible to add the interceptor
through infinispan subsystem :/
As a workaround, you can probably try to do it programatically. You may
need to create your own InfinispanConnectionProviderFactory and
configure it in keycloak-server.json . It can override
DefaultInfinispanConnectionProviderFactory and add the interceptor
programatically to realms and users caches. Sorry, don't have better
proposal to avoid this issue right now :( We likely need to wait until
https://issues.jboss.org/browse/ISPN-6857 is fixed...
Marek
On 02/08/16 06:32, Sarp Kaya wrote:
>
> Hi Marek,
>
> How do I add the StateTransferInterceptor to the standalone.xml? Isn’t
> that only doable programmatically?
>
> Thanks,
> Sarp
>
> *From: *Marek Posolda <mposolda at redhat.com>
> *Date: *Tuesday, August 2, 2016 at 5:32 AM
> *To: *Abdullah Sarp <akaya at expedia.com>,
> "keycloak-user at lists.jboss.org" <keycloak-user at lists.jboss.org>
> *Subject: *Re: [keycloak-user] Handling SuspectExceptions in Keycloak
>
> See KC issue [1] and related infinispan issue [2] .
>
> The workaround is to add the StateTransferInterceptor to the proper
> place in chain to "realms" and "users" caches. See how I did it
> programatically. I think that based on that, you should be able to add
> it to infinispan subsystem as well.
>
> [1] https://issues.jboss.org/browse/KEYCLOAK-3306
> [2] https://issues.jboss.org/browse/ISPN-6857
>
> Marek
>
> On 28/07/16 11:53, Sarp Kaya wrote:
>
> Hello,
>
> There is already an existing bug report for Infinispan here:
>
> https://issues.jboss.org/browse/ISPN-6721
>
> Currently for Keycloak, if this exception is thrown then it sends
> an Internal Server Error page to the browser. Essentially what
> would be really good is that it sends the user back to the login
> page instead of displaying Internal Server Error.
>
> This happens when I am consistently sending login and logout
> (around 40 req/s) requests to two Keycloak instances (let’s call
> them kc1 and kc2), then one new keycloak instance is started kc3.
> Kc3 connects to kc1 and 2 in clustering mode.
>
> Now kc1 receives a new request (such as login) and while it is
> processing that, kc3 is gracefully shut including the cache with
> this log:
>
> 2016-07-28 09:15:53,656 INFO [org.jboss.as.clustering.infinispan]
> (ServerService Thread Pool -- 61) WFLYCLINF0003: Stopped sessions
> cache from keycloak container
>
> Just shortly after that (6 ms) kc1 throws an exception like this:
>
> 2016-07-28 09:15:53,662 ERROR [io.undertow.request] (default
> task-48) UT005023: Exception handling request to
> /auth/realms/{realm}/login-actions/authenticate:
> org.jboss.resteasy.spi.UnhandledException:
> org.infinispan.statetransfer.OutdatedTopologyException: Cache
> topology changed while the command was executing: expected 175,
> got 176
>
> at
> org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
>
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)
>
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)
>
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
>
> then shortly after(150 ms) kc1 wants to talk to kc3 and fails to
> do so with this exception:
>
> 2016-07-28 09:15:53,804 ERROR
> [org.infinispan.interceptors.InvocationContextInterceptor]
> (default task-54) ISPN000136: Error executing command
> RemoveCommand, writing keys
> [f9bde276-dd03-41c9-995b-b1aaf64c1489]:
> org.infinispan.remoting.transport.jgroups.SuspectException: Cache
> not running on node kc3
>
> at
> org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
>
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:763)
>
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
>
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
>
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
>
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
>
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
>
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
>
> at org.jgroups.blocks.Request.checkCompletion(Request.java:169)
>
> at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:261)
>
> at
> org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:331)
>
> at
> org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:242)
>
> at
> org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:684)
>
> at org.jgroups.JChannel.up(JChannel.java:738)
>
> at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:123)
>
> at org.jgroups.stack.Protocol.up(Protocol.java:374)
>
> at org.jgroups.protocols.FORK.up(FORK.java:118)
>
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
>
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:394)
>
> at org.jgroups.protocols.ENCRYPT.up(ENCRYPT.java:454)
>
> at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:735)
>
> at
> org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
>
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:922)
>
> at org.jgroups.stack.Protocol.up(Protocol.java:412)
>
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
>
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
>
> at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
>
> at
> org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
>
> at
> org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
>
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
>
> at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
>
> at org.jgroups.protocols.FD.up(FD.java:260)
>
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
>
> at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
>
> at org.jgroups.protocols.Discovery.up(Discovery.java:295)
>
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1577)
>
> at org.jgroups.protocols.TP$MyHandler.run(TP.java:1796)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
>
> The key that it tries to write is the user-id. After this, the
> browser receives an Internal Server Error page, which looks like
> this in html:
>
> <html>
>
> <head>
>
> <title>
>
> Error
>
> </title>
>
> </head>
>
> <body>
>
> Internal Server Error
>
> </body>
>
> </html>
>
> I have configured my infinispan cache settings as following (the
> rest are default):
>
> <distributed-cache name="sessions" mode="SYNC" owners="5"/>
>
> <distributed-cache name="offlineSessions" mode="SYNC" owners="1"/>
>
> <distributed-cache name="loginFailures" mode="SYNC" owners="1"/>
>
> I have tried many things (such as playing with owner amounts or
> instance amounts etc). It does not seem to fix this exception. I
> am well aware that this seems more Infinispan issue than Keycloak,
> but I believe that Keycloak at least should respond the end user a
> better error message (perhaps a login again page) rather than an
> Internal Server Error page. Could you please handle this exception?
>
> Kind Regards,
> Sarp Kaya
>
>
>
>
> _______________________________________________
>
> keycloak-user mailing list
>
> keycloak-user at lists.jboss.org <mailto:keycloak-user at lists.jboss.org>
>
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160804/e13bc9ac/attachment-0001.html
More information about the keycloak-user
mailing list