[keycloak-user] Handling SuspectExceptions in Keycloak
Marek Posolda
mposolda at redhat.com
Fri Aug 5 04:23:22 EDT 2016
We need to wait for infinispan fix here, so no idea :(
Option is to workaround on Keycloak, but we likely not going to do that,
but rather wait for being it fixed directly in infinispan.
Marek
On 05/08/16 02:17, Sarp Kaya wrote:
>
> Hi Marek,
>
> So the issue won’t be fixed by Keycloak 2.3.0?
>
> Sarp
>
> *From: *Marek Posolda <mposolda at redhat.com>
> *Date: *Thursday, August 4, 2016 at 7:50 PM
> *To: *Abdullah Sarp <akaya at expedia.com>,
> "keycloak-user at lists.jboss.org" <keycloak-user at lists.jboss.org>
> *Subject: *Re: [keycloak-user] Handling SuspectExceptions in Keycloak
>
> Hmm... so according to
> https://docs.jboss.org/author/display/WFLY10/Infinispan+Subsystem it
> seems that you're right. It's not easily possible to add the
> interceptor through infinispan subsystem :/
>
> As a workaround, you can probably try to do it programatically. You
> may need to create your own InfinispanConnectionProviderFactory and
> configure it in keycloak-server.json . It can override
> DefaultInfinispanConnectionProviderFactory and add the interceptor
> programatically to realms and users caches. Sorry, don't have better
> proposal to avoid this issue right now :( We likely need to wait until
> https://issues.jboss.org/browse/ISPN-6857 is fixed...
>
> Marek
>
> On 02/08/16 06:32, Sarp Kaya wrote:
>
> Hi Marek,
>
> How do I add the StateTransferInterceptor to the standalone.xml?
> Isn’t that only doable programmatically?
>
> Thanks,
> Sarp
>
> *From: *Marek Posolda <mposolda at redhat.com>
> <mailto:mposolda at redhat.com>
> *Date: *Tuesday, August 2, 2016 at 5:32 AM
> *To: *Abdullah Sarp <akaya at expedia.com>
> <mailto:akaya at expedia.com>, "keycloak-user at lists.jboss.org"
> <mailto:keycloak-user at lists.jboss.org>
> <keycloak-user at lists.jboss.org> <mailto:keycloak-user at lists.jboss.org>
> *Subject: *Re: [keycloak-user] Handling SuspectExceptions in Keycloak
>
> See KC issue [1] and related infinispan issue [2] .
>
> The workaround is to add the StateTransferInterceptor to the
> proper place in chain to "realms" and "users" caches. See how I
> did it programatically. I think that based on that, you should be
> able to add it to infinispan subsystem as well.
>
> [1] https://issues.jboss.org/browse/KEYCLOAK-3306
> [2] https://issues.jboss.org/browse/ISPN-6857
>
> Marek
>
> On 28/07/16 11:53, Sarp Kaya wrote:
>
> Hello,
>
> There is already an existing bug report for Infinispan here:
>
> https://issues.jboss.org/browse/ISPN-6721
>
> Currently for Keycloak, if this exception is thrown then it
> sends an Internal Server Error page to the browser.
> Essentially what would be really good is that it sends the
> user back to the login page instead of displaying Internal
> Server Error.
>
> This happens when I am consistently sending login and logout
> (around 40 req/s) requests to two Keycloak instances (let’s
> call them kc1 and kc2), then one new keycloak instance is
> started kc3. Kc3 connects to kc1 and 2 in clustering mode.
>
> Now kc1 receives a new request (such as login) and while it is
> processing that, kc3 is gracefully shut including the cache
> with this log:
>
> 2016-07-28 09:15:53,656 INFO
> [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 61) WFLYCLINF0003: Stopped sessions cache from
> keycloak container
>
> Just shortly after that (6 ms) kc1 throws an exception like this:
>
> 2016-07-28 09:15:53,662 ERROR [io.undertow.request] (default
> task-48) UT005023: Exception handling request to
> /auth/realms/{realm}/login-actions/authenticate:
> org.jboss.resteasy.spi.UnhandledException:
> org.infinispan.statetransfer.OutdatedTopologyException: Cache
> topology changed while the command was executing: expected
> 175, got 176
>
> at
> org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)
>
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)
>
> at
> org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)
>
> at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)
>
> then shortly after(150 ms) kc1 wants to talk to kc3 and fails
> to do so with this exception:
>
> 2016-07-28 09:15:53,804 ERROR
> [org.infinispan.interceptors.InvocationContextInterceptor]
> (default task-54) ISPN000136: Error executing command
> RemoveCommand, writing keys
> [f9bde276-dd03-41c9-995b-b1aaf64c1489]:
> org.infinispan.remoting.transport.jgroups.SuspectException:
> Cache not running on node kc3
>
> at
> org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
>
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:763)
>
> at
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)
>
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
>
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
>
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
>
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
>
> at
> org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
>
> at org.jgroups.blocks.Request.checkCompletion(Request.java:169)
>
> at
> org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:261)
>
> at
> org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:331)
>
> at
> org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:242)
>
> at
> org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:684)
>
> at org.jgroups.JChannel.up(JChannel.java:738)
>
> at
> org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:123)
>
> at org.jgroups.stack.Protocol.up(Protocol.java:374)
>
> at org.jgroups.protocols.FORK.up(FORK.java:118)
>
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
>
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:394)
>
> at org.jgroups.protocols.ENCRYPT.up(ENCRYPT.java:454)
>
> at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:735)
>
> at
> org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
>
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:922)
>
> at org.jgroups.stack.Protocol.up(Protocol.java:412)
>
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
>
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
>
> at
> org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
>
> at
> org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
>
> at
> org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
>
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
>
> at
> org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
>
> at org.jgroups.protocols.FD.up(FD.java:260)
>
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
>
> at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
>
> at org.jgroups.protocols.Discovery.up(Discovery.java:295)
>
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1577)
>
> at org.jgroups.protocols.TP$MyHandler.run(TP.java:1796)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
>
> The key that it tries to write is the user-id. After this, the
> browser receives an Internal Server Error page, which looks
> like this in html:
>
> <html>
>
> <head>
>
> <title>
>
> Error
>
> </title>
>
> </head>
>
> <body>
>
> Internal Server Error
>
> </body>
>
> </html>
>
> I have configured my infinispan cache settings as following
> (the rest are default):
>
> <distributed-cache name="sessions" mode="SYNC" owners="5"/>
>
> <distributed-cache name="offlineSessions" mode="SYNC"
> owners="1"/>
>
> <distributed-cache name="loginFailures" mode="SYNC"
> owners="1"/>
>
> I have tried many things (such as playing with owner amounts
> or instance amounts etc). It does not seem to fix this
> exception. I am well aware that this seems more Infinispan
> issue than Keycloak, but I believe that Keycloak at least
> should respond the end user a better error message (perhaps a
> login again page) rather than an Internal Server Error page.
> Could you please handle this exception?
>
> Kind Regards,
> Sarp Kaya
>
>
>
>
>
> _______________________________________________
>
> keycloak-user mailing list
>
> keycloak-user at lists.jboss.org
> <mailto:keycloak-user at lists.jboss.org>
>
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160805/6dec91c4/attachment-0001.html
More information about the keycloak-user
mailing list