<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">Hmm... so according to
      <a class="moz-txt-link-freetext" href="https://docs.jboss.org/author/display/WFLY10/Infinispan+Subsystem">https://docs.jboss.org/author/display/WFLY10/Infinispan+Subsystem</a>
      it seems that you're right. It's not easily possible to add the
      interceptor through infinispan subsystem :/<br>
      <br>
      As a workaround, you can probably try to do it programatically.
      You may need to create your own
      InfinispanConnectionProviderFactory and configure it in
      keycloak-server.json . It can override
      DefaultInfinispanConnectionProviderFactory and add the interceptor
      programatically to realms and users caches. Sorry, don't have
      better proposal to avoid this issue right now :( We likely need to
      wait until <a moz-do-not-send="true"
        href="https://issues.jboss.org/browse/ISPN-6857">https://issues.jboss.org/browse/ISPN-6857</a>
      is fixed...<br>
      <br>
      Marek<br>
      <br>
      On 02/08/16 06:32, Sarp Kaya wrote:<br>
    </div>
    <blockquote
      cite="mid:510A5664-A01D-4494-8188-51084A6CF946@expedia.com"
      type="cite">
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      <meta name="Title" content="">
      <meta name="Keywords" content="">
      <meta name="Generator" content="Microsoft Word 15 (filtered
        medium)">
      <style><!--
/* Font Definitions */
@font-face
        {font-family:"Courier New";
        panose-1:2 7 3 9 2 2 5 2 4 4;}
@font-face
        {font-family:"Cambria Math";
        panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
        {font-family:Menlo;
        panose-1:2 11 6 9 3 8 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:Calibri;}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:#0563C1;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:#954F72;
        text-decoration:underline;}
pre
        {mso-style-priority:99;
        mso-style-link:"HTML Preformatted Char";
        margin:0in;
        margin-bottom:.0001pt;
        font-size:10.0pt;
        font-family:"Courier New";}
span.HTMLPreformattedChar
        {mso-style-name:"HTML Preformatted Char";
        mso-style-priority:99;
        mso-style-link:"HTML Preformatted";
        font-family:"Courier New";}
span.EmailStyle19
        {mso-style-type:personal;
        font-family:Calibri;
        color:windowtext;}
span.EmailStyle20
        {mso-style-type:personal-reply;
        font-family:Calibri;
        color:windowtext;}
span.msoIns
        {mso-style-type:export-only;
        mso-style-name:"";
        text-decoration:underline;
        color:teal;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-size:10.0pt;}
@page WordSection1
        {size:8.5in 11.0in;
        margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
        {page:WordSection1;}
--></style>
      <div class="WordSection1">
        <p class="MsoNormal"><span style="font-size:11.0pt">Hi Marek,<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">How do I add
            the </span>StateTransferInterceptor to the standalone.xml?
          Isn’t that only doable programmatically?<o:p></o:p></p>
        <p class="MsoNormal"><o:p> </o:p></p>
        <p class="MsoNormal">Thanks,<br>
          Sarp<o:p></o:p></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <div style="border:none;border-top:solid #B5C4DF
          1.0pt;padding:3.0pt 0in 0in 0in">
          <p class="MsoNormal"><b><span style="color:black">From: </span></b><span
              style="color:black">Marek Posolda
              <a class="moz-txt-link-rfc2396E" href="mailto:mposolda@redhat.com">&lt;mposolda@redhat.com&gt;</a><br>
              <b>Date: </b>Tuesday, August 2, 2016 at 5:32 AM<br>
              <b>To: </b>Abdullah Sarp <a class="moz-txt-link-rfc2396E" href="mailto:akaya@expedia.com">&lt;akaya@expedia.com&gt;</a>,
              <a class="moz-txt-link-rfc2396E" href="mailto:keycloak-user@lists.jboss.org">"keycloak-user@lists.jboss.org"</a>
              <a class="moz-txt-link-rfc2396E" href="mailto:keycloak-user@lists.jboss.org">&lt;keycloak-user@lists.jboss.org&gt;</a><br>
              <b>Subject: </b>Re: [keycloak-user] Handling
              SuspectExceptions in Keycloak<o:p></o:p></span></p>
        </div>
        <div>
          <p class="MsoNormal"><span style="font-family:&quot;Times New
              Roman&quot;"><o:p> </o:p></span></p>
        </div>
        <div>
          <div>
            <div>
              <p class="MsoNormal">See KC issue [1] and related
                infinispan issue [2] . <br>
                <br>
                The workaround is to add the StateTransferInterceptor to
                the proper place in chain to "realms" and "users"
                caches. See how I did it programatically. I think that
                based on that, you should be able to add it to
                infinispan subsystem as well.<br>
                <br>
                [1] <a moz-do-not-send="true"
                  href="https://issues.jboss.org/browse/KEYCLOAK-3306">https://issues.jboss.org/browse/KEYCLOAK-3306</a><br>
                [2] <a moz-do-not-send="true"
                  href="https://issues.jboss.org/browse/ISPN-6857">https://issues.jboss.org/browse/ISPN-6857</a><br>
                <br>
                Marek<br>
                <br>
                On 28/07/16 11:53, Sarp Kaya wrote:<o:p></o:p></p>
            </div>
            <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
              <p class="MsoNormal"><span style="font-size:11.0pt">Hello,</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">There
                  is already an existing bug report for Infinispan here:</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"><a
                    moz-do-not-send="true"
                    href="https://issues.jboss.org/browse/ISPN-6721"><a class="moz-txt-link-freetext" href="https://issues.jboss.org/browse/ISPN-6721">https://issues.jboss.org/browse/ISPN-6721</a></a></span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">Currently
                  for Keycloak, if this exception is thrown then it
                  sends an Internal Server Error page to the browser.
                  Essentially what would be really good is that it sends
                  the user back to the login page instead of displaying
                  Internal Server Error.</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">This
                  happens when I am consistently sending login and
                  logout (around 40 req/s) requests to two Keycloak
                  instances (let’s call them kc1 and kc2), then one new
                  keycloak instance is started kc3. Kc3 connects to kc1
                  and 2 in clustering mode.</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">Now
                  kc1 receives a new request (such as login) and while
                  it is processing that, kc3 is gracefully shut
                  including the cache with this log:</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">2016-07-28
                  09:15:53,656 INFO 
                  [org.jboss.as.clustering.infinispan] (ServerService
                  Thread Pool -- 61) WFLYCLINF0003: Stopped sessions
                  cache from keycloak container</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">Just
                  shortly after that (6 ms) kc1 throws an exception like
                  this:</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">2016-07-28
                  09:15:53,662 ERROR [io.undertow.request] (default
                  task-48) UT005023: Exception handling request to
                  /auth/realms/{realm}/login-actions/authenticate:
                  org.jboss.resteasy.spi.UnhandledException:
                  org.infinispan.statetransfer.OutdatedTopologyException:
                  Cache topology changed while the command was
                  executing: expected 175, got 176</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:247)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:168)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:471)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:415)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">then
                  shortly after(150 ms) kc1 wants to talk to kc3 and
                  fails to do so with this exception:</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">2016-07-28
                  09:15:53,804 ERROR
                  [org.infinispan.interceptors.InvocationContextInterceptor]
                  (default task-54) ISPN000136: Error executing command
                  RemoveCommand, writing keys
                  [f9bde276-dd03-41c9-995b-b1aaf64c1489]:
                  org.infinispan.remoting.transport.jgroups.SuspectException:
                  Cache not running on node kc3</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:763)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$73(JGroupsTransport.java:612)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.blocks.Request.checkCompletion(Request.java:169)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:261)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:331)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:242)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:684)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.JChannel.up(JChannel.java:738)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:123)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.stack.Protocol.up(Protocol.java:374)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.protocols.FORK.up(FORK.java:118)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.FlowControl.up(FlowControl.java:394)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.protocols.ENCRYPT.up(ENCRYPT.java:454)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.pbcast.GMS.installView(GMS.java:735)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.protocols.pbcast.GMS.up(GMS.java:922)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.stack.Protocol.up(Protocol.java:412)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.protocols.FD.up(FD.java:260)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.Discovery.up(Discovery.java:295)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.TP.passMessageUp(TP.java:1577)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
                  org.jgroups.protocols.TP$MyHandler.run(TP.java:1796)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">               
                  at java.lang.Thread.run(Thread.java:745)</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">The
                  key that it tries to write is the user-id. After this,
                  the browser receives an Internal Server Error page,
                  which looks like this in html:</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">&lt;html&gt;</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> 
                  &lt;head&gt;</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">   
                  &lt;title&gt;</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">     
                  Error</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">   
                  &lt;/title&gt;</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> 
                  &lt;/head&gt;</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> 
                  &lt;body&gt;</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">   
                  Internal Server Error</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> 
                  &lt;/body&gt;</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">&lt;/html&gt;</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">I have
                  configured my infinispan cache settings as following
                  (the rest are default):</span><o:p></o:p></p>
              <p class="MsoNormal" style="background:#2B2B2B"><span
                  style="font-size:9.0pt;font-family:Menlo;color:#E8BF6A">   
                  &lt;distributed-cache
                </span><span
                  style="font-size:9.0pt;font-family:Menlo;color:#BABABA">name</span><span
style="font-size:9.0pt;font-family:Menlo;color:#6A8759">="sessions"
                </span><span
                  style="font-size:9.0pt;font-family:Menlo;color:#BABABA">mode</span><span
style="font-size:9.0pt;font-family:Menlo;color:#6A8759">="SYNC"
                </span><span
                  style="font-size:9.0pt;font-family:Menlo;color:#BABABA">owners</span><span
style="font-size:9.0pt;font-family:Menlo;color:#6A8759">="5"</span><span
style="font-size:9.0pt;font-family:Menlo;color:#E8BF6A">/&gt;<br>
                  <br>
                      &lt;distributed-cache </span><span
                  style="font-size:9.0pt;font-family:Menlo;color:#BABABA">name</span><span
style="font-size:9.0pt;font-family:Menlo;color:#6A8759">="offlineSessions"
                </span><span
                  style="font-size:9.0pt;font-family:Menlo;color:#BABABA">mode</span><span
style="font-size:9.0pt;font-family:Menlo;color:#6A8759">="SYNC"
                </span><span
                  style="font-size:9.0pt;font-family:Menlo;color:#BABABA">owners</span><span
style="font-size:9.0pt;font-family:Menlo;color:#6A8759">="1"</span><span
style="font-size:9.0pt;font-family:Menlo;color:#E8BF6A">/&gt;<br>
                  <br>
                      &lt;distributed-cache </span><span
                  style="font-size:9.0pt;font-family:Menlo;color:#BABABA">name</span><span
style="font-size:9.0pt;font-family:Menlo;color:#6A8759">="loginFailures"
                </span><span
                  style="font-size:9.0pt;font-family:Menlo;color:#BABABA">mode</span><span
style="font-size:9.0pt;font-family:Menlo;color:#6A8759">="SYNC"
                </span><span
                  style="font-size:9.0pt;font-family:Menlo;color:#BABABA">owners</span><span
style="font-size:9.0pt;font-family:Menlo;color:#6A8759">="1"</span><span
style="font-size:9.0pt;font-family:Menlo;color:#E8BF6A">/&gt;</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">I have
                  tried many things (such as playing with owner amounts
                  or instance amounts etc). It does not seem to fix this
                  exception. I am well aware that this seems more
                  Infinispan issue than Keycloak, but I believe that
                  Keycloak at least should respond the end user a better
                  error message (perhaps a login again page) rather than
                  an Internal Server Error page. Could you please handle
                  this exception?</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">Kind
                  Regards,<br>
                  Sarp Kaya</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-family:&quot;Times
                  New Roman&quot;"><br>
                  <br>
                  <br>
                  <o:p></o:p></span></p>
              <pre>_______________________________________________<o:p></o:p></pre>
              <pre>keycloak-user mailing list<o:p></o:p></pre>
              <pre><a moz-do-not-send="true" href="mailto:keycloak-user@lists.jboss.org">keycloak-user@lists.jboss.org</a><o:p></o:p></pre>
              <pre><a moz-do-not-send="true" href="https://lists.jboss.org/mailman/listinfo/keycloak-user">https://lists.jboss.org/mailman/listinfo/keycloak-user</a><o:p></o:p></pre>
            </blockquote>
            <p class="MsoNormal"><span style="font-family:&quot;Times
                New Roman&quot;"><o:p> </o:p></span></p>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>