We are testing setting up a configuration where the API gateway, the API manager UI, and Keycloak are all behind their own load balancers on AWS. Keycloak is clustered using JDBC_PING.

When I try to access the apimanui URL after logging in via Keycloak, sometimes the admin page is rendered; sometimes it isn't and I have to refresh it a few times. I see a flood of requests coming into both of the Keycloak instances.

From what I can see, after the POST to Keycloak happens, there is a sequence of 302 redirects that eventually results in a successful GET to index.html. After that, however, each request for a resource on the page — css, javascript, fonts, whatever — also gets a 302 and is redirected to Keycloak and redirected back before the request is successful. I'm getting the impression from what I'm seeing that the bearer token is not being received by the browser and/or submitted with requests.

Below is an example from the browser request log. All the browser requests are to various subdomains of us-west-2.elb.amazonaws.com (the load balancers); the instances of apiman and Keycloak are all on subdomains of us-west-2.compute.amazonaws.com. There is currently no session affinity set up in the load balancers for Keycloak, the apiman gateway, or the apiman management UI.

Any ideas on what might be causing this?

*** Part 1: Browser login via Keycloak and request for index.html ***

POST https://[KEYCLOAK]/auth/realms/apiman/login-actions/authenticate?code=[CODE-01]&execution=[EXECUTION-01]
  Cookie:"KC_RESTART=[RESTART-01]"
Response: 302
  Location:"https://[KEYCLOAK]/auth/realms/apiman/login-actions/authenticate?code=[CODE-01]"
  
GET https://[KEYCLOAK]/auth/realms/apiman/login-actions/authenticate?code=[CODE-01]
  Cookie:"KC_RESTART=[RESTART-01]"
Response: 302
  Location:"https://[KEYCLOAK]/auth/realms/apiman/login-actions/required-action?code=[CODE-02]"
  
GET https://[KEYCLOAK]/auth/realms/apiman/login-actions/required-action?code=[CODE-02]
  Cookie:"KC_RESTART=[RESTART-01]"
Response: 302
  Location:"https://[API_MANAGER]/apimanui/index.html?state=[STATE-01]"
  Set-Cookie:"KEYCLOAK_IDENTITY=[IDENTITY-01]; Version=1; Path=/auth/realms/apiman; HttpOnly
              KEYCLOAK_SESSION=apiman/[KC_SESS-01]; Version=1; Expires=Wed, 06-Jan-2016 06:09:59 GMT; Max-Age=36000; Path=/auth/realms/apiman
              KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/auth/realms/apiman; HttpOnly"

GET https://[API_MANAGER]/apimanui/index.html?state=[STATE-01]&code=[CODE-03]
  Cookie:"OAuth_Token_Request_State=[STATE-01]"
Response: 302
  Location:"https://[API_MANAGER]/apimanui/index.html"  
  Set-Cookie:"JSESSIONID=[APIMAN_JSESS-01].[SUFFIX-01]; path=/apimanui
              OAuth_Token_Request_State=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:00 GMT"

GET https://[API_MANAGER]/apimanui/index.html
  Cookie:"JSESSIONID=[APIMAN_JSESS-01].[SUFFIX-01]"
Response: 302
  Location:"https://[KEYCLOAK]/auth/realms/apiman/protocol/openid-connect/auth?response_type=code&client_id=apimanui&redirect_uri=https%3A%2F%2F[API_MANAGER]%2Fapimanui%2Findex.html&state=[STATE-02]&login=true"
  Set-Cookie:"JSESSIONID=[APIMAN_JSESS-01].[SUFFIX-02]; path=/apimanui
              OAuth_Token_Request_State=[STATE-02]; secure"

GET https://[KEYCLOAK]/auth/realms/apiman/protocol/openid-connect/auth?response_type=code&client_id=apimanui&redirect_uri=https://[API_MANAGER]/apimanui/index.html&state=[STATE-02]&login=true
  Cookie:"KEYCLOAK_IDENTITY=[IDENTITY-01]; KEYCLOAK_SESSION=apiman/[KC_SESS-01]"
Response: 302
  Location:"https://[KEYCLOAK]/auth/realms/apiman/login-actions/required-action?code=[CODE-04]"
  Set-Cookie:"KC_RESTART=[RESTART-02]; Version=1; Path=/auth/realms/apiman; HttpOnly"

GET https://[KEYCLOAK]/auth/realms/apiman/login-actions/required-action?code=[CODE-04]
  Cookie:"KEYCLOAK_IDENTITY=[IDENTITY-01]; KEYCLOAK_SESSION=apiman/[KC_SESS-01]; KC_RESTART=[RESTART-02]"
Response: 302
  Location:"https://[API_MANAGER]/apimanui/index.html?state=[STATE-02]&code=[CODE-05]"
  Set-Cookie:"KEYCLOAK_IDENTITY=[IDENTITY-02]; Version=1; Path=/auth/realms/apiman; HttpOnly
              KEYCLOAK_SESSION=apiman/[KC_SESS-01]; Version=1; Expires=Wed, 06-Jan-2016 06:10:00 GMT; Max-Age=36000; Path=/auth/realms/apiman
              KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/auth/realms/apiman; HttpOnly"

GET https://[API_MANAGER]/apimanui/index.html?state=[STATE-02]&code=[CODE-05]
  Cookie:"OAuth_Token_Request_State=[STATE-02]; JSESSIONID=[APIMAN_JSESS-01].[SUFFIX-02]"
Response: 200
  Set-Cookie:"JSESSIONID=[APIMAN_JSESS-01].[SUFFIX-01]; path=/apimanui"

*** Part 2: Subsequent requests for resources (here, bootstrap-select.css) ***

GET https://[API_MANAGER]/apimanui/libs/bootstrap-select/bootstrap-select.css?cid=2015-10-23_16:50
  Cookie:"OAuth_Token_Request_State=[STATE-02]; JSESSIONID=[APIMAN_JSESS-01].[SUFFIX-01]"
Response: 302
  Location:"https://[KEYCLOAK]/auth/realms/apiman/protocol/openid-connect/auth?response_type=code&client_id=apimanui&redirect_uri=https%3A%2F%2F[API_MANAGER]%2Fapimanui%2Flibs%2Fbootstrap-select%2Fbootstrap-select.css?cid%3D2015-10-23_16%3A50&state=[STATE-03]&login=true"
  Set-Cookie:"JSESSIONID=[APIMAN_JSESS-01].[SUFFIX-02]; path=/apimanui
              OAuth_Token_Request_State=[STATE-03]; secure"

GET https://[KEYCLOAK]/auth/realms/apiman/protocol/openid-connect/auth?response_type=code&client_id=apimanui&redirect_uri=https://[API_MANAGER]/apimanui/libs/bootstrap-select/bootstrap-select.css?cid=2015-10-23_16:50&state=[STATE-03]&login=true
  Cookie:"KEYCLOAK_IDENTITY=[IDENTITY-03]; KEYCLOAK_SESSION=apiman/[KC_SESS-01]"
Response: 302
  Location:"https://[KEYCLOAK]/auth/realms/apiman/login-actions/required-action?code=[CODE-06]"
  Set-Cookie:"KC_RESTART=[RESTART-03]; Version=1; Path=/auth/realms/apiman; HttpOnly"

GET https://[KEYCLOAK]/auth/realms/apiman/login-actions/required-action?code=[CODE-06]
  Cookie:"KEYCLOAK_IDENTITY=[IDENTITY-03]; KEYCLOAK_SESSION=apiman/[KC_SESS-01]; KC_RESTART=[RESTART-03]"
Response: 302
  Location:"https://[API_MANAGER]/apimanui/libs/bootstrap-select/bootstrap-select.css?cid=2015-10-23_16:50&state=[STATE-03]&code=[CODE-07]"
  Set-Cookie:"KEYCLOAK_IDENTITY=[IDENTITY-04]; Version=1; Path=/auth/realms/apiman; HttpOnly
              KEYCLOAK_SESSION=apiman/[KC_SESS-01]; Version=1; Expires=Wed, 06-Jan-2016 06:10:02 GMT; Max-Age=36000; Path=/auth/realms/apiman
              KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/auth/realms/apiman; HttpOnly"

GET https://[API_MANAGER]/apimanui/libs/bootstrap-select/bootstrap-select.css?cid=2015-10-23_16:50&state=[STATE-03]&code=[CODE-07]
  Cookie:"OAuth_Token_Request_State=445/4a12cbb7-c16d-42a5-90c7-cf296616674a; OAuth_Token_Request_State=[STATE-02]; JSESSIONID=[APIMAN_JSESS-01].[SUFFIX-02]"
Response: 400
  Set-Cookie:"OAuth_Token_Request_State=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:00 GMT"

*** Meanwhile, in Keycloak — the logs have the following segment repeatedly: ***

DEBUG [org.keycloak.protocol.oidc.utils.RedirectUtils] (default task-23) replacing relative valid redirect with: https://[API_MANAGER]/apimanui/*
DEBUG [org.keycloak.authentication.AuthenticationProcessor] (default task-23) AUTHENTICATE
DEBUG [org.keycloak.authentication.AuthenticationProcessor] (default task-23) authenticator: auth-cookie
DEBUG [org.keycloak.services.managers.AuthenticationManager] (default task-23) token active - active: true, issued-at: 1,452,019,157, not-before: 1,452,014,329
DEBUG [org.keycloak.authentication.AuthenticationProcessor] (default task-23) authenticator SUCCESS: auth-cookie
DEBUG [org.keycloak.authentication.AuthenticationProcessor] (default task-23) execution is processed