Drools/Kie-Server/Busines-Central 7.28.0 Is getting a 403 when kie-server accesses the websocket controller on business-central ------------------------------------------------------------------------------------------------------------------------------- Key: DROOLS-4705 URL: https://issues.jboss.org/browse/DROOLS-4705 Project: Drools Issue Type: Bug Components: kie server Affects Versions: 7.24.0.Final, 7.25.0.Final, 7.28.0.Final Environment: Linux, JDK8, and Wildfly 14.0.1 Reporter: Danny Rucker Assignee: Maciej Swiderski Priority: Major Fix For: 7.23.0.Final We're upgrading our business-central server from 7.23.0 to 7.28.0. We're noticing that our kie-servers can no longer connect via websocket to business-central: Server logs: {quote} Oct 28 12:56:43 business-central-1 business-central[18870]: #033[0m#033[0m12:56:43,962 INFO [org.kie.server.controller.websocket.notification.WebSocketNotificationService] (Thread-123) WebSocket notify on updated :: Updated server template{serverTemplate=ServerTemplateKey{id='host.subdomain.x.com', name='host.subdomain.x.com'}, resetBeforeUpdate=false} Oct 28 12:56:43 business-central-1 business-central[18870]: #033[0m#033[0m12:56:43,963 INFO [org.kie.server.controller.websocket.notification.WebSocketNotificationService] (Thread-123) WebSocket notify on instance disconnected :: ServerInstanceDisconnected{serverInstanceId='host.subdomain.x.com@host.subdomain.x.com:36204'} {quote} Client logs: {quote} Oct 28 20:21:53 host kie-server[1375]: #033[0m#033[33m20:21:53,090 WARN [org.kie.server.common.KeyStoreHelperUtil] (KieServer-ControllerConnect) Unable to load key store. Using password from configuration Oct 28 20:21:53 host kie-server[1375]: #033[0m#033[33m20:21:53,146 WARN [org.kie.server.controller.websocket.client.WebSocketKieServerControllerImpl] (KieServer-ControllerConnect) Exception encountered while syncing with controller at wss://business-central-1.x.com/business-central/websocket/controller/host... error Invalid response code 403 {quote} The actual break occurred between 7.23.0 and 7.24.0, as 7.24.0 produces the same error message: Invalid response code 403. The user that kie-server is connecting to business-central with has kie-server as a group, but that doesn't seem to help. We're running business-central on Wildfly 14.0.1. I'm looking through the commits for a web.xml change for filtering web reources but can't seem to find anything. Actually, I'm not even sure which github project I should be looking in: https://github.com/kiegroup tcpdump from 7.24.0+ versions(none worked): {quote} GET /business-central/websocket/controller/apps-3-staging HTTP/1.1 Authorization: Basic <base64-stuff-here> Sec-WebSocket-Key: YZ5r8liCJKut6VJ2YG7sPQ== Connection: upgrade Sec-WebSocket-Version: 13 Upgrade: websocket Host: localhost:8080 HTTP/1.1 403 Forbidden Expires: 0 Cache-Control: no-cache, no-store, must-revalidate X-Powered-By: JSP/2.3 Set-Cookie: JSESSIONID=cBz0q1sK09Fq2jtXA8Mad1FHPKxvQ38akKGNMP9R.business-central-1; path=/business-central; HttpOnly; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:00 GMT Pragma: no-cache Date: Mon, 28 Oct 2019 18:23:21 GMT Connection: keep-alive Content-Type: text/html;charset=UTF-8 Content-Length: 1102 Content-Language: en- {quote} tcpdump from 7.23.0 version(which works): {quote} GET /business-central/websocket/controller/apps-3-staging HTTP/1.1 Authorization: Basic <base64-stuff-here> Sec-WebSocket-Key: yhykxYXh0z+KF0Zv/8P76g== Connection: upgrade Sec-WebSocket-Version: 13 Host: localhost:8080 Upgrade: websocket HTTP/1.1 101 Switching Protocols Connection: Upgrade Set-Cookie: JSESSIONID=UCZsXbdV1ZUjTBTOcbP9j-ppd3y2NH0mzOqPQcjP.business-central; path=/business-central; HttpOnly Sec-WebSocket-Location: wss://business-central-1/business-central/websocket/controller/apps-3-staging X-XSS-Protection: 1; mode=block Upgrade: WebSocket X-FRAME-OPTIONS: SAMEORIGIN Sec-WebSocket-Accept: 6qSH/TZNvoukpIE+ZJYulWzGge0= Date: Mon, 28 Oct 2019 17:21:57 GMT {quote}