[jboss-jira] [JBoss JIRA] (WFLY-12163) [org.jgroups.protocols.SYM_ENCRYPT] rejected decryption of unicast message from non-member

[org.jgroups.protocols.SYM_ENCRYPT] rejected decryption of unicast message from non-member ------------------------------------------------------------------------------------------ Key: WFLY-12163 URL: https://issues.jboss.org/browse/WFLY-12163 Project: WildFly Issue Type: Bug Components: Clustering Affects Versions: 17.0.0.Beta1 Reporter: Tommasso Borgato Assignee: Paul Ferraro Priority: Critical The error is observed in EAP Clustering fail-over tests where the communication at the JGroups level is symmetrically encrypted . The error causes sampling errors on the client (about 3000) and an overall 0.65% fail rate. Basically, right after the deployment when the cache starts, we observe a big number of the following errors in the logs: {noformat} 2019-05-30 12:19:34,445 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 82) WFLYUT0021: Registered web context: '/clusterbench-passivating' for server 'default-server' 2019-05-30 12:19:34,592 INFO [org.jboss.as.server] (management-handler-thread - 2) WFLYSRV0010: Deployed "clusterbench-ee8.ear" (runtime-name : "clusterbench-ee8.ear") 2019-05-30 12:20:21,180 ERROR [org.jgroups.protocols.SYM_ENCRYPT] (thread-4,null,null) wildfly1: rejected decryption of unicast message from non-member wildfly2 2019-05-30 12:20:21,182 WARN [org.jgroups.protocols.SYM_ENCRYPT] (thread-4,null,null) wildfly1: unrecognized cipher; discarding message from wildfly2 {noformat} Complete logs: * [eap-7.x-clustering-http-session-encrypt-sym/27|https://eap-qe-jenkins.rhe...] * [eap-7.x-clustering-http-session-encrypt-sym/28|https://eap-qe-jenkins.rhe...] * [eap-7.x-clustering-http-session-encrypt-sym/29|https://eap-qe-jenkins.rhe...] This is the setup for every node in the cluster that can be used to reproduce the issue: {noformat} embed-server --server-config=standalone-ha.xml /subsystem=infinispan/cache-container=web/distributed-cache=testDist:add() /subsystem=infinispan/cache-container=web/distributed-cache=testDist/component=locking:write-attribute(name=isolation, value=REPEATABLE_READ) /subsystem=infinispan/cache-container=web/distributed-cache=testDist/component=transaction:write-attribute(name=mode, value=BATCH) /subsystem=infinispan/cache-container=web:write-attribute(name=default-cache, value=testDist) /subsystem=jgroups/channel=ee:write-attribute(name=stack,value=tcp) batch /subsystem=jgroups/stack=udp/protocol=AUTH:add(add-index=8) /subsystem=jgroups/stack=udp/protocol=AUTH/token=digest:add(algorithm=SHA-512, shared-secret-reference={clear-text=123PIPPOBAUDO}) /subsystem=jgroups/stack=tcp/protocol=AUTH:add(add-index=8) /subsystem=jgroups/stack=tcp/protocol=AUTH/token=digest:add(algorithm=SHA-512, shared-secret-reference={clear-text=123PIPPOBAUDO}) run-batch /subsystem=elytron/key-store=jgroups-keystore:add(path=jgroups.keystore,credential-reference={clear-text=123PIPPOBAUDO},type=JCEKS,relative-to=jboss.server.config.dir) /subsystem=jgroups/stack=udp/protocol=SYM_ENCRYPT:add(add-index=5,key-store=jgroups-keystore,key-alias=mykey,key-credential-reference={clear-text=123PIPPOBAUDO}) /subsystem=jgroups/stack=tcp/protocol=SYM_ENCRYPT:add(add-index=5,key-store=jgroups-keystore,key-alias=mykey,key-credential-reference={clear-text=123PIPPOBAUDO}) {noformat} This is the command used to generate the key-store referenced in the cli script above: {noformat} cd $JBOSS_HOME/standalone/configuration/ java -cp $(find /tmp/tests-clustering/jboss-eap-1/modules -name "jgroups-[0-9]*.jar") org.jgroups.demos.KeyStoreGenerator --alg AES --size 128 --storeName jgroups.keystore --storepass 123PIPPOBAUDO --alias mykey {noformat}