[JBoss JIRA] (WFLY-13527) Thousand of unauthorized requests in between balancer and backend if backend is running in a cluster
by Flavia Rainone (Jira)
Flavia Rainone created WFLY-13527:
-------------------------------------
Summary: Thousand of unauthorized requests in between balancer and backend if backend is running in a cluster
Key: WFLY-13527
URL: https://issues.redhat.com/browse/WFLY-13527
Project: WildFly
Issue Type: Bug
Reporter: Flavia Rainone
Assignee: Masafumi Miura
A standalone client application is calling EJBs on a backend server through an Undertow loadbalancer.
The client looks like this:
{code:java}
Properties props = new Properties();
props.put(Context.INITIAL_CONTEXT_FACTORY, WildFlyInitialContextFactory.class.getName());
props.put(Context.PROVIDER_URL, "http://10.0.0.1:9080/wildfly-services");
props.put(Context.SECURITY_PRINCIPAL, "some-user");
props.put(Context.SECURITY_CREDENTIALS, "some-password");
InitialContext ctx = new InitialContext(props);
String name="ejb:/playground-jar/JBossManIntClientBean!org.jboss.playground.JBossManIntClient";
JBossManIntClient bean = (JBossManIntClient) ctx.lookup(name);
{code}
A client invoking the same EJB twice result in thousands of requests from the balancer to the backend servers, e. g.:
* 13468 times:
{code}
INFO [io.undertow.accesslog] (default I/O-3) 10.0.0.1 - - [24/Sep/2019:12:03:03 +0200] "POST /wildfly-services/ejb/v1/invoke/-/playground-jar/-/JBossManIntClientBean/-/org.jboss.playground.JBossManIntClient/getHost HTTP/2.0" 401 77 "-" "-" Cookie: "-" Set-Cookie: "-" SessionID: - Thread: "default I/O-3" TimeTaken: 5063
{code}
* 2 times:
{code}
INFO [io.undertow.accesslog] (default I/O-3) 10.0.0.1 - - [24/Sep/2019:12:06:53 +0200] "POST /wildfly-services/ejb/v1/invoke/-/playground-jar/-/JBossManIntClientBean/-/org.jboss.playground.JBossManIntClient/getHost HTTP/2.0" 200 155 "-" "-" Cookie: "-" Set-Cookie: "-" SessionID: - Thread: "default I/O-3" TimeTaken: 614
{code}
*Note:* This behavior only occurs if there's more than a single backend server running in a cluster...
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (DROOLS-5342) Default value of "drools.lambda.introspector.cache.size" for externalized lambda
by Toshiya Kobayashi (Jira)
[ https://issues.redhat.com/browse/DROOLS-5342?page=com.atlassian.jira.plug... ]
Toshiya Kobayashi closed DROOLS-5342.
-------------------------------------
Resolution: Won't Do
DROOLS-5353 solved ClassLoader retention concern so now the default value '32' is fine regardless of externalise lambda is enabled or not. Closing.
> Default value of "drools.lambda.introspector.cache.size" for externalized lambda
> --------------------------------------------------------------------------------
>
> Key: DROOLS-5342
> URL: https://issues.redhat.com/browse/DROOLS-5342
> Project: Drools
> Issue Type: Task
> Components: executable model
> Affects Versions: 7.37.0.Final
> Reporter: Toshiya Kobayashi
> Assignee: Toshiya Kobayashi
> Priority: Major
>
> Regarding the cache size default value of LambdaIntrospector.methodFingerprintsMap,
> If "drools.externaliseCanonicalModelLambda" option is not enabled, the cache is frequently used so it will not likely retain old classloaders. So the default value '32' would be fine.
> If "drools.externaliseCanonicalModelLambda" option is enabled, the cache is used in rare occasions:
> A) Lambda for binding variable (DROOLS-5328)
> B) Lambda which ExecModelLambdaPostProcessor failed to externalize (DROOLS-5329)
> So one build may use only a few entries in a cache Map. Then, the cache eventually may retain several old classloaders after multiple builds. Generally, the classloader leak issue is prioritized over build time performance benefit so '0' is the good default value when externaliseCanonicalModelLambda is enabled.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (JGRP-2479) [GSS](7.3.x) JBDC_PING has a possibility not to discover other members and create singleton clusters (split-brain issue) when restarting a coordinator node
by Masafumi Miura (Jira)
Masafumi Miura created JGRP-2479:
------------------------------------
Summary: [GSS](7.3.x) JBDC_PING has a possibility not to discover other members and create singleton clusters (split-brain issue) when restarting a coordinator node
Key: JGRP-2479
URL: https://issues.redhat.com/browse/JGRP-2479
Project: JGroups
Issue Type: Bug
Affects Versions: 4.1.9, 4.0.22
Reporter: Masafumi Miura
Assignee: Radoslav Husar
Fix For: 4.2.5
After [the change|https://github.com/belaban/JGroups/commit/215cdb6] for JGRP-2199, JDBC_PING deletes all entries from the table during the shutdown of the coordinator node.
This behavior has a possibility to cause a split-brain when restarting a coordinator node. Because, as all entries are lost in the following scenario, the restarting node can not find any information about existing nodes from the table and does not form a cluster.
0. node1 and node2 form a cluster. The node1 is a coordinator.
1. Trigger a restart of the node1
2. The node1 removes their node information from the table
3. The node2 becomes a new coordinator
4. The node2 updates their node information in the table
5. The node1 clears all entries from the table
6. The node1 starts again
7. The node1 does not join the existing cluster because there's no node information in the table
Note: If step 5 happens before step 4, the split-brain issue does not happen. However, as step 4 and step 5 happen on different nodes, these steps can happen in parallel. So, the order is undefined. So, for example, if the shutdown of node1 takes a long time, there's a high possibility to face this issue.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (ELY-1975) Update AcmeClientSpi#obtainCertificate so that it obtains the order URL from the response to newOrder
by Farah Juma (Jira)
[ https://issues.redhat.com/browse/ELY-1975?page=com.atlassian.jira.plugin.... ]
Farah Juma updated ELY-1975:
----------------------------
Description: {{AcmeClientSpi#obtainCertificate}} currently determines the order URL that should be used by taking the {{Location}} header from the response to finalizing the order (the ACME protocol specification has an example response that includes this header). This works when obtaining certificates from Let's Encrypt and Boulder. However, it seems that EJBCA does not include the {{Location}} header in the response to finalizing the order. {{AcmeClientSpi#obtainCertificate}} should instead use the Location header from the response to {{newOrder}}. (was: {{AcmeClientSpi#obtainCertificate}} currently determines the order URL that should be used by taking the {{Location}} header from the response to finalizing the order (the ACME protocol specification has an example response that includes this header). This works when obtaining certificates from Let's Encrypt and Boulder. However, it seems that EJBCA does not include the {{Location}} header in the response to finalizing the order. {{AcmeClientSpi#obtainCertificate}} should instead attempt to use the Location header from the response to {{newOrder}}.)
> Update AcmeClientSpi#obtainCertificate so that it obtains the order URL from the response to newOrder
> -----------------------------------------------------------------------------------------------------
>
> Key: ELY-1975
> URL: https://issues.redhat.com/browse/ELY-1975
> Project: WildFly Elytron
> Issue Type: Task
> Reporter: Farah Juma
> Assignee: Farah Juma
> Priority: Major
>
> {{AcmeClientSpi#obtainCertificate}} currently determines the order URL that should be used by taking the {{Location}} header from the response to finalizing the order (the ACME protocol specification has an example response that includes this header). This works when obtaining certificates from Let's Encrypt and Boulder. However, it seems that EJBCA does not include the {{Location}} header in the response to finalizing the order. {{AcmeClientSpi#obtainCertificate}} should instead use the Location header from the response to {{newOrder}}.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (ELY-1975) Update AcmeClientSpi#obtainCertificate so that it obtains the order URL from the newOrder request
by Farah Juma (Jira)
Farah Juma created ELY-1975:
-------------------------------
Summary: Update AcmeClientSpi#obtainCertificate so that it obtains the order URL from the newOrder request
Key: ELY-1975
URL: https://issues.redhat.com/browse/ELY-1975
Project: WildFly Elytron
Issue Type: Task
Reporter: Farah Juma
Assignee: Farah Juma
{{AcmeClientSpi#obtainCertificate}} currently determines the order URL that should be used by taking the {{Location}} header from the response to finalizing the order (the ACME protocol specification has an example response that includes this header). This works when obtaining certificates from Let's Encrypt and Boulder. However, it seems that EJBCA does not include the {{Location}} header in the response to finalizing the order. {{AcmeClientSpi#obtainCertificate}} should instead attempt to use the Location header from the response to {{newOrder}}.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (ELY-1975) Update AcmeClientSpi#obtainCertificate so that it obtains the order URL from the response to newOrder
by Farah Juma (Jira)
[ https://issues.redhat.com/browse/ELY-1975?page=com.atlassian.jira.plugin.... ]
Farah Juma updated ELY-1975:
----------------------------
Summary: Update AcmeClientSpi#obtainCertificate so that it obtains the order URL from the response to newOrder (was: Update AcmeClientSpi#obtainCertificate so that it obtains the order URL from the newOrder request)
> Update AcmeClientSpi#obtainCertificate so that it obtains the order URL from the response to newOrder
> -----------------------------------------------------------------------------------------------------
>
> Key: ELY-1975
> URL: https://issues.redhat.com/browse/ELY-1975
> Project: WildFly Elytron
> Issue Type: Task
> Reporter: Farah Juma
> Assignee: Farah Juma
> Priority: Major
>
> {{AcmeClientSpi#obtainCertificate}} currently determines the order URL that should be used by taking the {{Location}} header from the response to finalizing the order (the ACME protocol specification has an example response that includes this header). This works when obtaining certificates from Let's Encrypt and Boulder. However, it seems that EJBCA does not include the {{Location}} header in the response to finalizing the order. {{AcmeClientSpi#obtainCertificate}} should instead attempt to use the Location header from the response to {{newOrder}}.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (JGRP-2470) JBDC_PING can face a split-brain issue when restarting a coordinator node
by Radoslav Husar (Jira)
[ https://issues.redhat.com/browse/JGRP-2470?page=com.atlassian.jira.plugin... ]
Radoslav Husar commented on JGRP-2470:
--------------------------------------
bq. What's the rationale for reverting JGRP-2199? If the old coord removes all information, the new one will re-insert it (actually multiple times, if configured)...
That's the problem – these are not synchronized in any way and thus it's racey. As Masafumi explained, the problem is that if these are ordered in the database in a way 1. the new coordinator reinserted the data and 2. then the stopping coordinators clear table query {{clearTable(String clustername)}} the restarted member won't discovery anything and start a singleton cluster.
bq. It seems that a periodic findMembers() triggered by MERGE3 can heal the singleton clusters situation. The findMembers() can insert their own node information when the table is empty. But this happens after the interval calculated by Math.max(min_interval, Util.random(max_interval) + max_interval/2) where min_interval is 10000 and max_interval is 30000 in JBoss EAP by default. And, the actual merge happens after this. So, it could take a long time to heal the singleton clusters situation.
We don't even need to go into details on merging partitions, discovery can never lead by design into this as this causes partitions and data loss.
> JBDC_PING can face a split-brain issue when restarting a coordinator node
> -------------------------------------------------------------------------
>
> Key: JGRP-2470
> URL: https://issues.redhat.com/browse/JGRP-2470
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 4.1.9, 4.0.22
> Reporter: Masafumi Miura
> Assignee: Radoslav Husar
> Priority: Major
> Fix For: 4.2.5
>
>
> After [the change|https://github.com/belaban/JGroups/commit/215cdb6] for JGRP-2199, JDBC_PING deletes all entries from the table during the shutdown of the coordinator node.
> This behavior has a possibility to cause a split-brain when restarting a coordinator node. Because, as all entries are lost in the following scenario, the restarting node can not find any information about existing nodes from the table and does not form a cluster.
> 0. node1 and node2 form a cluster. The node1 is a coordinator.
> 1. Trigger a restart of the node1
> 2. The node1 removes their node information from the table
> 3. The node2 becomes a new coordinator
> 4. The node2 updates their node information in the table
> 5. The node1 clears all entries from the table
> 6. The node1 starts again
> 7. The node1 does not join the existing cluster because there's no node information in the table
> Note: If step 5 happens before step 4, the split-brain issue does not happen. However, as step 4 and step 5 happen on different nodes, these steps can happen in parallel. So, the order is undefined. So, for example, if the shutdown of node1 takes a long time, there's a high possibility to face this issue.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months