[JBoss JIRA] (JGRP-2234) Unlocked locks stay locked forever
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2234?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2234:
--------------------------------
The current design has the concept of Owner; multiple owners would entail quite a lot of changes (owner is also used partly for identity).
Yes, handling data reconciliation after partitions heal are particularly difficult with the {{Lock}} interface (counters are simpler). Perhaps {{Lock}} only makes sense in a CP system, such as jgroups-raft...
This definitely needs more thought... I'll tackle that in JGRP-2249.
> Unlocked locks stay locked forever
> ----------------------------------
>
> Key: JGRP-2234
> URL: https://issues.jboss.org/browse/JGRP-2234
> Project: JGroups
> Issue Type: Bug
> Reporter: Bram Klein Gunnewiek
> Assignee: Bela Ban
> Fix For: 4.0.11
>
> Attachments: ClusterSplitLockTest.java, jg_clusterlock_output_testfail.txt
>
>
> As discussed in the mailing list we have issues where locks from the central lock protocol stay locked forever when the coordinator of the cluster disconnects. We can reproduce this with the attached ClusterSplitLockTest.java. Its a race condition and we need to run the test a lot of times (sometimes > 20) before we encounter a failure.
> What we think is happening:
> In a three node cluster (node A, B and C where node A is the coordinator) unlock requests from B and/or C can be missed when node A leaves and B and/or C don't have the new view installed yet. When, for example, node B takes over coordination it creates the lock table based on the back-ups. Lets say node C has locked the lock with name 'lockX'. Node C performs an unlock of 'lockX' just after node A (gracefully) leaves and sends the unlock request to node A since node C doesn't have the correct view installed yet. Node B has recreated the lock table where 'lockX' is locked by Node C. Node C doesn't resend the unlock request so 'lockX' gets locked forever.
> Attached is the testng test we wrote and the output of a test failure.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (JGRP-2234) Unlocked locks stay locked forever
by Bram Klein Gunnewiek (JIRA)
[ https://issues.jboss.org/browse/JGRP-2234?page=com.atlassian.jira.plugin.... ]
Bram Klein Gunnewiek edited comment on JGRP-2234 at 2/9/18 4:47 AM:
--------------------------------------------------------------------
Not sure how it can be solved, but during a merge of [A,B] and [C,D] there are basically two locktables (of [A,B] and [C,D]). When merging those lock tables you can detect that both lock tables have an owner for lockX (lets say A and C both locked lockX). When B is the new coordinator of the merged cluster, you could implement it so that B saves both A and C as lockowner and only grants new lock() requests for lockX when both A and C performed an unlock();
I don't think its possible to take away locks, the only thing you could do is implement some sort of event system that fires an event when it detects that a lock has multiple holders after a cluster merge. This could be handled (or ignored) by the application using JGroups, I don't think its something JGroups itself can solve.
was (Author: bramklg):
Not sure how it can be solved, but during a merge of [A,B] and [C,D] there are basically two locktables (of [A,B] and [C,D]). When merging those lock tables you can detect that both lock tables have an owner for lockX (lets say A and C both locked lockX). Lets say B is the new coordinator of the merged cluster. You could implement it so that B saves both A and C as lockowner and only grants new lock() requests for lockX when both A and C performed an unlock();
I don't think its possible to take away locks, the only thing you could do is implement some sort of event system that fires an event when it detects that a lock has multiple holders after a cluster merge. This could be handled (or ignored) by the application using JGroups, I don't think its something JGroups itself can solve.
> Unlocked locks stay locked forever
> ----------------------------------
>
> Key: JGRP-2234
> URL: https://issues.jboss.org/browse/JGRP-2234
> Project: JGroups
> Issue Type: Bug
> Reporter: Bram Klein Gunnewiek
> Assignee: Bela Ban
> Fix For: 4.0.11
>
> Attachments: ClusterSplitLockTest.java, jg_clusterlock_output_testfail.txt
>
>
> As discussed in the mailing list we have issues where locks from the central lock protocol stay locked forever when the coordinator of the cluster disconnects. We can reproduce this with the attached ClusterSplitLockTest.java. Its a race condition and we need to run the test a lot of times (sometimes > 20) before we encounter a failure.
> What we think is happening:
> In a three node cluster (node A, B and C where node A is the coordinator) unlock requests from B and/or C can be missed when node A leaves and B and/or C don't have the new view installed yet. When, for example, node B takes over coordination it creates the lock table based on the back-ups. Lets say node C has locked the lock with name 'lockX'. Node C performs an unlock of 'lockX' just after node A (gracefully) leaves and sends the unlock request to node A since node C doesn't have the correct view installed yet. Node B has recreated the lock table where 'lockX' is locked by Node C. Node C doesn't resend the unlock request so 'lockX' gets locked forever.
> Attached is the testng test we wrote and the output of a test failure.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (JGRP-2234) Unlocked locks stay locked forever
by Bram Klein Gunnewiek (JIRA)
[ https://issues.jboss.org/browse/JGRP-2234?page=com.atlassian.jira.plugin.... ]
Bram Klein Gunnewiek commented on JGRP-2234:
--------------------------------------------
Not sure how it can be solved, but during a merge of [A,B] and [C,D] there are basically two locktables (of [A,B] and [C,D]). When merging those lock tables you can detect that both lock tables have an owner for lockX (lets say A and C both locked lockX). Lets say B is the new coordinator of the merged cluster. You could implement it so that B saves both A and C as lockowner and only grants new lock() requests for lockX when both A and C performed an unlock();
I don't think its possible to take away locks, the only thing you could do is implement some sort of event system that fires an event when it detects that a lock has multiple holders after a cluster merge. This could be handled (or ignored) by the application using JGroups, I don't think its something JGroups itself can solve.
> Unlocked locks stay locked forever
> ----------------------------------
>
> Key: JGRP-2234
> URL: https://issues.jboss.org/browse/JGRP-2234
> Project: JGroups
> Issue Type: Bug
> Reporter: Bram Klein Gunnewiek
> Assignee: Bela Ban
> Fix For: 4.0.11
>
> Attachments: ClusterSplitLockTest.java, jg_clusterlock_output_testfail.txt
>
>
> As discussed in the mailing list we have issues where locks from the central lock protocol stay locked forever when the coordinator of the cluster disconnects. We can reproduce this with the attached ClusterSplitLockTest.java. Its a race condition and we need to run the test a lot of times (sometimes > 20) before we encounter a failure.
> What we think is happening:
> In a three node cluster (node A, B and C where node A is the coordinator) unlock requests from B and/or C can be missed when node A leaves and B and/or C don't have the new view installed yet. When, for example, node B takes over coordination it creates the lock table based on the back-ups. Lets say node C has locked the lock with name 'lockX'. Node C performs an unlock of 'lockX' just after node A (gracefully) leaves and sends the unlock request to node A since node C doesn't have the correct view installed yet. Node B has recreated the lock table where 'lockX' is locked by Node C. Node C doesn't resend the unlock request so 'lockX' gets locked forever.
> Attached is the testng test we wrote and the output of a test failure.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (JGRP-2234) Unlocked locks stay locked forever
by Bram Klein Gunnewiek (JIRA)
[ https://issues.jboss.org/browse/JGRP-2234?page=com.atlassian.jira.plugin.... ]
Bram Klein Gunnewiek edited comment on JGRP-2234 at 2/9/18 4:39 AM:
--------------------------------------------------------------------
[~belaban] Thanks for resolving this! Using the testcase provided is fine obviously.
A side note about the "multiple members holding the same lock in a split brain": since JGroups can't prevent these scenario's I would suggest that in case of "multiple lock holders" the lock should be released again when *all* lock holders have released it. E.G. in a cluster of [A,B,C,D] that splits into [A,B][C,D] where A and C hold the same lock (lock X) and they merge back into one cluster [A,B,C,D] I think think lock X should be 'released' after both A and C unlock it. But that might be something to think about in JGRP-2249.
was (Author: bramklg):
[~belaban] Thanks for resolving this! Using the testcase provided is fine obviously.
A side note about the "multiple members holding the same lock in a split brain": since JGroups can't prevent these scenario's I would suggest that in case of "multiple lock holders" the lock should be released again when *all* lock owners have released it. E.G. in a cluster of [A,B,C,D] that splits into [A,B][C,D] where A and C hold the same lock (lock X) and they merge back into one cluster [A,B,C,D] I think think lock X should be 'released' after both A and C unlock it. But that might be something to think about in JGRP-2249.
> Unlocked locks stay locked forever
> ----------------------------------
>
> Key: JGRP-2234
> URL: https://issues.jboss.org/browse/JGRP-2234
> Project: JGroups
> Issue Type: Bug
> Reporter: Bram Klein Gunnewiek
> Assignee: Bela Ban
> Fix For: 4.0.11
>
> Attachments: ClusterSplitLockTest.java, jg_clusterlock_output_testfail.txt
>
>
> As discussed in the mailing list we have issues where locks from the central lock protocol stay locked forever when the coordinator of the cluster disconnects. We can reproduce this with the attached ClusterSplitLockTest.java. Its a race condition and we need to run the test a lot of times (sometimes > 20) before we encounter a failure.
> What we think is happening:
> In a three node cluster (node A, B and C where node A is the coordinator) unlock requests from B and/or C can be missed when node A leaves and B and/or C don't have the new view installed yet. When, for example, node B takes over coordination it creates the lock table based on the back-ups. Lets say node C has locked the lock with name 'lockX'. Node C performs an unlock of 'lockX' just after node A (gracefully) leaves and sends the unlock request to node A since node C doesn't have the correct view installed yet. Node B has recreated the lock table where 'lockX' is locked by Node C. Node C doesn't resend the unlock request so 'lockX' gets locked forever.
> Attached is the testng test we wrote and the output of a test failure.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (JGRP-2234) Unlocked locks stay locked forever
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2234?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2234:
--------------------------------
[~bramklg] You're thinking about some kind of refcount? Possible, but that means lock holders can hold on to acquired locks even after a partition has healed...
Even worse though: taking away a lock is not something easily doable in the java.util.lock.Lock interface...
> Unlocked locks stay locked forever
> ----------------------------------
>
> Key: JGRP-2234
> URL: https://issues.jboss.org/browse/JGRP-2234
> Project: JGroups
> Issue Type: Bug
> Reporter: Bram Klein Gunnewiek
> Assignee: Bela Ban
> Fix For: 4.0.11
>
> Attachments: ClusterSplitLockTest.java, jg_clusterlock_output_testfail.txt
>
>
> As discussed in the mailing list we have issues where locks from the central lock protocol stay locked forever when the coordinator of the cluster disconnects. We can reproduce this with the attached ClusterSplitLockTest.java. Its a race condition and we need to run the test a lot of times (sometimes > 20) before we encounter a failure.
> What we think is happening:
> In a three node cluster (node A, B and C where node A is the coordinator) unlock requests from B and/or C can be missed when node A leaves and B and/or C don't have the new view installed yet. When, for example, node B takes over coordination it creates the lock table based on the back-ups. Lets say node C has locked the lock with name 'lockX'. Node C performs an unlock of 'lockX' just after node A (gracefully) leaves and sends the unlock request to node A since node C doesn't have the correct view installed yet. Node B has recreated the lock table where 'lockX' is locked by Node C. Node C doesn't resend the unlock request so 'lockX' gets locked forever.
> Attached is the testng test we wrote and the output of a test failure.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (JGRP-2234) Unlocked locks stay locked forever
by Bram Klein Gunnewiek (JIRA)
[ https://issues.jboss.org/browse/JGRP-2234?page=com.atlassian.jira.plugin.... ]
Bram Klein Gunnewiek edited comment on JGRP-2234 at 2/9/18 4:38 AM:
--------------------------------------------------------------------
[~belaban] Thanks for resolving this! Using the testcase provided is fine obviously.
A side note about the "multiple members holding the same lock in a split brain": since JGroups can't prevent these scenario's I would suggest that in case of "multiple lock holders" the lock should be released again when *all* lock owners have released it. E.G. in a cluster of [A,B,C,D] that splits into [A,B][C,D] where A and C hold the same lock (lock X) and they merge back into one cluster [A,B,C,D] I think think lock X should be 'released' after both A and C unlock it. But that might be something to think about in JGRP-2249.
was (Author: bramklg):
[~belaban] Thanks for resolving this! Using the testcase provided is fine obviously.
A side note about the "multiple members holding the same lock in a split brain": since JGroups can't prevent these scenario's I would suggest that in case of "multiple locks" the lock should be released again when *all* lock owners have released it. E.G. in a cluster of [A,B,C,D] that splits into [A,B][C,D] where A and C hold the same lock (lock X) and they merge back into one cluster [A,B,C,D] I think think lock X should be 'released' after both A and C unlock it. But that might be something to think about in JGRP-2249.
> Unlocked locks stay locked forever
> ----------------------------------
>
> Key: JGRP-2234
> URL: https://issues.jboss.org/browse/JGRP-2234
> Project: JGroups
> Issue Type: Bug
> Reporter: Bram Klein Gunnewiek
> Assignee: Bela Ban
> Fix For: 4.0.11
>
> Attachments: ClusterSplitLockTest.java, jg_clusterlock_output_testfail.txt
>
>
> As discussed in the mailing list we have issues where locks from the central lock protocol stay locked forever when the coordinator of the cluster disconnects. We can reproduce this with the attached ClusterSplitLockTest.java. Its a race condition and we need to run the test a lot of times (sometimes > 20) before we encounter a failure.
> What we think is happening:
> In a three node cluster (node A, B and C where node A is the coordinator) unlock requests from B and/or C can be missed when node A leaves and B and/or C don't have the new view installed yet. When, for example, node B takes over coordination it creates the lock table based on the back-ups. Lets say node C has locked the lock with name 'lockX'. Node C performs an unlock of 'lockX' just after node A (gracefully) leaves and sends the unlock request to node A since node C doesn't have the correct view installed yet. Node B has recreated the lock table where 'lockX' is locked by Node C. Node C doesn't resend the unlock request so 'lockX' gets locked forever.
> Attached is the testng test we wrote and the output of a test failure.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (JGRP-2234) Unlocked locks stay locked forever
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2234?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2234:
--------------------------------
[~dwhitejazz] No action needed from the application; this is all handled by JGroups
> Unlocked locks stay locked forever
> ----------------------------------
>
> Key: JGRP-2234
> URL: https://issues.jboss.org/browse/JGRP-2234
> Project: JGroups
> Issue Type: Bug
> Reporter: Bram Klein Gunnewiek
> Assignee: Bela Ban
> Fix For: 4.0.11
>
> Attachments: ClusterSplitLockTest.java, jg_clusterlock_output_testfail.txt
>
>
> As discussed in the mailing list we have issues where locks from the central lock protocol stay locked forever when the coordinator of the cluster disconnects. We can reproduce this with the attached ClusterSplitLockTest.java. Its a race condition and we need to run the test a lot of times (sometimes > 20) before we encounter a failure.
> What we think is happening:
> In a three node cluster (node A, B and C where node A is the coordinator) unlock requests from B and/or C can be missed when node A leaves and B and/or C don't have the new view installed yet. When, for example, node B takes over coordination it creates the lock table based on the back-ups. Lets say node C has locked the lock with name 'lockX'. Node C performs an unlock of 'lockX' just after node A (gracefully) leaves and sends the unlock request to node A since node C doesn't have the correct view installed yet. Node B has recreated the lock table where 'lockX' is locked by Node C. Node C doesn't resend the unlock request so 'lockX' gets locked forever.
> Attached is the testng test we wrote and the output of a test failure.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (DROOLS-2311) Drools Engine not able to load rules in spring-Boot project
by Chandresh Mishra (JIRA)
[ https://issues.jboss.org/browse/DROOLS-2311?page=com.atlassian.jira.plugi... ]
Chandresh Mishra commented on DROOLS-2311:
------------------------------------------
[~mfusco] Are you able to replicate the issue?
Regards
Chandresh Mishra
> Drools Engine not able to load rules in spring-Boot project
> -----------------------------------------------------------
>
> Key: DROOLS-2311
> URL: https://issues.jboss.org/browse/DROOLS-2311
> Project: Drools
> Issue Type: Bug
> Components: core engine
> Affects Versions: 7.5.0.Final
> Reporter: Chandresh Mishra
> Assignee: Mario Fusco
> Attachments: Drools-Issue.rar
>
>
> am using the Drools 7.5.0-final in the spring Boot project.
> I am working on a multi-module Gradle project. Rules and kmodule.xml is present in the sub-module project. This module is packaged as a Jar inside a fat spring boot Jar file.
> I am building the container from the classpath.
> KieContainer kContainer = kieService.newKieClasspathContainer();
> on server start-up (java -jar) it loads the Kmodule.xml and prints all the kbase and sessionName.
> But when I fire rules it returns zero rules fired.
> It works fine when I run as spring boot app from Eclipse. Please help.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (JGRP-2234) Unlocked locks stay locked forever
by Bram Klein Gunnewiek (JIRA)
[ https://issues.jboss.org/browse/JGRP-2234?page=com.atlassian.jira.plugin.... ]
Bram Klein Gunnewiek commented on JGRP-2234:
--------------------------------------------
[~belaban] Thanks for resolving this! Using the testcase provided is fine obviously.
A side note about the "multiple members holding the same lock in a split brain": since JGroups can't prevent these scenario's I would suggest that in case of "multiple locks" the lock should be released again when *all* lock owners have released it. E.G. in a cluster of [A,B,C,D] that splits into [A,B][C,D] where A and C hold the same lock (lock X) and they merge back into one cluster [A,B,C,D] I think think lock X should be 'released' after both A and C unlock it. But that might be something to think about in JGRP-2249.
> Unlocked locks stay locked forever
> ----------------------------------
>
> Key: JGRP-2234
> URL: https://issues.jboss.org/browse/JGRP-2234
> Project: JGroups
> Issue Type: Bug
> Reporter: Bram Klein Gunnewiek
> Assignee: Bela Ban
> Fix For: 4.0.11
>
> Attachments: ClusterSplitLockTest.java, jg_clusterlock_output_testfail.txt
>
>
> As discussed in the mailing list we have issues where locks from the central lock protocol stay locked forever when the coordinator of the cluster disconnects. We can reproduce this with the attached ClusterSplitLockTest.java. Its a race condition and we need to run the test a lot of times (sometimes > 20) before we encounter a failure.
> What we think is happening:
> In a three node cluster (node A, B and C where node A is the coordinator) unlock requests from B and/or C can be missed when node A leaves and B and/or C don't have the new view installed yet. When, for example, node B takes over coordination it creates the lock table based on the back-ups. Lets say node C has locked the lock with name 'lockX'. Node C performs an unlock of 'lockX' just after node A (gracefully) leaves and sends the unlock request to node A since node C doesn't have the correct view installed yet. Node B has recreated the lock table where 'lockX' is locked by Node C. Node C doesn't resend the unlock request so 'lockX' gets locked forever.
> Attached is the testng test we wrote and the output of a test failure.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (WFLY-9784) Splitting as transaction module to not be gathering all dependencies
by Ondra Chaloupka (JIRA)
Ondra Chaloupka created WFLY-9784:
-------------------------------------
Summary: Splitting as transaction module to not be gathering all dependencies
Key: WFLY-9784
URL: https://issues.jboss.org/browse/WFLY-9784
Project: WildFly
Issue Type: Task
Components: Transactions
Affects Versions: 11.0.0.Final
Reporter: Ondra Chaloupka
Assignee: Ondra Chaloupka
Attachments: eap71-txn-dependencies.jpg
Current org.jboss.as.transaction module is heavy lifting and has many dependencies and lot of work competencies. E.g. org.jboss.jts module depends on that too.
It should be investigated how to disentangle all the dependencies.
Plus this could be beneficiary for WildFly Swarm if we are able to split the dependency chains, maybe not using one big jar in the transaction module but provide more granularity. Needs to be investigated.
See attached image for a view of dependencies.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months