[JBoss JIRA] (JGRP-2143) TP: use only one thread per member to pass up regular messages
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2143?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-2143:
---------------------------
Description:
This applies only to _regular_ messages; OOB and internal messages are processed by passing them to the thread pool directly when they're received.
The processing of a message received from B is as follows:
* A regular message (or message batch) is assigned a thread from the thread pool and passed up to the reliability protocol, e.g. NAKACK2 or UNICAST3.
* There is is added to the table for B.
* The thread sees if another thread is already delivering messages from B to the application. If not, it grabs as many consecutive (ordered) messages from the table as it can and delivers them to the application. Otherwise, it returns and can be assigned other tasks.
The problem here is that more than one thread may be passing up messages from a given sender B; only at the NAKACK2 or UNICAST3 level will a single thread be selected to deliver the messages to the application.
This causes higher thread pool usage than required, with all of its drawbacks, e.g. more context switching, higher contention on adding messages to the table for B, and possibly exhaustion of the thread pool.
An example of where service is denied or delayed:
* We have a cluster of \{A,B,C,D\}
* A receives 10 messages from B, 4 from C and 1 from D
* The thread pool's max size is 20
* The 10 messages from B are processed; all 10 threads add their messages to the table, but only 1 delivers them to the application and the other 9 return to the pool
* 4 messages from C are added to C's table, 1 thread delivers them and 3 return
* The 1 message from D is added to D's table and the same thread is used to deliver the message up the stack to the application
So while we receive 15 messages, effectively only 3 threads are needed to deliver them to the application: as these are regular messages, they need to be delivered in _sender order_.
The 9 threads which process messages from B are only adding them to B's table and then return immediately. This causes increased context switching, plus more contention on B's table (which is synchronized), and possibly exhaustion of the thread pool. For example, if the pool's max size was only 10, then processing the first 10 messages from B would exhaust the table, and the other messages from C and D would be processed in newly spawned threads.
SOLUTION
* (Only applicable to _regular_ messages)
* When a message (or batch) from sender P is received, we check if another thread is already passing up messages from B. If not, we pass the message up by grabbing a thread from the thread pool. This will add the message to P's table and deliver as many messages (from from the table) as possible to the application.
* If there's currently a thread delivering P's message, we simply add the message (or batch) to a queue for P and return.
* When the delivery thread returns, it checks the queue for P and delivers all queued messages, or returns if the queue is empty.
* (The queue is actually a MessageBatch, and new messages are simply appended to it. On delivery, the batch is cleared)
The effects of this for regular messages are
* Fewer threads: the thread pool only has a max of <cluster-members> threads for regular messages where <cluster-members> is the number of members in the cluster from whom we are concurrently receiving messages. E.g. for a cluster \{A,B,C,D\}, if we're receiving messages at the same time from all members, then the max size is 4.
** Of course, OOB and internal messages, plus timer tasks will add to this number.
* Less contention on the table for a given member: instead of 10 threads all adding their messages to B's table (contention on the table lock) and then CASing a boolean, only 1 thread ever adds and removes messages to/from the table. This means uncontended (= fast) lock acquisition for regular messages (of course, if we use OOB messages, then we do have contention).
* Appending to a batch is much faster then adding to a table
* The downside is that we're storing messages actually twice: once in the batch for P and once in P's table. But these are arrays of pointers, so not a lot of memory required.
Example: the 10 threads for messages from B above, will create a batch of 9 messages in B's queue and grab 1 thread from the pool to deliver its message. When the thread is done, it will grab the message batch of 9 and also add it to the table and deliver it.
This is similar to the bulkhead pattern \[1\].
\[1\] http://stackoverflow.com/questions/30391809/what-is-bulkhead-pattern-used...
was:
This applies only to _regular_ messages; OOB and internal messages are processed by passing them to the thread pool directly when they they're received.
The processing of a message received from B is as follows:
* A regular message (or message batch) is assigned a thread from the thread pool and passed up to the reliability protocol, e.g. NAKACK2 or UNICAST3.
* There is is added to the table for B.
* The thread sees if another thread is already delivering messages from B to the application. If not, it grabs as many consecutive (ordered) messages from the table as it can and delivers them to the application. Otherwise, it returns and can be assigned other tasks.
The problem here is that more than one thread may be passing up messages from a given sender B; only at the NAKACK2 or UNICAST3 level will a single thread be selected to deliver the messages to the application.
This causes higher thread pool usage than required, with all of its drawbacks, e.g. more context switching, higher contention on adding messages to the table for B, and possibly exhaustion of the thread pool.
An example of where service is denied or delayed:
* We have a cluster of \{A,B,C,D\}
* A receives 10 messages from B, 4 from C and 1 from D
* The thread pool's max size is 20
* The 10 messages from B are processed; all 10 threads add their messages to the table, but only 1 delivers them to the application and the other 9 return to the pool
* 4 messages from C are added to C's table, 1 thread delivers them and 3 return
* The 1 message from D is added to D's table and the same thread is used to deliver the message up the stack to the application
So while we receive 15 messages, effectively only 3 threads are needed to deliver them to the application: as these are regular messages, they need to be delivered in _sender order_.
The 9 threads which process messages from B are only adding them to B's table and then return immediately. This causes increased context switching, plus more contention on B's table (which is synchronized), and possibly exhaustion of the thread pool. For example, if the pool's max size was only 10, then processing the first 10 messages from B would exhaust the table, and the other messages from C and D would be processed in newly spawned threads.
SOLUTION
* (Only applicable to _regular_ messages)
* When a message (or batch) from sender P is received, we check if another thread is already passing up messages from B. If not, we pass the message up by grabbing a thread from the thread pool. This will add the message to P's table and deliver as many messages (from from the table) as possible to the application.
* If there's currently a thread delivering P's message, we simply add the message (or batch) to a queue for P and return.
* When the delivery thread returns, it checks the queue for P and delivers all queued messages, or returns if the queue is empty.
* (The queue is actually a MessageBatch, and new messages are simply appended to it. On delivery, the batch is cleared)
The effects of this for regular messages are
* Fewer threads: the thread pool only has a max of <cluster-members> threads for regular messages where <cluster-members> is the number of members in the cluster from whom we are concurrently receiving messages. E.g. for a cluster \{A,B,C,D\}, if we're receiving messages at the same time from all members, then the max size is 4.
** Of course, OOB and internal messages, plus timer tasks will add to this number.
* Less contention on the table for a given member: instead of 10 threads all adding their messages to B's table (contention on the table lock) and then CASing a boolean, only 1 thread ever adds and removes messages to/from the table. This means uncontended (= fast) lock acquisition for regular messages (of course, if we use OOB messages, then we do have contention).
* Appending to a batch is much faster then adding to a table
* The downside is that we're storing messages actually twice: once in the batch for P and once in P's table. But these are arrays of pointers, so not a lot of memory required.
Example: the 10 threads for messages from B above, will create a batch of 9 messages in B's queue and grab 1 thread from the pool to deliver its message. When the thread is done, it will grab the message batch of 9 and also add it to the table and deliver it.
This is similar to the bulkhead pattern \[1\].
\[1\] http://stackoverflow.com/questions/30391809/what-is-bulkhead-pattern-used...
> TP: use only one thread per member to pass up regular messages
> --------------------------------------------------------------
>
> Key: JGRP-2143
> URL: https://issues.jboss.org/browse/JGRP-2143
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 4.0
>
>
> This applies only to _regular_ messages; OOB and internal messages are processed by passing them to the thread pool directly when they're received.
> The processing of a message received from B is as follows:
> * A regular message (or message batch) is assigned a thread from the thread pool and passed up to the reliability protocol, e.g. NAKACK2 or UNICAST3.
> * There is is added to the table for B.
> * The thread sees if another thread is already delivering messages from B to the application. If not, it grabs as many consecutive (ordered) messages from the table as it can and delivers them to the application. Otherwise, it returns and can be assigned other tasks.
> The problem here is that more than one thread may be passing up messages from a given sender B; only at the NAKACK2 or UNICAST3 level will a single thread be selected to deliver the messages to the application.
> This causes higher thread pool usage than required, with all of its drawbacks, e.g. more context switching, higher contention on adding messages to the table for B, and possibly exhaustion of the thread pool.
> An example of where service is denied or delayed:
> * We have a cluster of \{A,B,C,D\}
> * A receives 10 messages from B, 4 from C and 1 from D
> * The thread pool's max size is 20
> * The 10 messages from B are processed; all 10 threads add their messages to the table, but only 1 delivers them to the application and the other 9 return to the pool
> * 4 messages from C are added to C's table, 1 thread delivers them and 3 return
> * The 1 message from D is added to D's table and the same thread is used to deliver the message up the stack to the application
> So while we receive 15 messages, effectively only 3 threads are needed to deliver them to the application: as these are regular messages, they need to be delivered in _sender order_.
> The 9 threads which process messages from B are only adding them to B's table and then return immediately. This causes increased context switching, plus more contention on B's table (which is synchronized), and possibly exhaustion of the thread pool. For example, if the pool's max size was only 10, then processing the first 10 messages from B would exhaust the table, and the other messages from C and D would be processed in newly spawned threads.
> SOLUTION
> * (Only applicable to _regular_ messages)
> * When a message (or batch) from sender P is received, we check if another thread is already passing up messages from B. If not, we pass the message up by grabbing a thread from the thread pool. This will add the message to P's table and deliver as many messages (from from the table) as possible to the application.
> * If there's currently a thread delivering P's message, we simply add the message (or batch) to a queue for P and return.
> * When the delivery thread returns, it checks the queue for P and delivers all queued messages, or returns if the queue is empty.
> * (The queue is actually a MessageBatch, and new messages are simply appended to it. On delivery, the batch is cleared)
> The effects of this for regular messages are
> * Fewer threads: the thread pool only has a max of <cluster-members> threads for regular messages where <cluster-members> is the number of members in the cluster from whom we are concurrently receiving messages. E.g. for a cluster \{A,B,C,D\}, if we're receiving messages at the same time from all members, then the max size is 4.
> ** Of course, OOB and internal messages, plus timer tasks will add to this number.
> * Less contention on the table for a given member: instead of 10 threads all adding their messages to B's table (contention on the table lock) and then CASing a boolean, only 1 thread ever adds and removes messages to/from the table. This means uncontended (= fast) lock acquisition for regular messages (of course, if we use OOB messages, then we do have contention).
> * Appending to a batch is much faster then adding to a table
> * The downside is that we're storing messages actually twice: once in the batch for P and once in P's table. But these are arrays of pointers, so not a lot of memory required.
> Example: the 10 threads for messages from B above, will create a batch of 9 messages in B's queue and grab 1 thread from the pool to deliver its message. When the thread is done, it will grab the message batch of 9 and also add it to the table and deliver it.
> This is similar to the bulkhead pattern \[1\].
> \[1\] http://stackoverflow.com/questions/30391809/what-is-bulkhead-pattern-used...
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months
[JBoss JIRA] (WFLY-7849) Missing input username wildcard for role/attribute search in Elytron ldap-realm
by Jan Kalina (JIRA)
[ https://issues.jboss.org/browse/WFLY-7849?page=com.atlassian.jira.plugin.... ]
Jan Kalina moved ELY-860 to WFLY-7849:
--------------------------------------
Project: WildFly (was: WildFly Elytron)
Key: WFLY-7849 (was: ELY-860)
Component/s: Security
(was: Realms)
Affects Version/s: (was: 1.1.0.Beta13)
> Missing input username wildcard for role/attribute search in Elytron ldap-realm
> -------------------------------------------------------------------------------
>
> Key: WFLY-7849
> URL: https://issues.jboss.org/browse/WFLY-7849
> Project: WildFly
> Issue Type: Bug
> Components: Security
> Reporter: Jan Kalina
> Assignee: Jan Kalina
> Priority: Critical
>
> Scenario: I am trying to assign role from LDAP to user. I would like to use input username (e.g. admin) in filter, not full user DN (e.g. uid=admin,ou=People,dc=jboss,dc=org). It seems Elytron ldap-realm does not provide any wildcard which can be used for input username.
> In EAP 7.0 (with PicketBox), LdapExtLoginModule provides roleFilter option which filter can contain following wildcards:
> * {{\{0\}}} - for input username
> * {{\{1\}}} - for authenticated full user DN
> It seems that Elytron supports only wildcard for authenticated full user DN (through {{\{0\}}} wildcard). Wildcard for input username should be added.
> It would be useful, when order of wildcards will be the same as in EAP 7.0 - i.e. not just add the new {{\{1\}}} for input username, but use {{\{0\}}} for input username and {{\{1\}}} for authenticated full user DN. This order is also better due to wildcard {{\{0\}}} will mean the same in identity filter and in role/attribute filter.
> Missing this feature in Elytron can lead to situation when migration from PicketBox to Elytron will not be possible since LDAP structure for role assignment used by legacy solution will not be able to work correctly with Elytron.
> Example of usage:
> I would like to use filter like {{(description=SOME_INPUT_USERNAME_WILDCARD)}} for assigning role JBossAdmin to user jduke in following ldif:
> {code}
> dn: ou=People,dc=jboss,dc=org
> objectclass: top
> objectclass: organizationalUnit
> ou: People
> dn: uid=jduke,ou=People,dc=jboss,dc=org
> objectclass: top
> objectclass: person
> objectclass: inetOrgPerson
> uid: jduke
> cn: Java Duke
> sn: Duke
> userPassword: Password
> dn: uid=notUsedUser,ou=People,dc=jboss,dc=org
> objectclass: top
> objectclass: person
> objectclass: inetOrgPerson
> uid: notUsedUser
> cn: not used user
> sn: notUsedUser
> userPassword: Password
> dn: ou=Roles,dc=jboss,dc=org
> objectclass: top
> objectclass: organizationalUnit
> ou: Roles
> dn: cn=JBossAdmin,ou=Roles,dc=jboss,dc=org
> objectclass: top
> objectclass: groupOfNames
> cn: JBossAdmin
> member: uid=notUsedUser,ou=People,dc=jboss,dc=org
> description: jduke
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months
[JBoss JIRA] (ELY-860) Missing input username wildcard for role/attribute search in Elytron ldap-realm
by Jan Kalina (JIRA)
Jan Kalina created ELY-860:
------------------------------
Summary: Missing input username wildcard for role/attribute search in Elytron ldap-realm
Key: ELY-860
URL: https://issues.jboss.org/browse/ELY-860
Project: WildFly Elytron
Issue Type: Bug
Components: Realms
Affects Versions: 1.1.0.Beta13
Reporter: Jan Kalina
Assignee: Jan Kalina
Priority: Critical
Scenario: I am trying to assign role from LDAP to user. I would like to use input username (e.g. admin) in filter, not full user DN (e.g. uid=admin,ou=People,dc=jboss,dc=org). It seems Elytron ldap-realm does not provide any wildcard which can be used for input username.
In EAP 7.0 (with PicketBox), LdapExtLoginModule provides roleFilter option which filter can contain following wildcards:
* {{\{0\}}} - for input username
* {{\{1\}}} - for authenticated full user DN
It seems that Elytron supports only wildcard for authenticated full user DN (through {{\{0\}}} wildcard). Wildcard for input username should be added.
It would be useful, when order of wildcards will be the same as in EAP 7.0 - i.e. not just add the new {{\{1\}}} for input username, but use {{\{0\}}} for input username and {{\{1\}}} for authenticated full user DN. This order is also better due to wildcard {{\{0\}}} will mean the same in identity filter and in role/attribute filter.
Missing this feature in Elytron can lead to situation when migration from PicketBox to Elytron will not be possible since LDAP structure for role assignment used by legacy solution will not be able to work correctly with Elytron.
Example of usage:
I would like to use filter like {{(description=SOME_INPUT_USERNAME_WILDCARD)}} for assigning role JBossAdmin to user jduke in following ldif:
{code}
dn: ou=People,dc=jboss,dc=org
objectclass: top
objectclass: organizationalUnit
ou: People
dn: uid=jduke,ou=People,dc=jboss,dc=org
objectclass: top
objectclass: person
objectclass: inetOrgPerson
uid: jduke
cn: Java Duke
sn: Duke
userPassword: Password
dn: uid=notUsedUser,ou=People,dc=jboss,dc=org
objectclass: top
objectclass: person
objectclass: inetOrgPerson
uid: notUsedUser
cn: not used user
sn: notUsedUser
userPassword: Password
dn: ou=Roles,dc=jboss,dc=org
objectclass: top
objectclass: organizationalUnit
ou: Roles
dn: cn=JBossAdmin,ou=Roles,dc=jboss,dc=org
objectclass: top
objectclass: groupOfNames
cn: JBossAdmin
member: uid=notUsedUser,ou=People,dc=jboss,dc=org
description: jduke
{code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months
[JBoss JIRA] (WFLY-7848) Elytron ldap-realm does not support principal to group mapping (memberOf)
by Jan Kalina (JIRA)
[ https://issues.jboss.org/browse/WFLY-7848?page=com.atlassian.jira.plugin.... ]
Jan Kalina moved ELY-859 to WFLY-7848:
--------------------------------------
Project: WildFly (was: WildFly Elytron)
Key: WFLY-7848 (was: ELY-859)
Component/s: Security
(was: Realms)
Affects Version/s: (was: 1.1.0.Beta16)
> Elytron ldap-realm does not support principal to group mapping (memberOf)
> -------------------------------------------------------------------------
>
> Key: WFLY-7848
> URL: https://issues.jboss.org/browse/WFLY-7848
> Project: WildFly
> Issue Type: Bug
> Components: Security
> Reporter: Jan Kalina
> Assignee: Jan Kalina
> Priority: Blocker
>
> Elytron ldap-realm is not able to work with LDAP which uses principal to group mapping. It seems that there is currently no way how to configure principal to group mapping in application server.
> Simplified example:
> {code}
> dn: uid=TestUserOne,ou=users,dc=principal-to-group,dc=example,dc=org
> objectClass: groupMember
> memberOf: uid=GroupOne,ou=groups,dc=principal-to-group,dc=example,dc=org
> memberOf: uid=Slashy/Group,ou=groups,dc=principal-to-group,dc=example,dc=org
> dn: uid=GroupOne,ou=groups,dc=principal-to-group,dc=example,dc=org
> objectClass: groupMember
> objectClass: group
> memberOf: uid=GroupFive,ou=subgroups,ou=groups,dc=principal-to-group,dc=example,dc=org
> {code}
> Example for reproducing: (by olukas)
> Role {{SomeRole}} is currently not able to be assigned to user {{someUser}} when following ldif is used. In this case principal to group mapping is provided by attribute {{description}}, but in can be provided by any attribute (e.g. memberOf). User {{thisUserIsNotUsed}} is used only for simpler reproduction of issue.
> {code}
> dn: ou=People,dc=jboss,dc=org
> objectclass: top
> objectclass: organizationalUnit
> ou: People
> dn: uid=someUser,ou=People,dc=jboss,dc=org
> objectclass: top
> objectclass: person
> objectclass: inetOrgPerson
> uid: someUser
> cn: some User
> sn: User
> userPassword: Password
> description: cn=SomeRole,ou=Roles,dc=jboss,dc=org
> dn: uid=thisUserIsNotUsed,ou=People,dc=jboss,dc=org
> objectclass: top
> objectclass: person
> objectclass: inetOrgPerson
> uid: thisUserIsNotUsed
> cn: this User Is Not Used
> sn: this User Is Not Used
> userPassword: Password
> dn: ou=Roles,dc=jboss,dc=org
> objectclass: top
> objectclass: organizationalUnit
> ou: Roles
> dn: cn=SomeRole,ou=Roles,dc=jboss,dc=org
> objectclass: top
> objectclass: groupOfNames
> cn: SomeRole
> member: uid=thisUserIsNotUsed,ou=People,dc=jboss,dc=org
> {code}
> Mentioned ldif works with legacy security solution.
> This missing feature can cause that migration from legacy security solution will not be possible -> we request blocker.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months
[JBoss JIRA] (ELY-859) Elytron ldap-realm does not support principal to group mapping (memberOf)
by Jan Kalina (JIRA)
Jan Kalina created ELY-859:
------------------------------
Summary: Elytron ldap-realm does not support principal to group mapping (memberOf)
Key: ELY-859
URL: https://issues.jboss.org/browse/ELY-859
Project: WildFly Elytron
Issue Type: Bug
Components: Realms
Affects Versions: 1.1.0.Beta16
Reporter: Jan Kalina
Assignee: Jan Kalina
Priority: Blocker
Elytron ldap-realm is not able to work with LDAP which uses principal to group mapping. It seems that there is currently no way how to configure principal to group mapping in application server.
Simplified example:
{code}
dn: uid=TestUserOne,ou=users,dc=principal-to-group,dc=example,dc=org
objectClass: groupMember
memberOf: uid=GroupOne,ou=groups,dc=principal-to-group,dc=example,dc=org
memberOf: uid=Slashy/Group,ou=groups,dc=principal-to-group,dc=example,dc=org
dn: uid=GroupOne,ou=groups,dc=principal-to-group,dc=example,dc=org
objectClass: groupMember
objectClass: group
memberOf: uid=GroupFive,ou=subgroups,ou=groups,dc=principal-to-group,dc=example,dc=org
{code}
Example for reproducing: (by olukas)
Role {{SomeRole}} is currently not able to be assigned to user {{someUser}} when following ldif is used. In this case principal to group mapping is provided by attribute {{description}}, but in can be provided by any attribute (e.g. memberOf). User {{thisUserIsNotUsed}} is used only for simpler reproduction of issue.
{code}
dn: ou=People,dc=jboss,dc=org
objectclass: top
objectclass: organizationalUnit
ou: People
dn: uid=someUser,ou=People,dc=jboss,dc=org
objectclass: top
objectclass: person
objectclass: inetOrgPerson
uid: someUser
cn: some User
sn: User
userPassword: Password
description: cn=SomeRole,ou=Roles,dc=jboss,dc=org
dn: uid=thisUserIsNotUsed,ou=People,dc=jboss,dc=org
objectclass: top
objectclass: person
objectclass: inetOrgPerson
uid: thisUserIsNotUsed
cn: this User Is Not Used
sn: this User Is Not Used
userPassword: Password
dn: ou=Roles,dc=jboss,dc=org
objectclass: top
objectclass: organizationalUnit
ou: Roles
dn: cn=SomeRole,ou=Roles,dc=jboss,dc=org
objectclass: top
objectclass: groupOfNames
cn: SomeRole
member: uid=thisUserIsNotUsed,ou=People,dc=jboss,dc=org
{code}
Mentioned ldif works with legacy security solution.
This missing feature can cause that migration from legacy security solution will not be possible -> we request blocker.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months
[JBoss JIRA] (JGRP-2150) More efficient message adding and draining
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2150?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-2150:
---------------------------
Fix Version/s: 4.0
> More efficient message adding and draining
> ------------------------------------------
>
> Key: JGRP-2150
> URL: https://issues.jboss.org/browse/JGRP-2150
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 4.0
>
>
> In NAKACK2, UNICAST3 and in MaxOneThreadPerSenderPolicy, we have a pattern where aone or more producers add messages (to a table in NAKACK2 and UNICAST3, or to a MessageBatch in MaxOneThreadPerSenderPolicy) and then only *a single thread* can remove and deliver messages up the stack.
> This requires synchronization around (1) determining the thread will be remove messages, (2) adding messages to the table (or batch) and (3) removing messages from the table or batch.
> Unit tests DrainTest and MessageBatchDrainTest show how a simple AtomicInteger can be used to do this.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months
[JBoss JIRA] (JGRP-2150) More efficient message adding and draining
by Bela Ban (JIRA)
Bela Ban created JGRP-2150:
------------------------------
Summary: More efficient message adding and draining
Key: JGRP-2150
URL: https://issues.jboss.org/browse/JGRP-2150
Project: JGroups
Issue Type: Enhancement
Reporter: Bela Ban
Assignee: Bela Ban
In NAKACK2, UNICAST3 and in MaxOneThreadPerSenderPolicy, we have a pattern where aone or more producers add messages (to a table in NAKACK2 and UNICAST3, or to a MessageBatch in MaxOneThreadPerSenderPolicy) and then only *a single thread* can remove and deliver messages up the stack.
This requires synchronization around (1) determining the thread will be remove messages, (2) adding messages to the table (or batch) and (3) removing messages from the table or batch.
Unit tests DrainTest and MessageBatchDrainTest show how a simple AtomicInteger can be used to do this.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months
[JBoss JIRA] (JGRP-2143) TP: use only one thread per member to pass up regular messages
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2143?page=com.atlassian.jira.plugin.... ]
Bela Ban reopened JGRP-2143:
----------------------------
Use a bounded MessageBatch in OneThreadPerSenderPolicy: a special add() never increases the capacity and only messages that fit into the batch are accepted, returns true (all messages were added) or false (all or some messages were discarded).
Discarding is ok as retransmission will resend them later.
> TP: use only one thread per member to pass up regular messages
> --------------------------------------------------------------
>
> Key: JGRP-2143
> URL: https://issues.jboss.org/browse/JGRP-2143
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 4.0
>
>
> This applies only to _regular_ messages; OOB and internal messages are processed by passing them to the thread pool directly when they they're received.
> The processing of a message received from B is as follows:
> * A regular message (or message batch) is assigned a thread from the thread pool and passed up to the reliability protocol, e.g. NAKACK2 or UNICAST3.
> * There is is added to the table for B.
> * The thread sees if another thread is already delivering messages from B to the application. If not, it grabs as many consecutive (ordered) messages from the table as it can and delivers them to the application. Otherwise, it returns and can be assigned other tasks.
> The problem here is that more than one thread may be passing up messages from a given sender B; only at the NAKACK2 or UNICAST3 level will a single thread be selected to deliver the messages to the application.
> This causes higher thread pool usage than required, with all of its drawbacks, e.g. more context switching, higher contention on adding messages to the table for B, and possibly exhaustion of the thread pool.
> An example of where service is denied or delayed:
> * We have a cluster of \{A,B,C,D\}
> * A receives 10 messages from B, 4 from C and 1 from D
> * The thread pool's max size is 20
> * The 10 messages from B are processed; all 10 threads add their messages to the table, but only 1 delivers them to the application and the other 9 return to the pool
> * 4 messages from C are added to C's table, 1 thread delivers them and 3 return
> * The 1 message from D is added to D's table and the same thread is used to deliver the message up the stack to the application
> So while we receive 15 messages, effectively only 3 threads are needed to deliver them to the application: as these are regular messages, they need to be delivered in _sender order_.
> The 9 threads which process messages from B are only adding them to B's table and then return immediately. This causes increased context switching, plus more contention on B's table (which is synchronized), and possibly exhaustion of the thread pool. For example, if the pool's max size was only 10, then processing the first 10 messages from B would exhaust the table, and the other messages from C and D would be processed in newly spawned threads.
> SOLUTION
> * (Only applicable to _regular_ messages)
> * When a message (or batch) from sender P is received, we check if another thread is already passing up messages from B. If not, we pass the message up by grabbing a thread from the thread pool. This will add the message to P's table and deliver as many messages (from from the table) as possible to the application.
> * If there's currently a thread delivering P's message, we simply add the message (or batch) to a queue for P and return.
> * When the delivery thread returns, it checks the queue for P and delivers all queued messages, or returns if the queue is empty.
> * (The queue is actually a MessageBatch, and new messages are simply appended to it. On delivery, the batch is cleared)
> The effects of this for regular messages are
> * Fewer threads: the thread pool only has a max of <cluster-members> threads for regular messages where <cluster-members> is the number of members in the cluster from whom we are concurrently receiving messages. E.g. for a cluster \{A,B,C,D\}, if we're receiving messages at the same time from all members, then the max size is 4.
> ** Of course, OOB and internal messages, plus timer tasks will add to this number.
> * Less contention on the table for a given member: instead of 10 threads all adding their messages to B's table (contention on the table lock) and then CASing a boolean, only 1 thread ever adds and removes messages to/from the table. This means uncontended (= fast) lock acquisition for regular messages (of course, if we use OOB messages, then we do have contention).
> * Appending to a batch is much faster then adding to a table
> * The downside is that we're storing messages actually twice: once in the batch for P and once in P's table. But these are arrays of pointers, so not a lot of memory required.
> Example: the 10 threads for messages from B above, will create a batch of 9 messages in B's queue and grab 1 thread from the pool to deliver its message. When the thread is done, it will grab the message batch of 9 and also add it to the table and deliver it.
> This is similar to the bulkhead pattern \[1\].
> \[1\] http://stackoverflow.com/questions/30391809/what-is-bulkhead-pattern-used...
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months
[JBoss JIRA] (WFLY-7841) ManagedScheduledExecutorService could not find EntityManager produced by CDI producer. No active context.
by Martin Kouba (JIRA)
[ https://issues.jboss.org/browse/WFLY-7841?page=com.atlassian.jira.plugin.... ]
Martin Kouba commented on WFLY-7841:
------------------------------------
Well, it makes sense. However, we should be careful here.
bq. Tasks that are submitted to a managed instance of ExecutorService may still be running after the lifecycle of the submitting component.
(2.3.2.1 Tasks and Contexts and Dependency Injection)
If you submit a task which is a client proxy of a {{@RequestScoped}} bean, then the real bean instance executed will be (most probably) a completely different object than the one a user might think he/she is passing. This might be really confusing.
So I think it's ok to activate request context and use {{@RequestScoped}} beans during task execution but the task itself should not be {{@RequestScoped}}.
> ManagedScheduledExecutorService could not find EntityManager produced by CDI producer. No active context.
> ---------------------------------------------------------------------------------------------------------
>
> Key: WFLY-7841
> URL: https://issues.jboss.org/browse/WFLY-7841
> Project: WildFly
> Issue Type: Bug
> Components: CDI / Weld
> Affects Versions: 10.0.0.Final, 10.1.0.Final
> Reporter: Tibor Digana
> Assignee: Stuart Douglas
>
> The problem is that _EntityManager_ does not have managed delegate within a job executed by _ManagedScheduledExecutorService#scheduleAtFixedRate()_.
> The _EntityManager_ is produced by CDI producer and entire web application is able to work with injected _EntityManager_ except for the jobs.
> I tried to schedule executing the job in two ways and both finished with same issue (No managed context in _EntityManager_) :
> * _@ApplicationScoped_ job instance was passed to executor from caller.
> * The job was changed to _RequestScoped_ and _ContextProxy#createContextualProxy()_ created the Job bean instance and then I called _scheduleAtFixedRate(job, 5, 60, SECONDS)_
> It looks to me that _javax.enterprise.concurrent_ has a different _BeanManager_ and therefore has no notion about my CDI Producer of _EntityManager_. The producer is regular producer of _EntityManager_ in JavaEE.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months
[JBoss JIRA] (WFLY-7841) ManagedScheduledExecutorService could not find EntityManager produced by CDI producer. No active context.
by Stuart Douglas (JIRA)
[ https://issues.jboss.org/browse/WFLY-7841?page=com.atlassian.jira.plugin.... ]
Stuart Douglas commented on WFLY-7841:
--------------------------------------
I think we should probably do it as a non standard extension.
> ManagedScheduledExecutorService could not find EntityManager produced by CDI producer. No active context.
> ---------------------------------------------------------------------------------------------------------
>
> Key: WFLY-7841
> URL: https://issues.jboss.org/browse/WFLY-7841
> Project: WildFly
> Issue Type: Bug
> Components: CDI / Weld
> Affects Versions: 10.0.0.Final, 10.1.0.Final
> Reporter: Tibor Digana
> Assignee: Stuart Douglas
>
> The problem is that _EntityManager_ does not have managed delegate within a job executed by _ManagedScheduledExecutorService#scheduleAtFixedRate()_.
> The _EntityManager_ is produced by CDI producer and entire web application is able to work with injected _EntityManager_ except for the jobs.
> I tried to schedule executing the job in two ways and both finished with same issue (No managed context in _EntityManager_) :
> * _@ApplicationScoped_ job instance was passed to executor from caller.
> * The job was changed to _RequestScoped_ and _ContextProxy#createContextualProxy()_ created the Job bean instance and then I called _scheduleAtFixedRate(job, 5, 60, SECONDS)_
> It looks to me that _javax.enterprise.concurrent_ has a different _BeanManager_ and therefore has no notion about my CDI Producer of _EntityManager_. The producer is regular producer of _EntityManager_ in JavaEE.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 3 months