[JBoss JIRA] (JGRP-2262) "Frozen" coordinator causes the whole cluster to hang
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2262?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2262:
--------------------------------
yes
> "Frozen" coordinator causes the whole cluster to hang
> -----------------------------------------------------
>
> Key: JGRP-2262
> URL: https://issues.jboss.org/browse/JGRP-2262
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.6.7
> Reporter: Pietro Paolini
> Assignee: Bela Ban
> Fix For: 4.0.12, 3.6.16
>
> Attachments: jdbc_test.xml, jgroup.zip
>
>
> This is the result of an investigation I carried out for a problem we have experienced within our
> application, the scenario it has been re-created by pausing the JVM using a debugger.
> The discovery mechanism is JDBC_PING.
> If the coordinator's JVM gets fronzen (for whatever reason) before the coordinator sets itself as the cluster coordinator and another node is started after that it will be unable to join the cluster and it will hang indefinitely.
> This seems to be caused by the "continue" statement at
> https://github.com/belaban/JGroups/blob/master/src/org/jgroups/protocols/...
> I have prepared a simple application which can help in replicating the problem.
> To replicate the problem :
> 1) Make sure the JGROUPSPING is empty
> 2) Run the application using an IDE and attaching a debugger to cause the JVM to
> be paused at line Main.java:67, wait for it.
> 3) Run the application in non debug mode or with gradle using "gradle run" and it will
> hang indefinitely
> Depending on the UUID/IP Address being used generated/assigned this may not happen all the time but it happened quite often in my local tests.
>
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years
[JBoss JIRA] (JGRP-2281) MERGE3 blocks unnecessarily in discovery when non-multicast transport is used
by Bela Ban (JIRA)
Bela Ban created JGRP-2281:
------------------------------
Summary: MERGE3 blocks unnecessarily in discovery when non-multicast transport is used
Key: JGRP-2281
URL: https://issues.jboss.org/browse/JGRP-2281
Project: JGroups
Issue Type: Enhancement
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 4.0.13
When MERGE3 uses TCP, it cannot multicast its INFO message, and therefore uses the discovery protocol (e.g. MPING) to fetch the targets to send the INFO message to.
Since we don't know how many responses to expect, we simply block for {{(min_interval + max_interval /2) ms}}. This is bad, as it delays the sending of INFO messages, which results in a partial merge as we're likely no to get responses from *all* members. This delays a full merge, e.g. when we have many singleton subclusters. A heavily split cluster will therefore likely require more merge rounds than necessary when using TCP, compared to (e.g.) UDP.
h4. Solution:
* The discovery process should be _reactive_ rather than blocking: instead of waiting for N seconds, we simply pass a function to the discovery protocol that gets invoked whenever a response has been received
* When the function gets invoked, it sends an INFO to the respective member
* This prevents up 1 thread from blocking for N seconds
See \[1\] for details.
\[1\] https://github.com/belaban/JGroups/pull/389
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years
[JBoss JIRA] (JGRP-2262) "Frozen" coordinator causes the whole cluster to hang
by Sibin Karnavar (JIRA)
[ https://issues.jboss.org/browse/JGRP-2262?page=com.atlassian.jira.plugin.... ]
Sibin Karnavar commented on JGRP-2262:
--------------------------------------
Did you release 4.0.12 already?
> "Frozen" coordinator causes the whole cluster to hang
> -----------------------------------------------------
>
> Key: JGRP-2262
> URL: https://issues.jboss.org/browse/JGRP-2262
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.6.7
> Reporter: Pietro Paolini
> Assignee: Bela Ban
> Fix For: 4.0.12, 3.6.16
>
> Attachments: jdbc_test.xml, jgroup.zip
>
>
> This is the result of an investigation I carried out for a problem we have experienced within our
> application, the scenario it has been re-created by pausing the JVM using a debugger.
> The discovery mechanism is JDBC_PING.
> If the coordinator's JVM gets fronzen (for whatever reason) before the coordinator sets itself as the cluster coordinator and another node is started after that it will be unable to join the cluster and it will hang indefinitely.
> This seems to be caused by the "continue" statement at
> https://github.com/belaban/JGroups/blob/master/src/org/jgroups/protocols/...
> I have prepared a simple application which can help in replicating the problem.
> To replicate the problem :
> 1) Make sure the JGROUPSPING is empty
> 2) Run the application using an IDE and attaching a debugger to cause the JVM to
> be paused at line Main.java:67, wait for it.
> 3) Run the application in non debug mode or with gradle using "gradle run" and it will
> hang indefinitely
> Depending on the UUID/IP Address being used generated/assigned this may not happen all the time but it happened quite often in my local tests.
>
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years
[JBoss JIRA] (WFLY-10637) NPE when reading oracle jdbc 12.2.0.1 datasource with include-runtime
by Claudio Miranda (JIRA)
[ https://issues.jboss.org/browse/WFLY-10637?page=com.atlassian.jira.plugin... ]
Claudio Miranda resolved WFLY-10637.
------------------------------------
Resolution: Rejected
> NPE when reading oracle jdbc 12.2.0.1 datasource with include-runtime
> ---------------------------------------------------------------------
>
> Key: WFLY-10637
> URL: https://issues.jboss.org/browse/WFLY-10637
> Project: WildFly
> Issue Type: Bug
> Components: JCA
> Reporter: Claudio Miranda
> Assignee: Lin Gao
>
> Download oracle jdbc driver at http://www.oracle.com/technetwork/database/features/jdbc/jdbc-ucp-122-311...
> Install the driver as a module
> {code}
> $ cat modules/system/layers/base/com/oracle/jdbc/main/module.xml
> <?xml version="1.0" encoding="UTF-8"?>
> <module name="com.oracle.jdbc" xmlns="urn:jboss:module:1.5">
> <resources>
> <resource-root path="ojdbc8.jar"/>
> </resources>
> <dependencies>
> <module name="javax.api"/>
> <module name="javax.transaction.api"/>
> </dependencies>
> </module>
> {code}
> Add the jdbc-driver resource
> {code}
> /profile=full-ha/subsystem=datasources/jdbc-driver=oracle_8_xa:add(driver-module-name=com.oracle.jdbc,driver-name=oracle_8_xa,driver-datasource-class-name=oracle.jdbc.datasource.OracleDataSource,driver-xa-datasource-class-name=oracle.jdbc.xa.client.OracleXADataSource,driver-class-name=oracle.jdbc.OracleDriver)
> {code}
> Wildfly log shows it is ok
> {code}
> [Server:server-three] 13:44:28,939 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 73) WFLYJCA0004: Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 12.2)
> [Server:server-three] 13:44:28,939 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-6) WFLYJCA0018: Started Driver service with driver-name = oracle_8_xa
> {code}
> Read the jdbc-driver on a runtime server
> {code}
> /host=master/server=server-three/subsystem=datasources/jdbc-driver=oracle_8_xa:read-resource(include-runtime)
> {
> "outcome" => "success",
> "result" => undefined,
> "failure-description" => undefined
> {code}
> Wildfly log shows the error
> {code}
> 2018-06-25 13:45:00,165 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 73) WFLYCTL0013: Operation ("read-attribute") failed - address: ([
> ("subsystem" => "datasources"),
> ("jdbc-driver" => "oracle_8_xa")
> ]): java.lang.NullPointerException
> at org.jboss.as.connector.subsystems.datasources.GetDataSourceClassInfoOperationHandler.possiblePropsSetters(GetDataSourceClassInfoOperationHandler.java:182)
> at org.jboss.as.connector.subsystems.datasources.GetDataSourceClassInfoOperationHandler.findPropsFromCls(GetDataSourceClassInfoOperationHandler.java:124)
> at org.jboss.as.connector.subsystems.datasources.GetDataSourceClassInfoOperationHandler.dsClsInfoNode(GetDataSourceClassInfoOperationHandler.java:96)
> at org.jboss.as.connector.subsystems.datasources.GetDataSourceClassInfoOperationHandler$1.execute(GetDataSourceClassInfoOperationHandler.java:82)
> at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:999)
> at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:743)
> at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:467)
> at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1411)
> at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:423)
> at org.jboss.as.controller.ModelControllerImpl.lambda$execute$1(ModelControllerImpl.java:243)
> at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:263)
> at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:229)
> at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:243)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler.internalExecute(TransactionalProtocolOperationHandler.java:249)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler.doExecute(TransactionalProtocolOperationHandler.java:190)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:143)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:139)
> at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:263)
> at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:229)
> at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:198)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:162)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:158)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2.execute(TransactionalProtocolOperationHandler.java:158)
> at org.jboss.as.protocol.mgmt.ManagementRequestContextImpl$1.doExecute(ManagementRequestContextImpl.java:70)
> at org.jboss.as.protocol.mgmt.ManagementRequestContextImpl$AsyncTaskRunner.run(ManagementRequestContextImpl.java:160)
> at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> {code}
> It works if omit include-runtime, but there is no runtime information
> The following drivers works with include-runtime
> - postgresql-42.2.2.jre6.jar
> - ojdbc7.jar
> - mssql-jdbc-6.5.3.jre8-preview.jar
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years
[JBoss JIRA] (WFLY-10637) NPE when reading oracle jdbc 12.2.0.1 datasource with include-runtime
by Claudio Miranda (JIRA)
[ https://issues.jboss.org/browse/WFLY-10637?page=com.atlassian.jira.plugin... ]
Claudio Miranda commented on WFLY-10637:
----------------------------------------
Sorry for the false alarm, I should have noticed that as I did it correctly for the ojdbc7.jar. I will close as not a bug.
> NPE when reading oracle jdbc 12.2.0.1 datasource with include-runtime
> ---------------------------------------------------------------------
>
> Key: WFLY-10637
> URL: https://issues.jboss.org/browse/WFLY-10637
> Project: WildFly
> Issue Type: Bug
> Components: JCA
> Reporter: Claudio Miranda
> Assignee: Lin Gao
>
> Download oracle jdbc driver at http://www.oracle.com/technetwork/database/features/jdbc/jdbc-ucp-122-311...
> Install the driver as a module
> {code}
> $ cat modules/system/layers/base/com/oracle/jdbc/main/module.xml
> <?xml version="1.0" encoding="UTF-8"?>
> <module name="com.oracle.jdbc" xmlns="urn:jboss:module:1.5">
> <resources>
> <resource-root path="ojdbc8.jar"/>
> </resources>
> <dependencies>
> <module name="javax.api"/>
> <module name="javax.transaction.api"/>
> </dependencies>
> </module>
> {code}
> Add the jdbc-driver resource
> {code}
> /profile=full-ha/subsystem=datasources/jdbc-driver=oracle_8_xa:add(driver-module-name=com.oracle.jdbc,driver-name=oracle_8_xa,driver-datasource-class-name=oracle.jdbc.datasource.OracleDataSource,driver-xa-datasource-class-name=oracle.jdbc.xa.client.OracleXADataSource,driver-class-name=oracle.jdbc.OracleDriver)
> {code}
> Wildfly log shows it is ok
> {code}
> [Server:server-three] 13:44:28,939 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 73) WFLYJCA0004: Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 12.2)
> [Server:server-three] 13:44:28,939 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-6) WFLYJCA0018: Started Driver service with driver-name = oracle_8_xa
> {code}
> Read the jdbc-driver on a runtime server
> {code}
> /host=master/server=server-three/subsystem=datasources/jdbc-driver=oracle_8_xa:read-resource(include-runtime)
> {
> "outcome" => "success",
> "result" => undefined,
> "failure-description" => undefined
> {code}
> Wildfly log shows the error
> {code}
> 2018-06-25 13:45:00,165 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 73) WFLYCTL0013: Operation ("read-attribute") failed - address: ([
> ("subsystem" => "datasources"),
> ("jdbc-driver" => "oracle_8_xa")
> ]): java.lang.NullPointerException
> at org.jboss.as.connector.subsystems.datasources.GetDataSourceClassInfoOperationHandler.possiblePropsSetters(GetDataSourceClassInfoOperationHandler.java:182)
> at org.jboss.as.connector.subsystems.datasources.GetDataSourceClassInfoOperationHandler.findPropsFromCls(GetDataSourceClassInfoOperationHandler.java:124)
> at org.jboss.as.connector.subsystems.datasources.GetDataSourceClassInfoOperationHandler.dsClsInfoNode(GetDataSourceClassInfoOperationHandler.java:96)
> at org.jboss.as.connector.subsystems.datasources.GetDataSourceClassInfoOperationHandler$1.execute(GetDataSourceClassInfoOperationHandler.java:82)
> at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:999)
> at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:743)
> at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:467)
> at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1411)
> at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:423)
> at org.jboss.as.controller.ModelControllerImpl.lambda$execute$1(ModelControllerImpl.java:243)
> at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:263)
> at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:229)
> at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:243)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler.internalExecute(TransactionalProtocolOperationHandler.java:249)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler.doExecute(TransactionalProtocolOperationHandler.java:190)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:143)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:139)
> at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:263)
> at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:229)
> at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:198)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:162)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:158)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2.execute(TransactionalProtocolOperationHandler.java:158)
> at org.jboss.as.protocol.mgmt.ManagementRequestContextImpl$1.doExecute(ManagementRequestContextImpl.java:70)
> at org.jboss.as.protocol.mgmt.ManagementRequestContextImpl$AsyncTaskRunner.run(ManagementRequestContextImpl.java:160)
> at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> {code}
> It works if omit include-runtime, but there is no runtime information
> The following drivers works with include-runtime
> - postgresql-42.2.2.jre6.jar
> - ojdbc7.jar
> - mssql-jdbc-6.5.3.jre8-preview.jar
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years