[JBoss JIRA] (WFLY-10067) [Artemis 2.x Upgrade] Failover does not work with HTTP conncetors/acceptors
by Erich Duda (JIRA)
[ https://issues.jboss.org/browse/WFLY-10067?page=com.atlassian.jira.plugin... ]
Erich Duda updated WFLY-10067:
------------------------------
Summary: [Artemis 2.x Upgrade] Failover does not work with HTTP conncetors/acceptors (was: Failover does not work with HTTP conncetors/acceptors)
> [Artemis 2.x Upgrade] Failover does not work with HTTP conncetors/acceptors
> ---------------------------------------------------------------------------
>
> Key: WFLY-10067
> URL: https://issues.jboss.org/browse/WFLY-10067
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Affects Versions: 13.0.0.Beta1
> Reporter: Erich Duda
> Assignee: Jeff Mesnil
> Priority: Blocker
> Labels: feature-branch-blocker
> Attachments: clients.log, server1.log, server2.log, test.log
>
>
> Failover does not work with HTTP connectors/acceptors.
> *Scenario:*
> # There are two Wildfly servers configured as Live-Backup pair
> # There is one JMS producer and one JMS receiver which sends/receives messages
> # Live server is several times killed and restarted.
> *Expectation:* Always when the Live server is killed or restarted, clients do failover or failback.
> *Reality*: Sometimes happens that clients don't do failover.
> *Users impact:* One of basics feature of HA, failover, does not work properly.
> *Blocker* priority was set because this is regression against previous releases of Wildfly.
> *Detail description of issue*
> In the trace logs it can be seen that clients send HTTP handshake request to active backup, but the handshake fails. All checks (and logs) say that the backup is active in this time period. I tried to run the test with Netty connectors/acceptors and I didn't see this issue.
> {code:title=Backup log}
> 07:52:47,856 WARN [org.apache.activemq.artemis.core.server] (Thread-2 (ActiveMQ-IO-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@1f01f347)) AMQ222029: Could not locate page transaction 3 995, ignoring message on position PagePositionImpl [pageNr=2, messageNr=55, recordID=-1] on address=jms.queue.testQueue queue=jms.queue.testQueue
> 07:52:47,856 WARN [org.apache.activemq.artemis.core.server] (Thread-2 (ActiveMQ-IO-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@1f01f347)) AMQ222029: Could not locate page transaction 3 995, ignoring message on position PagePositionImpl [pageNr=2, messageNr=56, recordID=-1] on address=jms.queue.testQueue queue=jms.queue.testQueue
> 07:52:47,863 INFO [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=null) AMQ221005: Deleting pending large message as it was not completed: Pair[a=2147485068, b=2147485067]
> 07:52:47,863 INFO [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=null) AMQ221005: Deleting pending large message as it was not completed: Pair[a=2147485076, b=2147485075]
> 07:52:47,864 INFO [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=null) AMQ221005: Deleting pending large message as it was not completed: Pair[a=2147484450, b=2147484449]
> 07:52:47,864 INFO [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=null) AMQ221005: Deleting pending large message as it was not completed: Pair[a=2147485072, b=2147485071]
> 07:52:47,864 INFO [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=null) AMQ221005: Deleting pending large message as it was not completed: Pair[a=2147484454, b=2147484453]
> 07:52:47,866 WARN [org.apache.activemq.artemis.core.client] (activemq-discovery-group-thread-dg-group1) AMQ212034: There are more than one servers on the network broadcasting the same node id. You will see this message exactly once (per node) if a node is restarted, in which case it can be safely ignored. But if it is logged continuously it means you really do have more than one node on the same network active concurrently with the same node id. This could occur if you have a backup node active at the same time as its live node. nodeID=0dee81c9-2d9d-11e8-ba3f-cc3d825f79a4
> 07:52:47,867 INFO [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=null) AMQ221007: Server is now live
> {code}
> {code:title=Clients log}
> 07:52:49,274 Thread-80 (ActiveMQ-client-global-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:726] Remote destination: localhost/127.0.0.1:9080
> 07:52:49,274 Thread-79 (ActiveMQ-client-global-threads) DEBUG [org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl:1040] Connector towards NettyConnector [host=localhost, port=8080, httpEnabled=false, httpUpgradeEnabled=true, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true, activemqServerName=default, httpUpgradeEndpoint=acceptor] failed
> 07:52:49,275 Thread-79 (ActiveMQ-client-global-threads) DEBUG [org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl:1081] Trying backup config = TransportConfiguration(name=connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?httpUpgradeEndpoint=acceptor&activemqServerName=default&httpUpgradeEnabled=true&port=9080&host=localhost
> 07:52:49,275 Thread-79 (ActiveMQ-client-global-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:447] Connector NettyConnector [host=localhost, port=9080, httpEnabled=false, httpUpgradeEnabled=true, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true, activemqServerName=default, httpUpgradeEndpoint=acceptor] using native epoll
> 07:52:49,275 Thread-80 (ActiveMQ-client-global-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:797] Sending HTTP request DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
> GET HTTP/1.1
> host: localhost
> upgrade: activemq-remoting
> connection: upgrade
> activemqServerName: default
> httpUpgradeEndpoint: acceptor
> Sec-ActiveMQRemoting-Key: QWV3bCfgh75NjWH3pZV5Ew==
> 07:52:49,275 Thread-79 (ActiveMQ-client-global-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:601] AMQ211002: Started EPOLL Netty Connector version 4.1.16.Final to localhost:9080
> 07:52:49,275 Thread-79 (ActiveMQ-client-global-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:726] Remote destination: localhost/127.0.0.1:9080
> 07:52:49,276 Thread-79 (ActiveMQ-client-global-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector:797] Sending HTTP request DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
> GET HTTP/1.1
> host: localhost
> upgrade: activemq-remoting
> connection: upgrade
> activemqServerName: default
> httpUpgradeEndpoint: acceptor
> Sec-ActiveMQRemoting-Key: P9xBwRk1eZP5QjDWjqYuIg==
> 07:52:49,276 Thread-2 (ActiveMQ-client-netty-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector$HttpUpgradeHandler:876] Received msg=DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
> HTTP/1.1 200 OK
> Connection: keep-alive
> Last-Modified: Thu, 22 Mar 2018 06:47:03 GMT
> Content-Length: 2426
> Content-Type: text/html
> Accept-Ranges: bytes
> Date: Thu, 22 Mar 2018 06:52:49 GMT
> 07:52:49,276 Thread-2 (ActiveMQ-client-netty-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector$HttpUpgradeHandler:903] AMQ214023: HTTP Handshake failed, received DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
> HTTP/1.1 200 OK
> Connection: keep-alive
> Last-Modified: Thu, 22 Mar 2018 06:47:03 GMT
> Content-Length: 2426
> Content-Type: text/html
> Accept-Ranges: bytes
> Date: Thu, 22 Mar 2018 06:52:49 GMT
> 07:52:49,276 Thread-2 (ActiveMQ-client-netty-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector$HttpUpgradeHandler:876] Received msg=DefaultHttpContent(data: PooledSlicedByteBuf(ridx: 0, widx: 829, cap: 829/829, unwrapped: PooledUnsafeDirectByteBuf(ridx: 1024, widx: 1024, cap: 1024)), decoderResult: success)
> 07:52:49,276 Thread-2 (ActiveMQ-client-netty-threads) DEBUG [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector$HttpUpgradeHandler:903] AMQ214023: HTTP Handshake failed, received DefaultHttpContent(data: PooledSlicedByteBuf(ridx: 0, widx: 829, cap: 829/829, unwrapped: PooledUnsafeDirectByteBuf(ridx: 1024, widx: 1024, cap: 1024)), decoderResult: success)
> 07:52:49,277 Thread-80 (ActiveMQ-client-global-threads) DEBUG [org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl:1040] Connector towards NettyConnector [host=localhost, port=9080, httpEnabled=false, httpUpgradeEnabled=true, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true, activemqServerName=default, httpUpgradeEndpoint=acceptor] failed
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 1 month
[JBoss JIRA] (WFLY-10078) [Artemis 2.x Upgrade] Replicated HA: Live and Backup do not form cluster and initial synchronization is not triggered
by Erich Duda (JIRA)
Erich Duda created WFLY-10078:
---------------------------------
Summary: [Artemis 2.x Upgrade] Replicated HA: Live and Backup do not form cluster and initial synchronization is not triggered
Key: WFLY-10078
URL: https://issues.jboss.org/browse/WFLY-10078
Project: WildFly
Issue Type: Bug
Components: JMS
Affects Versions: 13.0.0.Beta1
Reporter: Erich Duda
Assignee: Jeff Mesnil
Priority: Blocker
Attachments: server1-trace.log, server1.log, server2-trace.log, server2.log
Across all replicated HA tests I often see an issue that initial synchronization between Live and Backup was not triggered.
*Scenario*
# There are two Wildfly servers configured as replicated Live-Backup pair.
# Live server is killed/shutdown
*Expectation:* Backup server becomes active.
*Reality:* Backup server does not become active, because initial synchronization with Live server was not triggered.
*Users impact:* Replicated HA feature doesn't work properly.
*Blocker* priority was set because this is regression against previous Wildfly releases.
*Detail description of the issue*
In the trace log there is last message from the {{SharedNothingBackupActivation}} which says that it is waiting on cluster connection. It is interested that JGroups subsystem is booting after this message was logged. However I am not sure if this could cause the issue.
{code}
13:42:06,489 DEBUG [org.apache.activemq.artemis.core.client.impl.Topology] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=null) Topology@6a834ff0[owner=ServerLocatorImpl [initialConnectors=[],
discoveryGroupConfiguration=DiscoveryGroupConfiguration{name='dg-group1', refreshTimeout=10000, discoveryInitialWaitTimeout=10000}]] is sending topology to org.apache.activemq.artemis.core.server.impl.NamedLiveN
odeLocatorForReplication@38150570
13:42:06,489 TRACE [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=null) Waiting on cluster connection
13:42:06,502 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000078: Starting JGroups channel ejb
13:42:06,503 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-7) ISPN000078: Starting JGroups channel ejb
13:42:06,504 INFO [org.infinispan.CLUSTER] (MSC service thread 1-6) ISPN000094: Received new cluster view for channel ejb: [node-1|1] (2) [node-1, node-2]
13:42:06,506 INFO [org.infinispan.CLUSTER] (MSC service thread 1-7) ISPN000094: Received new cluster view for channel ejb: [node-1|1] (2) [node-1, node-2]
13:42:06,506 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000078: Starting JGroups channel ejb
13:42:06,506 INFO [org.infinispan.CLUSTER] (MSC service thread 1-8) ISPN000094: Received new cluster view for channel ejb: [node-1|1] (2) [node-1, node-2]
13:42:06,510 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000078: Starting JGroups channel ejb
13:42:06,510 INFO [org.infinispan.CLUSTER] (MSC service thread 1-3) ISPN000094: Received new cluster view for channel ejb: [node-1|1] (2) [node-1, node-2]
13:42:06,516 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000079: Channel ejb local address is node-2, physical addresses are [127.0.0.1:56200]
13:42:06,527 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000079: Channel ejb local address is node-2, physical addresses are [127.0.0.1:56200]
13:42:06,527 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-7) ISPN000079: Channel ejb local address is node-2, physical addresses are [127.0.0.1:56200]
13:42:06,529 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000079: Channel ejb local address is node-2, physical addresses are [127.0.0.1:56200]
13:42:06,551 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-7) ISPN000128: Infinispan version: Infinispan 'Gaina' 9.2.0.Final
{code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 1 month
[JBoss JIRA] (WFLY-10077) [Artemis 2.x upgrade] journal-node-manager-store-table attribute does not preserve its value
by Miroslav Novak (JIRA)
Miroslav Novak created WFLY-10077:
-------------------------------------
Summary: [Artemis 2.x upgrade] journal-node-manager-store-table attribute does not preserve its value
Key: WFLY-10077
URL: https://issues.jboss.org/browse/WFLY-10077
Project: WildFly
Issue Type: Bug
Components: JMS
Reporter: Miroslav Novak
Assignee: Jeff Mesnil
Priority: Blocker
Writing to {{journal-node-manager-store-table}} attribute does not preserver its value and after restart default value is set again.
Output from CLI:
{code}
[standalone@localhost:9990 /] /subsystem=messaging-activemq/server=default:read-attribute(name=journal-node-manager-store-table)
{
"outcome" => "success",
"result" => "NODE_MANAGER_STORE_NODE_1",
"response-headers" => {"process-state" => "reload-required"}
}
...restart server here
[standalone@localhost:9990 /] /subsystem=messaging-activemq/server=default:read-attribute(name=journal-node-manager-store-table)
{
"outcome" => "success",
"result" => "NODE_MANAGER_STORE"
}
{code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 1 month
[JBoss JIRA] (WFLY-10076) Add a more verbose description on the IllegalStateException fix on WFLY-9821
by Martin Kouba (JIRA)
[ https://issues.jboss.org/browse/WFLY-10076?page=com.atlassian.jira.plugin... ]
Martin Kouba updated WFLY-10076:
--------------------------------
Component/s: CDI / Weld
> Add a more verbose description on the IllegalStateException fix on WFLY-9821
> ----------------------------------------------------------------------------
>
> Key: WFLY-10076
> URL: https://issues.jboss.org/browse/WFLY-10076
> Project: WildFly
> Issue Type: Bug
> Components: CDI / Weld
> Reporter: Viral Gohel
>
> Parent issue: https://issues.jboss.org/browse/WFLY-9821
> We throw an IlegalStateException, when an attempt is made to call the CDI container before the container is actually started with the use of CDI.current().
> Although the fix clearly states its not the correct way to set the state, we throw an Exception. The deployments succeed, but a non-expert CDI user might not expect an exception, and probably it would leave them confused on what is wrong, unless they understand what they have done in their own application code.
> My thought and wish was, if we can add some log.WARN message in the condition describing why this happens, would be nice! It provides you with hints on what is NOT to be done in your application code.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 1 month
[JBoss JIRA] (SWSQE-77) Automation[Java] - "Istio Mixer Detail" Page
by Jeeva Kandasamy (JIRA)
Jeeva Kandasamy created SWSQE-77:
------------------------------------
Summary: Automation[Java] - "Istio Mixer Detail" Page
Key: SWSQE-77
URL: https://issues.jboss.org/browse/SWSQE-77
Project: Swift Sunshine QE
Issue Type: Task
Reporter: Jeeva Kandasamy
Assignee: Jeeva Kandasamy
Automate "Istio Mixer detail" page features,
* Validate rule name
* Validate "match"
* Validate Action
* handler
* instance
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 1 month
[JBoss JIRA] (WFLY-10076) Add a more verbose description on the IllegalStateException fix on WFLY-9821
by Viral Gohel (JIRA)
[ https://issues.jboss.org/browse/WFLY-10076?page=com.atlassian.jira.plugin... ]
Viral Gohel reassigned WFLY-10076:
----------------------------------
Assignee: (was: Jason Greene)
> Add a more verbose description on the IllegalStateException fix on WFLY-9821
> ----------------------------------------------------------------------------
>
> Key: WFLY-10076
> URL: https://issues.jboss.org/browse/WFLY-10076
> Project: WildFly
> Issue Type: Bug
> Reporter: Viral Gohel
>
> Parent issue: https://issues.jboss.org/browse/WFLY-9821
> We throw an IlegalStateException, when an attempt is made to call the CDI container before the container is actually started with the use of CDI.current().
> Although the fix clearly states its not the correct way to set the state, we throw an Exception. The deployments succeed, but a non-expert CDI user might not expect an exception, and probably it would leave them confused on what is wrong, unless they understand what they have done in their own application code.
> My thought and wish was, if we can add some log.WARN message in the condition describing why this happens, would be nice! It provides you with hints on what is NOT to be done in your application code.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 1 month