[JBoss JIRA] (ISPN-7517) Investigate Netty's "Too many open files" exception in testing of infinispan-compatibility-mode-it
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-7517?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-7517:
------------------------------------
Looks like there is no leak after all: I added a 5 seconds delay before logging the leaked threads in {{TestNGTestListener.onFinish(org.testng.ISuite)}}, and now it only reports a few {{commonPool}} worker threads, no HotRod/Memcached worker threads.
The problem seems to be in {{AbstractProtocolServer.stop()}}, which calls {{EventLoopGroup.shutdownGracefully()}}, but then ignores the termination future.
Of course, the {{compatibility-mode-it}} could be further improved by starting less servers, either by using a single set of servers for the whole test suite, or by starting only the server that the test is actually using. The servers could also be configured to start a lot less threads than the production defaults.
> Investigate Netty's "Too many open files" exception in testing of infinispan-compatibility-mode-it
> --------------------------------------------------------------------------------------------------
>
> Key: ISPN-7517
> URL: https://issues.jboss.org/browse/ISPN-7517
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Server
> Reporter: Sanne Grinovero
> Fix For: 9.0.0.CR3
>
>
> See also discussion on ML:
> - http://lists.jboss.org/pipermail/infinispan-dev/2017-February/017232.html
> Running:
> {noformat}
> mci -rf :infinispan-compatibility-mode-it
> {noformat}
> {noformat}
> Failed tests:
> ByteArrayKeyDistEmbeddedHotRodTest.setup:27 » IllegalState failed to create a ...
> ByteArrayKeyReplEmbeddedHotRodTest.setup:87 » IllegalState failed to create a ...
> ByteArrayValueDistEmbeddedHotRodTest.setup:27 » IllegalState failed to create ...
> ByteArrayValueReplEmbeddedHotRodTest.setup:87 » IllegalState failed to create ...
> DistEmbeddedRestHotRodTest.setup:25 » IllegalState failed to create a child ev...
> DistL1EmbeddedHotRodTest.setup:30 » IllegalState failed to create a child even...
> DistMemcachedEmbeddedTest.setup:39 » IllegalState failed to create a child eve...
> org.infinispan.it.compatibility.EmbeddedHotRodCacheListenerTest.setup(org.infinispan.it.compatibility.EmbeddedHotRodCacheListenerTest)
> Run 1: EmbeddedHotRodCacheListenerTest.setup:36 » IllegalState failed to create a chi...
> Run 2: PASS
> Run 3: PASS
> EmbeddedHotRodTest.testEventFilteringStatic:274->lambda$testEventFilteringStatic$4:276 » Transport
> EmbeddedMemcachedCacheListenerTest.setup:39 » IllegalState failed to create a ...
> EmbeddedRestMemcachedHotRodTest.setup:50 » IllegalState failed to create a chi...
> ReplEmbeddedRestHotRodTest.setup:38 » IllegalState failed to create a child ev...
> Tests run: 101, Failures: 12, Errors: 0, Skipped: 47
> {noformat}
> Errors are all about a similar stacktrace.
> {noformat}
> java.lang.IllegalStateException: failed to create a child event loop
> at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:88)
> at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:58)
> at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:51)
> at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:87)
> at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:82)
> at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:63)
> at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:51)
> at org.jboss.resteasy.plugins.server.netty.NettyJaxrsServer.start(NettyJaxrsServer.java:239)
> at org.infinispan.rest.NettyRestServer.start(NettyRestServer.java:81)
> at org.infinispan.it.compatibility.CompatibilityCacheFactory.createRestCache(CompatibilityCacheFactory.java:199)
> at org.infinispan.it.compatibility.CompatibilityCacheFactory.createRestMemcachedCaches(CompatibilityCacheFactory.java:137)
> at org.infinispan.it.compatibility.CompatibilityCacheFactory.setup(CompatibilityCacheFactory.java:123)
> at org.infinispan.it.compatibility.DistMemcachedEmbeddedTest.setup(DistMemcachedEmbeddedTest.java:39)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
> at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:564)
> at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:213)
> at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:138)
> at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:175)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:107)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: io.netty.channel.ChannelException: failed to open a new selector
> at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:157)
> at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:148)
> at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:126)
> at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:36)
> at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:84)
> ... 32 more
> Caused by: java.io.IOException: Too many open files
> at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method)
> at sun.nio.ch.EPollArrayWrapper.<init>(EPollArrayWrapper.java:130)
> at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:69)
> at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
> at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:155)
> ... 36 more
> {noformat}
> N.B.
> The failure is quite consistent on my workstation, almost always results in the same tests failing. Yet occasionally they all pass.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (ISPN-7532) Configuration validation should only log warnings for concrete caches
by Tristan Tarrant (JIRA)
Tristan Tarrant created ISPN-7532:
-------------------------------------
Summary: Configuration validation should only log warnings for concrete caches
Key: ISPN-7532
URL: https://issues.jboss.org/browse/ISPN-7532
Project: Infinispan
Issue Type: Bug
Components: Configuration
Affects Versions: 9.0.0.CR2
Reporter: Tristan Tarrant
Assignee: Tristan Tarrant
Fix For: 9.0.0.Final
When defining a template it is sometimes necessary to define partial configurations. We should log warnings only when instantiating a configuration for a concrete cache.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (ISPN-7526) Write skew check throws even if the previous value was not read
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-7526?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-7526:
------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/4905
> Write skew check throws even if the previous value was not read
> ---------------------------------------------------------------
>
> Key: ISPN-7526
> URL: https://issues.jboss.org/browse/ISPN-7526
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Transactions
> Affects Versions: 9.0.0.CR2, 8.2.6.Final
> Reporter: Radim Vansa
> Assignee: Radim Vansa
>
> When the entry was overwritten without reading previous value and then we read it (this read is handled from the current context), write skew check is still applied and can fail, despite that it's unnecessary.
> {code:java}
> cache.put("k", "init");
> tm.begin();
> cache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).put("k", "v1");
> assertEquals("v1", cache.put("k", "v2"));
> Transaction tx = tm.suspend();
> assertEquals("init", cache.put("k", "other"));
> tm.resume(tx);
> tm.commit(); // fails with WriteSkewCheckException
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (ISPN-7526) Write skew check throws even if the previous value was not read
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-7526?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-7526:
------------------------------
Status: Open (was: New)
> Write skew check throws even if the previous value was not read
> ---------------------------------------------------------------
>
> Key: ISPN-7526
> URL: https://issues.jboss.org/browse/ISPN-7526
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Transactions
> Affects Versions: 9.0.0.CR2, 8.2.6.Final
> Reporter: Radim Vansa
> Assignee: Radim Vansa
>
> When the entry was overwritten without reading previous value and then we read it (this read is handled from the current context), write skew check is still applied and can fail, despite that it's unnecessary.
> {code:java}
> cache.put("k", "init");
> tm.begin();
> cache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).put("k", "v1");
> assertEquals("v1", cache.put("k", "v2"));
> Transaction tx = tm.suspend();
> assertEquals("init", cache.put("k", "other"));
> tm.resume(tx);
> tm.commit(); // fails with WriteSkewCheckException
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (ISPN-7505) Non-functional read in transaction with modifications returns old value
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-7505?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-7505:
------------------------------
Status: Open (was: New)
> Non-functional read in transaction with modifications returns old value
> -----------------------------------------------------------------------
>
> Key: ISPN-7505
> URL: https://issues.jboss.org/browse/ISPN-7505
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.0.0.CR1
> Reporter: Radim Vansa
> Assignee: Radim Vansa
>
> See this modified FunctionalTxInMemoryTest:
> {code:java}
> @Test(dataProvider = "owningModeAndReadWrites")
> public void testReadWriteAfterMods(boolean isOwner, WriteMethod method) throws Exception {
> Object KEY = getKey(isOwner);
> cache(0, DIST).put(KEY, "a");
> tm.begin();
> assertEquals("a", rw.eval(KEY, append("b")).join());
> assertEquals("ab", rw.evalMany(Collections.singleton(KEY), append("c")).findAny().get());
> assertEquals("abc", cache(0, DIST).get(KEY)); // <-- THIS FAILS
> assertEquals(null, rw.eval("otherKey", append("d")).join());
> assertEquals("abc", method.action.eval(KEY, wo, rw,
> MarshallableFunctions.returnReadOnlyFindOrNull(),
> (BiConsumer<EntryView.WriteEntryView<String>, String> & Serializable) (e, prev) -> {}, getClass()));
> tm.commit();
> }
> {code}
> When the entry was modified in given transaction, we have to do the read using functional identity read which passes all modifications along. We need to do this for all {{remoteGet}} s, including those invoked for {{putIfAbsent}} etc.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (ISPN-7489) org.jgroups.protocols.TCP emits errors when node leaves the cluster
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7489?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec updated ISPN-7489:
--------------------------------------
Priority: Critical (was: Major)
> org.jgroups.protocols.TCP emits errors when node leaves the cluster
> -------------------------------------------------------------------
>
> Key: ISPN-7489
> URL: https://issues.jboss.org/browse/ISPN-7489
> Project: Infinispan
> Issue Type: Bug
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
> Assignee: Sebastian Łaskawiec
> Priority: Critical
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> [transactions-repository-1-04x09] 18:09:12,193 ERROR [org.jgroups.protocols.TCP] (jgroups-30,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:12,310 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> [transactions-repository-1-04x09] 18:09:12,997 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:13,113 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> {noformat}
> Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * invoke `cd transaction-creator && mvn fabric8:run`
> * Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
> * Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (ISPN-7489) org.jgroups.protocols.TCP emits errors when node leaves the cluster
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/ISPN-7489?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on ISPN-7489:
--------------------------------
You should also reduce {{max_retransmit_time}} of 60s, or else you'll see these errors for 65s
> org.jgroups.protocols.TCP emits errors when node leaves the cluster
> -------------------------------------------------------------------
>
> Key: ISPN-7489
> URL: https://issues.jboss.org/browse/ISPN-7489
> Project: Infinispan
> Issue Type: Bug
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
> Assignee: Sebastian Łaskawiec
> Priority: Critical
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> [transactions-repository-1-04x09] 18:09:12,193 ERROR [org.jgroups.protocols.TCP] (jgroups-30,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:12,310 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> [transactions-repository-1-04x09] 18:09:12,997 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:13,113 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> {noformat}
> Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * invoke `cd transaction-creator && mvn fabric8:run`
> * Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
> * Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (ISPN-7489) org.jgroups.protocols.TCP emits errors when node leaves the cluster
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7489?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec commented on ISPN-7489:
-------------------------------------------
After another round of discussion with [~belaban] and [~dan.berindei] we figured out what happened:
{quote}
so here’s what happens: we have A|5=A,B,C and A leaves. A then installs view B|6=BC, and B and C send the VIEW-ACK to A
if A leaves before getting all VIEW-ACKs, you’ll see that error
the time A waits for all acks is defined by GMS.view_ack_collection_timeout
UNICAST3.conn_close_timeout should be lowered, the defualt in 4.final is 240s == 4 minutes
that 4 minutes plus the preceding max_retransmit_time of 1 min -> 5 minutes
{quote}
For the sake of the demo I've been using:
{code}
<protocol type="UNICAST3">
<property name="conn_close_timeout">5000</property>
</protocol>
{code}
> org.jgroups.protocols.TCP emits errors when node leaves the cluster
> -------------------------------------------------------------------
>
> Key: ISPN-7489
> URL: https://issues.jboss.org/browse/ISPN-7489
> Project: Infinispan
> Issue Type: Bug
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> [transactions-repository-1-04x09] 18:09:12,193 ERROR [org.jgroups.protocols.TCP] (jgroups-30,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:12,310 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> [transactions-repository-1-04x09] 18:09:12,997 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:13,113 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> {noformat}
> Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * invoke `cd transaction-creator && mvn fabric8:run`
> * Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
> * Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (ISPN-7489) org.jgroups.protocols.TCP emits errors when node leaves the cluster
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7489?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec reassigned ISPN-7489:
-----------------------------------------
Assignee: Sebastian Łaskawiec
> org.jgroups.protocols.TCP emits errors when node leaves the cluster
> -------------------------------------------------------------------
>
> Key: ISPN-7489
> URL: https://issues.jboss.org/browse/ISPN-7489
> Project: Infinispan
> Issue Type: Bug
> Components: Cloud Integrations, Core
> Affects Versions: 9.0.0.CR1
> Environment: * OpenShift {{v1.5.0-alpha.2+e4b43ee}}
> * Custom Infinispan Server build (based on [these instructions|https://github.com/slaskawi/infinispan-1/tree/custom_image]). SHA1 {{2b0731b21649a88a75ed71d21b9cc06ba365e947}}
> Reporter: Sebastian Łaskawiec
> Assignee: Sebastian Łaskawiec
>
> When I was performing [Spring Session and Kubernetes Rolling Update demo|https://bluejeans.com/s/pYKUg/] I encountered a couple of problems.
> One of the is this:
> {noformat}
> [transactions-repository-1-04x09] 18:09:12,193 ERROR [org.jgroups.protocols.TCP] (jgroups-30,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:12,310 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> [transactions-repository-1-04x09] 18:09:12,997 ERROR [org.jgroups.protocols.TCP] (jgroups-22,transactions-repository-1-04x09) JGRP000029: transactions-repository-1-04x09: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=5262, TP: [cluster_name=cluster]
> [transactions-repository-1-1f8dx] 18:09:13,113 ERROR [org.jgroups.protocols.TCP] (jgroups-16,transactions-repository-1-1f8dx) JGRP000029: transactions-repository-1-1f8dx: failed sending message to transactions-repository-1-4z05w (71 bytes): java.net.SocketTimeoutException: connect timed out, headers: GMS: GmsHeader[VIEW_ACK], UNICAST3: DATA, seqno=6259, TP: [cluster_name=cluster]
> {noformat}
> Full logs from Rolling Update process might be found here: https://gist.github.com/slaskawi/530241bb695f1f490bcb25eabaf9d676
> Steps to reproduce:
> * Start local OpenShift Cluster
> * invoke `./init_infrastructure.sh` from https://github.com/slaskawi/presentations/tree/ISPN-7487-reproducer
> * invoke `cd transaction-creator && mvn fabric8:run`
> * Do the rolling update: `oc deploy transactions-repository --latest -n myproject`
> * Observe logs `kubetail -l environment=infrastructure`
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months