[JBoss JIRA] (ISPN-8195) Transaction fails to commit when a node crashes
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-8195?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-8195:
------------------------------
Component/s: Transactions
> Transaction fails to commit when a node crashes
> -----------------------------------------------
>
> Key: ISPN-8195
> URL: https://issues.jboss.org/browse/ISPN-8195
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Transactions
> Affects Versions: 9.1.0.Final
> Reporter: Radim Vansa
> Attachments: log.zip
>
>
> Nodes ABC, key is owned by BC:
> 1. C prepares a transaction modifying only one key [BC], prepare succeeds on both
> 2. B crashes
> 3. C tries to send CommitCommand to B and gets {{CacheNotFoundResponse}}
> 4. C throws OTE, which gets handled by STI and retried
> 5. A becomes an owner of key in the next topology
> 6. C sends CommitCommand to all owners, including A
> 7. A does not find the transaction prepared and throws {{IllegalStateException: Remote transaction not found: GlobalTx:test-NodeC-45028:1}}
> 8. C fails the transaction because of the {{IllegalStateException}}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 4 months
[JBoss JIRA] (ISPN-8195) Transaction fails to commit when a node crashes
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-8195?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-8195:
------------------------------
Attachment: log.zip
> Transaction fails to commit when a node crashes
> -----------------------------------------------
>
> Key: ISPN-8195
> URL: https://issues.jboss.org/browse/ISPN-8195
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.1.0.Final
> Reporter: Radim Vansa
> Attachments: log.zip
>
>
> Nodes ABC, key is owned by BC:
> 1. C prepares a transaction modifying only one key [BC], prepare succeeds on both
> 2. B crashes
> 3. C tries to send CommitCommand to B and gets {{CacheNotFoundResponse}}
> 4. C throws OTE, which gets handled by STI and retried
> 5. A becomes an owner of key in the next topology
> 6. C sends CommitCommand to all owners, including A
> 7. A does not find the transaction prepared and throws {{IllegalStateException: Remote transaction not found: GlobalTx:test-NodeC-45028:1}}
> 8. C fails the transaction because of the {{IllegalStateException}}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 4 months
[JBoss JIRA] (ISPN-8195) Transaction fails to commit when a node crashes
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-8195?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-8195:
------------------------------
Summary: Transaction fails to commit when a node crashes (was: Transaction fails to commit when the a node crashes)
> Transaction fails to commit when a node crashes
> -----------------------------------------------
>
> Key: ISPN-8195
> URL: https://issues.jboss.org/browse/ISPN-8195
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.1.0.Final
> Reporter: Radim Vansa
>
> Nodes ABC, key is owned by BC:
> 1. C prepares a transaction modifying only one key [BC], prepare succeeds on both
> 2. B crashes
> 3. C tries to send CommitCommand to B and gets {{CacheNotFoundResponse}}
> 4. C throws OTE, which gets handled by STI and retried
> 5. A becomes an owner of key in the next topology
> 6. C sends CommitCommand to all owners, including A
> 7. A does not find the transaction prepared and throws {{IllegalStateException: Remote transaction not found: GlobalTx:test-NodeC-45028:1}}
> 8. C fails the transaction because of the {{IllegalStateException}}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 4 months
[JBoss JIRA] (ISPN-8195) Transaction fails to commit when the a node crashes
by Radim Vansa (JIRA)
Radim Vansa created ISPN-8195:
---------------------------------
Summary: Transaction fails to commit when the a node crashes
Key: ISPN-8195
URL: https://issues.jboss.org/browse/ISPN-8195
Project: Infinispan
Issue Type: Bug
Components: Core
Affects Versions: 9.1.0.Final
Reporter: Radim Vansa
Nodes ABC, key is owned by BC:
1. C prepares a transaction modifying only one key [BC], prepare succeeds on both
2. B crashes
3. C tries to send CommitCommand to B and gets {{CacheNotFoundResponse}}
4. C throws OTE, which gets handled by STI and retried
5. A becomes an owner of key in the next topology
6. C sends CommitCommand to all owners, including A
7. A does not find the transaction prepared and throws {{IllegalStateException: Remote transaction not found: GlobalTx:test-NodeC-45028:1}}
8. C fails the transaction because of the {{IllegalStateException}}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 4 months
[JBoss JIRA] (ISPN-8194) Random failures in the hotrod server suite due to port conficts
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-8194?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-8194:
------------------------------------
Description:
{noformat}
[ERROR] testValidateProtocolServerNullProperties(org.infinispan.server.hotrod.HotRodServerTest) Time elapsed: 0.005 s <<< FAILURE!
io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: Address already in use
at io.netty.channel.unix.Errors.newIOException(Errors.java:117)
at io.netty.channel.unix.Socket.bind(Socket.java:234)
at io.netty.channel.epoll.EpollServerSocketChannel.doBind(EpollServerSocketChannel.java:91)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:554)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1258)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:980)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:250)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:365)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:748)
{noformat}
was:
{format}
[ERROR] testValidateProtocolServerNullProperties(org.infinispan.server.hotrod.HotRodServerTest) Time elapsed: 0.005 s <<< FAILURE!
io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: Address already in use
at io.netty.channel.unix.Errors.newIOException(Errors.java:117)
at io.netty.channel.unix.Socket.bind(Socket.java:234)
at io.netty.channel.epoll.EpollServerSocketChannel.doBind(EpollServerSocketChannel.java:91)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:554)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1258)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:980)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:250)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:365)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:748)
{format}
> Random failures in the hotrod server suite due to port conficts
> ---------------------------------------------------------------
>
> Key: ISPN-8194
> URL: https://issues.jboss.org/browse/ISPN-8194
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 9.1.0.Final
> Reporter: Gustavo Fernandes
> Labels: testsuite_stability
>
> {noformat}
> [ERROR] testValidateProtocolServerNullProperties(org.infinispan.server.hotrod.HotRodServerTest) Time elapsed: 0.005 s <<< FAILURE!
> io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: Address already in use
> at io.netty.channel.unix.Errors.newIOException(Errors.java:117)
> at io.netty.channel.unix.Socket.bind(Socket.java:234)
> at io.netty.channel.epoll.EpollServerSocketChannel.doBind(EpollServerSocketChannel.java:91)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:554)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1258)
> at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
> at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
> at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:980)
> at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:250)
> at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:365)
> at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304)
> at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 4 months
[JBoss JIRA] (ISPN-8194) Random failures in the hotrod server suite due to port conficts
by Gustavo Fernandes (JIRA)
Gustavo Fernandes created ISPN-8194:
---------------------------------------
Summary: Random failures in the hotrod server suite due to port conficts
Key: ISPN-8194
URL: https://issues.jboss.org/browse/ISPN-8194
Project: Infinispan
Issue Type: Bug
Affects Versions: 9.1.0.Final
Reporter: Gustavo Fernandes
{format}
[ERROR] testValidateProtocolServerNullProperties(org.infinispan.server.hotrod.HotRodServerTest) Time elapsed: 0.005 s <<< FAILURE!
io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: Address already in use
at io.netty.channel.unix.Errors.newIOException(Errors.java:117)
at io.netty.channel.unix.Socket.bind(Socket.java:234)
at io.netty.channel.epoll.EpollServerSocketChannel.doBind(EpollServerSocketChannel.java:91)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:554)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1258)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:980)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:250)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:365)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:748)
{format}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 4 months
[JBoss JIRA] (ISPN-8186) DistTopologyChangeUnderLoadTest.testPutsSucceedWhileTopologyChanges sometimes fails with magic error
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-8186?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño commented on ISPN-8186:
----------------------------------------
Managed to replicate issue with TRACE on:
https://www.dropbox.com/s/72gt34bbs6wzxi3/trace-infinispan-8186.log?dl=0
> DistTopologyChangeUnderLoadTest.testPutsSucceedWhileTopologyChanges sometimes fails with magic error
> ----------------------------------------------------------------------------------------------------
>
> Key: ISPN-8186
> URL: https://issues.jboss.org/browse/ISPN-8186
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Protocols
> Affects Versions: 9.1.0.Final
> Reporter: Tristan Tarrant
> Assignee: Galder Zamarreño
>
> Happens sometimes on CI:
> DistTopologyChangeUnderLoadTest.testPutsSucceedWhileTopologyChanges
> {code}
> org.infinispan.client.hotrod.exceptions.InvalidResponseException::
> Invalid magic number. Expected 0xa1 and received 0xdb at
> org.infinispan.client.hotrod.impl.protocol.Codec20.readMagic(Codec20.java:333) at
> org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:135) at
> org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:60) at
> org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:58) at
> org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:34) at
> org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:56) at
> org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:268) at
> org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:77) at
> org.infinispan.client.hotrod.DistTopologyChangeUnderLoadTest.testPutsSucceedWhileTopologyChanges(DistTopologyChangeUnderLoadTest.java:57) at
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at
> java.lang.Thread.run(Thread.java:748) ... Removed 16 stack frames
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 4 months