[JBoss JIRA] (ISPN-9530) Event ignores useStorageFormat causing ClassCastException
by William Burns (JIRA)
William Burns created ISPN-9530:
-----------------------------------
Summary: Event ignores useStorageFormat causing ClassCastException
Key: ISPN-9530
URL: https://issues.jboss.org/browse/ISPN-9530
Project: Infinispan
Issue Type: Bug
Components: Listeners
Reporter: William Burns
Assignee: William Burns
The test https://ci.infinispan.org/job/Infinispan/job/PR-6232/17/testReport/junit/... seems to fail randomly. It is caused due to the fact that the event cannot be marshalled properly.
The following is the stack trace when the exception is met
{code}
09:11:00,159 TRACE (HotRod-client-async-pool-3:[]) [HeaderDecoder] Response 23 belongs to PutOperation{(default), key=[B0x034BEBCEF4CA, value=[B0x033E027632, flags=6} on [id: 0x876415fe, L:/127.0.0.1:41108 - R:127.0.0.1/127.0.0.1:34808]
09:11:00,159 TRACE (HotRod-client-async-pool-3:[]) [HeaderDecoder] Decoding header for PutOperation{(default), key=[B0x034BEBCEF4CA, value=[B0x033E027632, flags=6} on [id: 0x876415fe, L:/127.0.0.1:41108 - R:127.0.0.1/127.0.0.1:34808]
09:11:00,159 WARN (HotRod-client-async-pool-2:[]) [HeaderDecoder] ISPN004039: Unable to complete reading event from server 127.0.0.1/127.0.0.1:39088
org.infinispan.client.hotrod.exceptions.HotRodClientException: ISPN004034: Unable to unmarshall bytes 3C737472696E673E262378333B4BEFBFBDEFBFBDEFBFBDEFBFBD3C2F737472696E673E
at org.infinispan.client.hotrod.marshall.MarshallerUtil.bytes2obj(MarshallerUtil.java:47) ~[classes/:?]
at org.infinispan.client.hotrod.DataFormat.keyToObj(DataFormat.java:103) ~[classes/:?]
at org.infinispan.client.hotrod.impl.protocol.Codec21.readCacheEvent(Codec21.java:79) ~[classes/:?]
at org.infinispan.client.hotrod.impl.transport.netty.HeaderDecoder.decode(HeaderDecoder.java:153) [classes/:?]
at org.infinispan.client.hotrod.impl.transport.netty.HintedReplayingDecoder.callDecode(HintedReplayingDecoder.java:98) [classes/:?]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) [netty-handler-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1414) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:945) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:806) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886) [netty-common-4.1.22.Final.jar:4.1.22.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.io.IOException: Unsupported protocol version 60
at org.jboss.marshalling.river.RiverUnmarshaller.start(RiverUnmarshaller.java:1345) ~[jboss-marshalling-osgi-2.0.5.Final.jar:2.0.5.Final]
at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.startObjectInput(AbstractJBossMarshaller.java:129) ~[classes/:?]
at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromByteBuffer(AbstractJBossMarshaller.java:110) ~[classes/:?]
at org.infinispan.commons.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:82) ~[classes/:?]
at org.infinispan.client.hotrod.marshall.MarshallerUtil.bytes2obj(MarshallerUtil.java:31) ~[classes/:?]
... 23 more
{code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years
[JBoss JIRA] (ISPN-9511) Expired event is not raised when modifying an expired entry
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-9511?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-9511:
-------------------------------------
It looks like this was a bug in https://github.com/infinispan/infinispan/commit/14afdf555b826d78622a534e5...
We need to just set the value if the listener using storage format only. This prevents the extra serialization.
> Expired event is not raised when modifying an expired entry
> -----------------------------------------------------------
>
> Key: ISPN-9511
> URL: https://issues.jboss.org/browse/ISPN-9511
> Project: Infinispan
> Issue Type: Bug
> Components: Listeners
> Affects Versions: 9.3.3.Final
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 9.4.0.Final
>
>
> Due to the old way of implementing remove expired for lifespan, we didn't raise an expired event when writing to an entry. This was mostly to cause circular dependencies. But with the new remove expired max idle changes, this is now possible.
> Without this change listeners can be in an inconsistent state, possibly, as the following could happen:
> 1. Entry is created
> 2. Listener is notified of creation
> 3. Entry expires (no event yet)
> 4. Entry is written to (created)
> 5. Listener is notified of creation.
> In this case there is no intermediate state where the listener thought there was no entry. This also becomes problematic if you are listening only for events that don't include create.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years
[JBoss JIRA] (ISPN-9511) Expired event is not raised when modifying an expired entry
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-9511?page=com.atlassian.jira.plugin.... ]
William Burns edited comment on ISPN-9511 at 9/18/18 12:57 PM:
---------------------------------------------------------------
It looks like this was a bug in https://github.com/infinispan/infinispan/commit/14afdf555b826d78622a534e5...
We need to just set the value if the listener using storage format only. This prevents the extra serialization.
And now that the remote listeners all use storage format, this test found it.
was (Author: william.burns):
It looks like this was a bug in https://github.com/infinispan/infinispan/commit/14afdf555b826d78622a534e5...
We need to just set the value if the listener using storage format only. This prevents the extra serialization.
> Expired event is not raised when modifying an expired entry
> -----------------------------------------------------------
>
> Key: ISPN-9511
> URL: https://issues.jboss.org/browse/ISPN-9511
> Project: Infinispan
> Issue Type: Bug
> Components: Listeners
> Affects Versions: 9.3.3.Final
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 9.4.0.Final
>
>
> Due to the old way of implementing remove expired for lifespan, we didn't raise an expired event when writing to an entry. This was mostly to cause circular dependencies. But with the new remove expired max idle changes, this is now possible.
> Without this change listeners can be in an inconsistent state, possibly, as the following could happen:
> 1. Entry is created
> 2. Listener is notified of creation
> 3. Entry expires (no event yet)
> 4. Entry is written to (created)
> 5. Listener is notified of creation.
> In this case there is no intermediate state where the listener thought there was no entry. This also becomes problematic if you are listening only for events that don't include create.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years
[JBoss JIRA] (ISPN-9529) Docs: Create Product Attributes and Clean Up Invalid Syntax
by Don Naro (JIRA)
Don Naro created ISPN-9529:
------------------------------
Summary: Docs: Create Product Attributes and Clean Up Invalid Syntax
Key: ISPN-9529
URL: https://issues.jboss.org/browse/ISPN-9529
Project: Infinispan
Issue Type: Enhancement
Reporter: Don Naro
Assignee: Don Naro
To build ISPN documentation in the RH Pantheon system, we need to define attributes for the product, add some index files, and clean up some syntax that break validation during build with the RH tooling.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years
[JBoss JIRA] (ISPN-9517) State transfer times out if initiated with yet to be verified suspected member and reincarnated member
by Paul Ferraro (JIRA)
[ https://issues.jboss.org/browse/ISPN-9517?page=com.atlassian.jira.plugin.... ]
Paul Ferraro updated ISPN-9517:
-------------------------------
Attachment: Test.java
> State transfer times out if initiated with yet to be verified suspected member and reincarnated member
> ------------------------------------------------------------------------------------------------------
>
> Key: ISPN-9517
> URL: https://issues.jboss.org/browse/ISPN-9517
> Project: Infinispan
> Issue Type: Bug
> Components: State Transfer
> Affects Versions: 9.3.3.Final
> Reporter: Paul Ferraro
> Assignee: Paul Ferraro
> Attachments: Test.java, node-1.zip, node-2.zip
>
>
> Here's the scenario:
> 1. Cluster contains caches on 2 members, node-1 and node-2
> 2. node-2 is killed
> 3. node-2 is restarted (using same physical address)
> 4. State transfer initiates, view contains node-1, suspected node-2, and reincarnated node-2
> 5. State transfer times out
> Log of node-1 includes:
> {noformat}
> 12:09:51,882 WARN [org.infinispan.topology.ClusterTopologyManagerImpl] (transport-thread--p14-t4) ISPN000197: Error updating cluster member list: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 3 from node-2
> at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167)
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87)
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_181]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [rt.jar:1.8.0_181]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_181]
> Suppressed: org.infinispan.util.logging.TraceException
> at org.infinispan.remoting.transport.Transport.invokeRemotely(Transport.java:75)
> at org.infinispan.topology.ClusterTopologyManagerImpl.confirmMembersAvailable(ClusterTopologyManagerImpl.java:525)
> at org.infinispan.topology.ClusterTopologyManagerImpl.updateCacheMembers(ClusterTopologyManagerImpl.java:508)
> at org.infinispan.topology.ClusterTopologyManagerImpl.handleClusterView(ClusterTopologyManagerImpl.java:321)
> at org.infinispan.topology.ClusterTopologyManagerImpl.access$500(ClusterTopologyManagerImpl.java:87)
> at org.infinispan.topology.ClusterTopologyManagerImpl$ClusterViewListener.lambda$handleViewChange$0(ClusterTopologyManagerImpl.java:731)
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:175)
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:37)
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:227)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
> at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
> ... 1 more
> {noformat}
> I've attached trace logs from node-1 and node-2.
> Changing ClusterTopologyManagerImpl.confirmMembersAvailable() to use ResponseMode.SYNCHRONOUS_IGNORE_LEAVERS instead of ResponseMode.SYNCHRONOUS allows state transfer to complete successfully.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years
[JBoss JIRA] (ISPN-9517) State transfer times out if initiated with yet to be verified suspected member and reincarnated member
by Paul Ferraro (JIRA)
[ https://issues.jboss.org/browse/ISPN-9517?page=com.atlassian.jira.plugin.... ]
Paul Ferraro updated ISPN-9517:
-------------------------------
Attachment: (was: Test.java)
> State transfer times out if initiated with yet to be verified suspected member and reincarnated member
> ------------------------------------------------------------------------------------------------------
>
> Key: ISPN-9517
> URL: https://issues.jboss.org/browse/ISPN-9517
> Project: Infinispan
> Issue Type: Bug
> Components: State Transfer
> Affects Versions: 9.3.3.Final
> Reporter: Paul Ferraro
> Assignee: Paul Ferraro
> Attachments: node-1.zip, node-2.zip
>
>
> Here's the scenario:
> 1. Cluster contains caches on 2 members, node-1 and node-2
> 2. node-2 is killed
> 3. node-2 is restarted (using same physical address)
> 4. State transfer initiates, view contains node-1, suspected node-2, and reincarnated node-2
> 5. State transfer times out
> Log of node-1 includes:
> {noformat}
> 12:09:51,882 WARN [org.infinispan.topology.ClusterTopologyManagerImpl] (transport-thread--p14-t4) ISPN000197: Error updating cluster member list: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 3 from node-2
> at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167)
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87)
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_181]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [rt.jar:1.8.0_181]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_181]
> Suppressed: org.infinispan.util.logging.TraceException
> at org.infinispan.remoting.transport.Transport.invokeRemotely(Transport.java:75)
> at org.infinispan.topology.ClusterTopologyManagerImpl.confirmMembersAvailable(ClusterTopologyManagerImpl.java:525)
> at org.infinispan.topology.ClusterTopologyManagerImpl.updateCacheMembers(ClusterTopologyManagerImpl.java:508)
> at org.infinispan.topology.ClusterTopologyManagerImpl.handleClusterView(ClusterTopologyManagerImpl.java:321)
> at org.infinispan.topology.ClusterTopologyManagerImpl.access$500(ClusterTopologyManagerImpl.java:87)
> at org.infinispan.topology.ClusterTopologyManagerImpl$ClusterViewListener.lambda$handleViewChange$0(ClusterTopologyManagerImpl.java:731)
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:175)
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:37)
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:227)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
> at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
> ... 1 more
> {noformat}
> I've attached trace logs from node-1 and node-2.
> Changing ClusterTopologyManagerImpl.confirmMembersAvailable() to use ResponseMode.SYNCHRONOUS_IGNORE_LEAVERS instead of ResponseMode.SYNCHRONOUS allows state transfer to complete successfully.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years
[JBoss JIRA] (HRJS-76) Hot Rod 2.5 protocol support broken
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/HRJS-76?page=com.atlassian.jira.plugin.sy... ]
Galder Zamarreño updated HRJS-76:
---------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Hot Rod 2.5 protocol support broken
> -----------------------------------
>
> Key: HRJS-76
> URL: https://issues.jboss.org/browse/HRJS-76
> Project: Infinispan Javascript client
> Issue Type: Bug
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Fix For: 0.6.0
>
>
> Trying to run smoke tests with protocol 2.5 results in failure:
> {code}
> rm -drf tmp-tests.log && node node_modules/jasmine-node/lib/jasmine-node/cli.js spec/infinispan_local_spec.js --captureExceptions
> FFFF.FFFFFFFFFFFFTypeError: Cannot read property 'decodeMedia' of undefined
> at decoderMedia (/Users/g/1/infinispan-js-client/lib/protocols.js:73:43)
> at Protocol25.decodeEvent (/Users/g/1/infinispan-js-client/lib/protocols.js:589:28)
> at Socket.onData (/Users/g/1/infinispan-js-client/lib/io.js:121:40)
> at emitOne (events.js:116:13)
> at Socket.emit (events.js:211:7)
> at addChunk (_stream_readable.js:263:12)
> at readableAddChunk (_stream_readable.js:250:11)
> at Socket.Readable.push (_stream_readable.js:208:10)
> at TCP.onread (net.js:597:20)
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years
[JBoss JIRA] (ISPN-9517) State transfer times out if initiated with yet to be verified suspected member and reincarnated member
by Paul Ferraro (JIRA)
[ https://issues.jboss.org/browse/ISPN-9517?page=com.atlassian.jira.plugin.... ]
Paul Ferraro updated ISPN-9517:
-------------------------------
Attachment: (was: Test.java)
> State transfer times out if initiated with yet to be verified suspected member and reincarnated member
> ------------------------------------------------------------------------------------------------------
>
> Key: ISPN-9517
> URL: https://issues.jboss.org/browse/ISPN-9517
> Project: Infinispan
> Issue Type: Bug
> Components: State Transfer
> Affects Versions: 9.3.3.Final
> Reporter: Paul Ferraro
> Assignee: Paul Ferraro
> Attachments: Test.java, node-1.zip, node-2.zip
>
>
> Here's the scenario:
> 1. Cluster contains caches on 2 members, node-1 and node-2
> 2. node-2 is killed
> 3. node-2 is restarted (using same physical address)
> 4. State transfer initiates, view contains node-1, suspected node-2, and reincarnated node-2
> 5. State transfer times out
> Log of node-1 includes:
> {noformat}
> 12:09:51,882 WARN [org.infinispan.topology.ClusterTopologyManagerImpl] (transport-thread--p14-t4) ISPN000197: Error updating cluster member list: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 3 from node-2
> at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167)
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87)
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_181]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [rt.jar:1.8.0_181]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_181]
> Suppressed: org.infinispan.util.logging.TraceException
> at org.infinispan.remoting.transport.Transport.invokeRemotely(Transport.java:75)
> at org.infinispan.topology.ClusterTopologyManagerImpl.confirmMembersAvailable(ClusterTopologyManagerImpl.java:525)
> at org.infinispan.topology.ClusterTopologyManagerImpl.updateCacheMembers(ClusterTopologyManagerImpl.java:508)
> at org.infinispan.topology.ClusterTopologyManagerImpl.handleClusterView(ClusterTopologyManagerImpl.java:321)
> at org.infinispan.topology.ClusterTopologyManagerImpl.access$500(ClusterTopologyManagerImpl.java:87)
> at org.infinispan.topology.ClusterTopologyManagerImpl$ClusterViewListener.lambda$handleViewChange$0(ClusterTopologyManagerImpl.java:731)
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:175)
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:37)
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:227)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
> at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
> ... 1 more
> {noformat}
> I've attached trace logs from node-1 and node-2.
> Changing ClusterTopologyManagerImpl.confirmMembersAvailable() to use ResponseMode.SYNCHRONOUS_IGNORE_LEAVERS instead of ResponseMode.SYNCHRONOUS allows state transfer to complete successfully.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years
[JBoss JIRA] (ISPN-9517) State transfer times out if initiated with yet to be verified suspected member and reincarnated member
by Paul Ferraro (JIRA)
[ https://issues.jboss.org/browse/ISPN-9517?page=com.atlassian.jira.plugin.... ]
Paul Ferraro updated ISPN-9517:
-------------------------------
Attachment: Test.java
> State transfer times out if initiated with yet to be verified suspected member and reincarnated member
> ------------------------------------------------------------------------------------------------------
>
> Key: ISPN-9517
> URL: https://issues.jboss.org/browse/ISPN-9517
> Project: Infinispan
> Issue Type: Bug
> Components: State Transfer
> Affects Versions: 9.3.3.Final
> Reporter: Paul Ferraro
> Assignee: Paul Ferraro
> Attachments: Test.java, node-1.zip, node-2.zip
>
>
> Here's the scenario:
> 1. Cluster contains caches on 2 members, node-1 and node-2
> 2. node-2 is killed
> 3. node-2 is restarted (using same physical address)
> 4. State transfer initiates, view contains node-1, suspected node-2, and reincarnated node-2
> 5. State transfer times out
> Log of node-1 includes:
> {noformat}
> 12:09:51,882 WARN [org.infinispan.topology.ClusterTopologyManagerImpl] (transport-thread--p14-t4) ISPN000197: Error updating cluster member list: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 3 from node-2
> at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167)
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87)
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_181]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [rt.jar:1.8.0_181]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_181]
> Suppressed: org.infinispan.util.logging.TraceException
> at org.infinispan.remoting.transport.Transport.invokeRemotely(Transport.java:75)
> at org.infinispan.topology.ClusterTopologyManagerImpl.confirmMembersAvailable(ClusterTopologyManagerImpl.java:525)
> at org.infinispan.topology.ClusterTopologyManagerImpl.updateCacheMembers(ClusterTopologyManagerImpl.java:508)
> at org.infinispan.topology.ClusterTopologyManagerImpl.handleClusterView(ClusterTopologyManagerImpl.java:321)
> at org.infinispan.topology.ClusterTopologyManagerImpl.access$500(ClusterTopologyManagerImpl.java:87)
> at org.infinispan.topology.ClusterTopologyManagerImpl$ClusterViewListener.lambda$handleViewChange$0(ClusterTopologyManagerImpl.java:731)
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:175)
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:37)
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:227)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
> at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
> ... 1 more
> {noformat}
> I've attached trace logs from node-1 and node-2.
> Changing ClusterTopologyManagerImpl.confirmMembersAvailable() to use ResponseMode.SYNCHRONOUS_IGNORE_LEAVERS instead of ResponseMode.SYNCHRONOUS allows state transfer to complete successfully.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years
[JBoss JIRA] (ISPN-9511) Expired event is not raised when modifying an expired entry
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-9511?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-9511:
-------------------------------------
So the problem is that the actual error was never stored in the correct log file.
If I check the global trace file, I can find the exception causing the "failover".
{code}
09:11:00,159 TRACE (HotRod-client-async-pool-3:[]) [HeaderDecoder] Response 23 belongs to PutOperation{(default), key=[B0x034BEBCEF4CA, value=[B0x033E027632, flags=6} on [id: 0x876415fe, L:/127.0.0.1:41108 - R:127.0.0.1/127.0.0.1:34808]
09:11:00,159 TRACE (HotRod-client-async-pool-3:[]) [HeaderDecoder] Decoding header for PutOperation{(default), key=[B0x034BEBCEF4CA, value=[B0x033E027632, flags=6} on [id: 0x876415fe, L:/127.0.0.1:41108 - R:127.0.0.1/127.0.0.1:34808]
09:11:00,159 WARN (HotRod-client-async-pool-2:[]) [HeaderDecoder] ISPN004039: Unable to complete reading event from server 127.0.0.1/127.0.0.1:39088
org.infinispan.client.hotrod.exceptions.HotRodClientException: ISPN004034: Unable to unmarshall bytes 3C737472696E673E262378333B4BEFBFBDEFBFBDEFBFBDEFBFBD3C2F737472696E673E
at org.infinispan.client.hotrod.marshall.MarshallerUtil.bytes2obj(MarshallerUtil.java:47) ~[classes/:?]
at org.infinispan.client.hotrod.DataFormat.keyToObj(DataFormat.java:103) ~[classes/:?]
at org.infinispan.client.hotrod.impl.protocol.Codec21.readCacheEvent(Codec21.java:79) ~[classes/:?]
at org.infinispan.client.hotrod.impl.transport.netty.HeaderDecoder.decode(HeaderDecoder.java:153) [classes/:?]
at org.infinispan.client.hotrod.impl.transport.netty.HintedReplayingDecoder.callDecode(HintedReplayingDecoder.java:98) [classes/:?]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) [netty-handler-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1414) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:945) [netty-transport-4.1.22.Final.jar:4.1.22.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:806) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886) [netty-common-4.1.22.Final.jar:4.1.22.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.io.IOException: Unsupported protocol version 60
at org.jboss.marshalling.river.RiverUnmarshaller.start(RiverUnmarshaller.java:1345) ~[jboss-marshalling-osgi-2.0.5.Final.jar:2.0.5.Final]
at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.startObjectInput(AbstractJBossMarshaller.java:129) ~[classes/:?]
at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromByteBuffer(AbstractJBossMarshaller.java:110) ~[classes/:?]
at org.infinispan.commons.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:82) ~[classes/:?]
at org.infinispan.client.hotrod.marshall.MarshallerUtil.bytes2obj(MarshallerUtil.java:31) ~[classes/:?]
... 23 more
{code}
> Expired event is not raised when modifying an expired entry
> -----------------------------------------------------------
>
> Key: ISPN-9511
> URL: https://issues.jboss.org/browse/ISPN-9511
> Project: Infinispan
> Issue Type: Bug
> Components: Listeners
> Affects Versions: 9.3.3.Final
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 9.4.0.Final
>
>
> Due to the old way of implementing remove expired for lifespan, we didn't raise an expired event when writing to an entry. This was mostly to cause circular dependencies. But with the new remove expired max idle changes, this is now possible.
> Without this change listeners can be in an inconsistent state, possibly, as the following could happen:
> 1. Entry is created
> 2. Listener is notified of creation
> 3. Entry expires (no event yet)
> 4. Entry is written to (created)
> 5. Listener is notified of creation.
> In this case there is no intermediate state where the listener thought there was no entry. This also becomes problematic if you are listening only for events that don't include create.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years