[JBoss JIRA] (ISPN-2550) NoSuchElementException in Hot Rod Encoder
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2550?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-2550:
-----------------------------------------------
Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 886565|https://bugzilla.redhat.com/show_bug.cgi?id=886565] from NEW to ASSIGNED
> NoSuchElementException in Hot Rod Encoder
> -----------------------------------------
>
> Key: ISPN-2550
> URL: https://issues.jboss.org/browse/ISPN-2550
> Project: Infinispan
> Issue Type: Bug
> Components: Remote protocols
> Affects Versions: 5.2.0.Beta4
> Reporter: Michal Linhard
> Assignee: Dan Berindei
> Priority: Blocker
> Fix For: 5.2.0.CR1
>
>
> Tomas noticed this a while ago in a specific functional test:
> https://bugzilla.redhat.com/show_bug.cgi?id=875151
> I'm creating a more general JIRA, cause I'm having this in resilience test.
> What I found by quick debug, is that here:
> https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/ma...
> {code}
> for (segmentIdx <- 0 until numSegments) {
> val denormalizedSegmentHashIds = allDenormalizedHashIds(segmentIdx)
> val segmentOwners = ch.locateOwnersForSegment(segmentIdx)
> for (ownerIdx <- 0 until segmentOwners.length) {
> val address = segmentOwners(ownerIdx % segmentOwners.size)
> val serverAddress = members(address)
> val hashId = denormalizedSegmentHashIds(ownerIdx)
> log.tracef("Writing hash id %d for %s:%s", hashId, serverAddress.host, serverAddress.port)
> writeString(serverAddress.host, buf)
> writeUnsignedShort(serverAddress.port, buf)
> buf.writeInt(hashId)
> }
> }
> {code}
> we're trying to obtain serverAddress for nonexistent address and NoSuchElementException is not handled properly.
> It hapens after I kill a node in a resilience test and the exception appears when querying for the node in the members cache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 4 months
[JBoss JIRA] (ISPN-2642) IndexOutOfBoundsException in HotRodDecoder
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2642?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-2642:
-----------------------------------------------
Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 887323|https://bugzilla.redhat.com/show_bug.cgi?id=887323] from NEW to ASSIGNED
> IndexOutOfBoundsException in HotRodDecoder
> ------------------------------------------
>
> Key: ISPN-2642
> URL: https://issues.jboss.org/browse/ISPN-2642
> Project: Infinispan
> Issue Type: Bug
> Components: Remote protocols
> Affects Versions: 5.2.0.Beta5
> Reporter: Michal Linhard
> Assignee: Galder Zamarreño
> Fix For: 5.2.0.CR1, 5.2.0.Final
>
>
> In one of the elasticity tests in hyperion I'm getting:
> {code}
> 03:48:04,765 ERROR [org.infinispan.server.hotrod.HotRodDecoder] (HotRodClientMaster-179) ISPN005009: Unexpected error before any request parameters read: java.lang.IndexOutOfBoundsException: 2
> at scala.collection.mutable.ResizableArray$class.apply(ResizableArray.scala:44)
> at scala.collection.mutable.ArrayBuffer.apply(ArrayBuffer.scala:47)
> at org.infinispan.server.hotrod.AbstractTopologyAwareEncoder1x$$anonfun$writeHashTopologyHeader$1$$anonfun$apply$mcVI$sp$1.apply$mcVI$sp(AbstractTopologyAwareEncoder1x.scala:99) [infinispan-server-hotrod-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:81)
> at org.infinispan.server.hotrod.AbstractTopologyAwareEncoder1x$$anonfun$writeHashTopologyHeader$1.apply$mcVI$sp(AbstractTopologyAwareEncoder1x.scala:96) [infinispan-server-hotrod-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78)
> at org.infinispan.server.hotrod.AbstractTopologyAwareEncoder1x.writeHashTopologyHeader(AbstractTopologyAwareEncoder1x.scala:93) [infinispan-server-hotrod-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at org.infinispan.server.hotrod.AbstractEncoder1x.writeHeader(AbstractEncoder1x.scala:62) [infinispan-server-hotrod-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at org.infinispan.server.hotrod.HotRodEncoder.encode(HotRodEncoder.scala:63) [infinispan-server-hotrod-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:67) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:60) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.channel.Channels.write(Channels.java:712) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.channel.Channels.write(Channels.java:679) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:248) [netty-3.5.9.Final.jar:]
> at org.infinispan.server.core.AbstractProtocolDecoder.writeResponse(AbstractProtocolDecoder.scala:179) [infinispan-server-core-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeKey(AbstractProtocolDecoder.scala:115) [infinispan-server-core-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:71) [infinispan-server-core-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:47) [infinispan-server-core-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:502) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:437) [netty-3.5.9.Final.jar:]
> at org.infinispan.server.core.AbstractProtocolDecoder.messageReceived(AbstractProtocolDecoder.scala:387) [infinispan-server-core-5.2.0.Beta5-redhat-1.jar:5.2.0.Beta5-redhat-1]
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:472) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:333) [netty-3.5.9.Final.jar:]
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35) [netty-3.5.9.Final.jar:]
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_33]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_33]
> at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_33]
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 4 months
[JBoss JIRA] (ISPN-2081) Transaction leak caused by reordering between prepare and rollback
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2081?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-2081:
-----------------------------------------------
Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 841889|https://bugzilla.redhat.com/show_bug.cgi?id=841889] from ASSIGNED to ON_QA
> Transaction leak caused by reordering between prepare and rollback
> ------------------------------------------------------------------
>
> Key: ISPN-2081
> URL: https://issues.jboss.org/browse/ISPN-2081
> Project: Infinispan
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 5.1.5.FINAL
> Reporter: Mircea Markus
> Assignee: Mircea Markus
> Priority: Blocker
> Labels: jdg, jdg6
> Fix For: 5.2.0.Beta3, 5.2.0.Final
>
> Attachments: DistL1WriteSkewTest.txt, RollbackNotSentBeforePrepareTest.java
>
>
> There's no ordering between the prepare and commit/rollback messages, as the later are sent OOB.
> With this in mind, the following transaction leak might happen:
> Tx1 send prepare on nodes {A,B}
> 1. the message reaches A and timeouts but hasn't yet been processed on B
> 2. The transaction originator reacts immediately to the timeout received from A without waiting the response from B and sends a rollback request
> 3. The rollback request is processed on A and B
> 4. The initial prepare is then processed on B
> At this point we have an orphan transaction prepare on B.
> Whilst this is not causing any inconsistencies, it keeps keys locked indefinitely and is a memory leak.
> The solution would be to wait at 2 for all the prepare messages *before* sending the rollback.
> Attached is a unit test to reproduce the issue.
> Related mailing list thread: http://infinispan.markmail.org/search/#query:%20list%3Aorg.jboss.lists.in...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 4 months
[JBoss JIRA] (ISPN-2137) Potential tx lock leaks when nodes are added to the cluster
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2137?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-2137:
-----------------------------------------------
Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 841891|https://bugzilla.redhat.com/show_bug.cgi?id=841891] from ASSIGNED to ON_QA
> Potential tx lock leaks when nodes are added to the cluster
> -----------------------------------------------------------
>
> Key: ISPN-2137
> URL: https://issues.jboss.org/browse/ISPN-2137
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer, Transactions
> Affects Versions: 5.1.5.FINAL
> Reporter: Mircea Markus
> Assignee: Mircea Markus
> Priority: Critical
> Labels: jdg6
> Fix For: 5.2.0.Beta2, 5.2.0.Final
>
>
> Here's the sequence that might cause the leak:
> - a tx locking K is prepared and committed on node A
> - node B joins and becomes the primary owner of K
> - tx and the lock on key is moved from A to B as part of the state transfer
> - tx completion notification for tx is sent (async+oob) to A and not to B(old view)
> - there's no way for A to reply to the the originator telling it to re-send the tx completion notification to B as the call is async+oob
> - this would cause the lock transaction to leak on B
> Suggested solution:
> After a chat with Dan, following fixed seemed to be the most appropriate:
> when receiving a tx completion notification for a tx that's being migrated over as result of state transfer forward it to the new lock owner. With current blocking ST, this fwd call waits only after the tx state is transferred to the new owner. This would need some more thought for the new NBST code.
> Note: there's no issue in the case of nodes leaving the cluster, as the current logic of backup nodes would assure a proper cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 4 months
[JBoss JIRA] (ISPN-2553) JBossMarshaller can be used before properly initialized
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2553?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-2553:
-----------------------------------------------
Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 880528|https://bugzilla.redhat.com/show_bug.cgi?id=880528] from ASSIGNED to ON_QA
> JBossMarshaller can be used before properly initialized
> -------------------------------------------------------
>
> Key: ISPN-2553
> URL: https://issues.jboss.org/browse/ISPN-2553
> Project: Infinispan
> Issue Type: Bug
> Components: Marshalling
> Affects Versions: 5.2.0.Beta4
> Reporter: Radim Vansa
> Assignee: Galder Zamarreño
> Fix For: 5.2.0.Beta6, 5.2.0.Final
>
>
> The {{JBossMarshaller}} can be used before its {{start()}} method is called. I've noticed that with replicated cache without transactions, an OOB thread can start demarshalling {{SingleRpcCommand}} in {{CacheRpcCommandExternalizer}} but when it tries to create a new unmarshaller (through {{AbstractJBossMarshaller.startObjectInput(...)}} and the {{marshallerTL.initialValue()}} the {{baseCfg}} configuration is not fully initialized yet and this results in creating marshallers in {{PerThreadInstanceHolder}} with {{objectTable == null}}. Then, objects are deserialized to {{null}}.
> I have verified this by inserting some log messages into constructors and start method:
> {code}
> 19:49:02,404 INFO [org.infinispan.marshall.jboss.AbstractJBossMarshaller] (pool-1-thread-1) Creating AbstractJBossMarshaller with org.jboss.marshalling.MarshallingConfiguration@1d296aa3: classExternalizerFactor
> y=<org.infinispan.marshall.jboss.SerializeWithExtFactory@a18024a> exceptionListener=<null> instanceCount=16 classCount=8 bufferSize=512 version=3
> 19:49:02,409 INFO [org.infinispan.marshall.jboss.AbstractJBossMarshaller] (pool-1-thread-1) Creating JBossMarshaller org.infinispan.marshall.jboss.JBossMarshaller@2e3e4d73
> 19:49:02,410 INFO [org.infinispan.marshall.jboss.AbstractJBossMarshaller] (pool-1-thread-1) Starting JBossMarshaller
> {code}
> and into the thread-local initialization and to {{getUnmarshaller()}} just before {{factory.createUnmarshaller}}:
> {code}
> 19:49:02,410 ERROR [org.infinispan.marshall.jboss.AbstractJBossMarshaller] (OOB-49,rvansa-22965) No object table in org.jboss.marshalling.MarshallingConfiguration@7c4ed0bc: classExternalizerFactory=<org.infinisp
> an.marshall.jboss.SerializeWithExtFactory@a18024a> exceptionListener=<null> instanceCount=16 classCount=8 bufferSize=512 version=3, base is org.jboss.marshalling.MarshallingConfiguration@1d296aa3: classExternali
> zerFactory=<org.infinispan.marshall.jboss.SerializeWithExtFactory@a18024a> exceptionListener=<null> instanceCount=16 classCount=8 bufferSize=512 version=3
> 19:49:02,453 ERROR [org.infinispan.marshall.jboss.AbstractJBossMarshaller] (OOB-49,rvansa-22965) Unmarshaller with cfg org.jboss.marshalling.MarshallingConfiguration@7c4ed0bc: classExternalizerFactory=<org.infin
> ispan.marshall.jboss.SerializeWithExtFactory@a18024a> exceptionListener=<null> instanceCount=16 classCount=8 bufferSize=512 version=3
> {code}
> See that the timestamps for {{start()}} and thread-local initialization are same, and the base configuration ({{baseCfg}}) does not have {{objectTable}} initialized.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 4 months
[JBoss JIRA] (ISPN-2574) Segment transfer not restarted if the owner fails
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2574?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-2574:
-----------------------------------------------
Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 882162|https://bugzilla.redhat.com/show_bug.cgi?id=882162] from MODIFIED to ON_QA
> Segment transfer not restarted if the owner fails
> -------------------------------------------------
>
> Key: ISPN-2574
> URL: https://issues.jboss.org/browse/ISPN-2574
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer
> Affects Versions: 5.2.0.Beta4
> Reporter: Radim Vansa
> Assignee: Adrian Nistor
> Priority: Critical
> Fix For: 5.2.0.CR1
>
>
> Imagine this situation in distributed cache with 3 owners:
> 1) The segment X is owned by nodes A, B, C
> 2) Node B fails -> CH_UPDATE and then REBALANCE_START are broadcasted
> 3) Node D starts transfer of segment X from C
> 4) Node C fails -> another CH_UPDATE is broadcasted
> 5) D handes the CH_UPDATE and removes the transfer of segment X from C, but does not start another transfer from A
> The {{addedSegments}} does not contain the restarted transfer, because all transfers from write consistent hash are removed from it in the beginning - the segment is considered received here although the transfer is still in progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 4 months
