[infinispan-issues] [JBoss JIRA] (ISPN-2781) NPE in AbstractTopologyAwareEncoder1x

[Galder Zamarreño (JIRA)]

     [ https://issues.jboss.org/browse/ISPN-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Galder Zamarreño updated ISPN-2781:
-----------------------------------

    Assignee: Dan Berindei  (was: Galder Zamarreño)


Assigning it to Dan. This seems related to changes in ISPN-2632 and ISPN-2738 whereby those server addresses that cannot be found in Hot Rod topology cache are still in the hashId/member cache distribution manager. This can result in null server address instances which is where the NPE comes from.

[~dan.berindei], This is probably a timing issue between crashed member detection listener, which removes nodes from topology cache based on JGroups view changes, and hash topology header generation logic. This is probably left over from when the transport view id was considered as main indicator to know whether a new view had to be sent to clients. With switch to cache topology id, this needs adjusting.

I still find this new logic a bit fragile, particularly checks like this:
https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/main/scala/org/infinispan/server/hotrod/AbstractEncoder1x.scala#L180

Earlier versions of this code kept track of a volatile view id that only got updated once we knew the cache had the right contents (i.e. once a node had been added or removed), and only at that point the view id changed and a new view was detected/sent.

The new logic that relies on double checking whether the cache has all the contents expected whenever a response has to be generated is IMO, more wasteful (needless collection contents checks) and more fragile (there's view/barrier/latch that marks that a new view is available and the raw data used to generate the view is reliable).

                
> NPE in AbstractTopologyAwareEncoder1x
> -------------------------------------
>
>                 Key: ISPN-2781
>                 URL: https://issues.jboss.org/browse/ISPN-2781
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Server
>    Affects Versions: 5.2.0.CR3
>            Reporter: Michal Linhard
>            Assignee: Dan Berindei
>             Fix For: 5.2.2.Final, 5.3.0.Final
>
>
> got this in an elasticity test (32nodes)
> {code}
> 06:23:20,660 ERROR [org.infinispan.server.hotrod.HotRodDecoder] (HotRodServerWorker-151) ISPN005009: Unexpected error before any request parameters read: java.lang.NullPointerException
> 	at org.infinispan.server.hotrod.AbstractTopologyAwareEncoder1x$$anonfun$writeHashTopologyUpdate11$3.apply(AbstractTopologyAwareEncoder1x.scala:102) [infinispan-server-hotrod-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.infinispan.server.hotrod.AbstractTopologyAwareEncoder1x$$anonfun$writeHashTopologyUpdate11$3.apply(AbstractTopologyAwareEncoder1x.scala:101) [infinispan-server-hotrod-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> 	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
> 	at org.infinispan.server.hotrod.AbstractTopologyAwareEncoder1x.writeHashTopologyUpdate11(AbstractTopologyAwareEncoder1x.scala:101) [infinispan-server-hotrod-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.infinispan.server.hotrod.AbstractTopologyAwareEncoder1x.writeHashTopologyUpdate(AbstractTopologyAwareEncoder1x.scala:52) [infinispan-server-hotrod-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.infinispan.server.hotrod.AbstractEncoder1x.writeHeader(AbstractEncoder1x.scala:63) [infinispan-server-hotrod-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.infinispan.server.hotrod.HotRodEncoder.encode(HotRodEncoder.scala:63) [infinispan-server-hotrod-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:66) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.channel.Channels.write(Channels.java:704) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.channel.Channels.write(Channels.java:671) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:248) [netty-3.6.1.Final.jar:]
> 	at org.infinispan.server.core.AbstractProtocolDecoder.writeResponse(AbstractProtocolDecoder.scala:179) [infinispan-server-core-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.infinispan.server.hotrod.HotRodDecoder.customDecodeHeader(HotRodDecoder.scala:157) [infinispan-server-hotrod-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.infinispan.server.core.AbstractProtocolDecoder.decodeHeader(AbstractProtocolDecoder.scala:105) [infinispan-server-core-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:70) [infinispan-server-core-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:47) [infinispan-server-core-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) [netty-3.6.1.Final.jar:]
> 	at org.infinispan.server.core.AbstractProtocolDecoder.messageReceived(AbstractProtocolDecoder.scala:387) [infinispan-server-core-5.2.0.CR3-redhat-1.jar:5.2.0.CR3-redhat-1]
> 	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:313) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88) [netty-3.6.1.Final.jar:]
> 	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.6.1.Final.jar:]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_33]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_33]
> 	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_33]
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira