[jboss-jira] [JBoss JIRA] (AS7-5393) "Problems unmarshalling remote command from byte buffer: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 50"

Doug Mayer (JIRA) jira-events at lists.jboss.org
Fri Oct 12 16:05:03 EDT 2012 

    [ https://issues.jboss.org/browse/AS7-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726055#comment-12726055 ] 

Doug Mayer commented on AS7-5393:
---------------------------------

Having tried removing start='EAGER' with only temporary relief, I can confirm that deploying a JBoss app to a 2- or 3-node cluster in quick succession will cause this error to occur and not be able to recover on its own.

We deploy to machines serially, waiting until the .dodeploy is cleaned up before moving onto the next server. When we pause for ~10 seconds AFTER dodeploy is gone before moving onto the next server, we only see the error once (if at all). The same can be seen when restarting JBoss (TorqueBox) servers - they log this error indefinitely when restarted too quickly, and are generally healthy when giving them some space between restarts.

This almost always comes after a "ISPN000172: Failed to prepare view CacheView" error for us.
                
> "Problems unmarshalling remote command from byte buffer: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 50"
> --------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AS7-5393
>                 URL: https://issues.jboss.org/browse/AS7-5393
>             Project: Application Server 7
>          Issue Type: Bug
>          Components: Clustering
>    Affects Versions: 7.1.3.Final (EAP)
>         Environment: affected: AS7 master
>            Reporter: Radoslav Husar
>            Assignee: Paul Ferraro
>             Fix For: 7.2.0.CR1, 7.1.4.Final (EAP)
>
>
> As noticed by Kabir in AS7 master TS runs upon failover:
> http://lightning.mw.lab.eng.bos.redhat.com/jenkins/job/as7-master-testsuite-ip6/3337/testReport/junit/org.jboss.as.test.clustering.cluster.web/ReplicationWebFailoverTestCase%28UDP%29/testGracefulUndeployFailover/
> {noformat}
> 23:00:59,836 WARN  [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (OOB-20,null) Problems unmarshalling remote command from byte buffer: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 50
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:750) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:99) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:935) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1205) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
> 	at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:119)
> 	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:162)
> 	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:66)
> 	at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:395)
> 	at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224)
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
> 	at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163)
> 	at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:114)
> 	at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:104)
> 	at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)
> 	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:200)
> 	at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:465)
> 	at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:372)
> 	at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
> 	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:601)
> 	at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
> 	at org.jgroups.JChannel.up(JChannel.java:715)
> 	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
> 	at org.jgroups.protocols.RSVP.up(RSVP.java:179)
> 	at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
> 	at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
> 	at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
> 	at org.jgroups.protocols.pbcast.GMS.up(GMS.java:899)
> 	at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)
> 	at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:744)
> 	at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:414)
> 	at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:572)
> 	at org.jgroups.protocols.BARRIER.up(BARRIER.java:102)
> 	at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
> 	at org.jgroups.protocols.FD.up(FD.java:273)
> 	at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
> 	at org.jgroups.protocols.MERGE2.up(MERGE2.java:205)
> 	at org.jgroups.protocols.Discovery.up(Discovery.java:359)
> 	at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2568)
> 	at org.jgroups.protocols.TP.passMessageUp(TP.java:1211)
> 	at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1775)
> 	at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1748)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_29]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_29]
> 	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_29]
> [0m[31m23:00:59,943 ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-3,node-0/web) ISPN000172: Failed to prepare view CacheView{viewId=4, members=[node-1/web, node-0/web]} for cache  default-host/distributable, rolling back to view CacheView{viewId=3, members=[node-1/web]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: Problems invoking command.
> 	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) [rt.jar:1.6.0_29]
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_29]
> 	at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:319) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:250) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:876) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [rt.jar:1.6.0_29]
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_29]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_29]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_29]
> 	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_29]
> Caused by: org.infinispan.CacheException: Problems invoking command.
> 	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:215) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:465) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:372) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:601) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.JChannel.up(JChannel.java:715) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.RSVP.up(RSVP.java:179) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.FRAG2.up(FRAG2.java:181) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.FlowControl.up(FlowControl.java:418) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.FlowControl.up(FlowControl.java:400) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.pbcast.GMS.up(GMS.java:899) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:744) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:414) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:572) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.BARRIER.up(BARRIER.java:102) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.FD.up(FD.java:273) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.Discovery.up(Discovery.java:359) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2568) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.TP.passMessageUp(TP.java:1211) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1775) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1748) [jgroups-3.1.0.Final.jar:3.1.0.Final]
> 	... 3 more
> Caused by: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 50
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:750) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:99) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:935) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1205) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
> 	at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:119) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:162) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:66) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:395) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
> 	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
> 	at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:114) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:104) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:200) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
> 	... 29 more
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

More information about the jboss-jira mailing list