[
https://issues.jboss.org/browse/AS7-5393?page=com.atlassian.jira.plugin.s...
]
Doug Mayer commented on AS7-5393:
---------------------------------
Having tried removing start='EAGER' with only temporary relief, I can confirm that
deploying a JBoss app to a 2- or 3-node cluster in quick succession will cause this error
to occur and not be able to recover on its own.
We deploy to machines serially, waiting until the .dodeploy is cleaned up before moving
onto the next server. When we pause for ~10 seconds AFTER dodeploy is gone before moving
onto the next server, we only see the error once (if at all). The same can be seen when
restarting JBoss (TorqueBox) servers - they log this error indefinitely when restarted too
quickly, and are generally healthy when giving them some space between restarts.
This almost always comes after a "ISPN000172: Failed to prepare view CacheView"
error for us.
"Problems unmarshalling remote command from byte buffer:
java.io.StreamCorruptedException: Unexpected byte found when reading an object: 50"
--------------------------------------------------------------------------------------------------------------------------------------------
Key: AS7-5393
URL:
https://issues.jboss.org/browse/AS7-5393
Project: Application Server 7
Issue Type: Bug
Components: Clustering
Affects Versions: 7.1.3.Final (EAP)
Environment: affected: AS7 master
Reporter: Radoslav Husar
Assignee: Paul Ferraro
Fix For: 7.2.0.CR1, 7.1.4.Final (EAP)
As noticed by Kabir in AS7 master TS runs upon failover:
http://lightning.mw.lab.eng.bos.redhat.com/jenkins/job/as7-master-testsui...
{noformat}
23:00:59,836 WARN [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher]
(OOB-20,null) Problems unmarshalling remote command from byte buffer:
java.io.StreamCorruptedException: Unexpected byte found when reading an object: 50
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:750)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
[jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:99)
[jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:935)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1205)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
[jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
at
org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:119)
at
org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:162)
at
org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:66)
at
org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:395)
at
org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224)
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
[jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
at
org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163)
at
org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:114)
at
org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:104)
at
org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)
at
org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:200)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:465)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:372)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:601)
at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
at org.jgroups.JChannel.up(JChannel.java:715)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
at org.jgroups.protocols.RSVP.up(RSVP.java:179)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:899)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)
at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:744)
at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:414)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:572)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:102)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD.up(FD.java:273)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:205)
at org.jgroups.protocols.Discovery.up(Discovery.java:359)
at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2568)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1211)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1775)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1748)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[rt.jar:1.6.0_29]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[rt.jar:1.6.0_29]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_29]
[0m[31m23:00:59,943 ERROR
[org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-3,node-0/web)
ISPN000172: Failed to prepare view CacheView{viewId=4, members=[node-1/web, node-0/web]}
for cache default-host/distributable, rolling back to view CacheView{viewId=3,
members=[node-1/web]}: java.util.concurrent.ExecutionException:
org.infinispan.CacheException: Problems invoking command.
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) [rt.jar:1.6.0_29]
at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_29]
at
org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:319)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:250)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:876)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [rt.jar:1.6.0_29]
at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_29]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[rt.jar:1.6.0_29]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[rt.jar:1.6.0_29]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_29]
Caused by: org.infinispan.CacheException: Problems invoking command.
at
org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:215)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:465)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:372)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:601)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.JChannel.up(JChannel.java:715) [jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.RSVP.up(RSVP.java:179) [jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.FRAG2.up(FRAG2.java:181) [jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:899)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:744)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:414)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:572)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.BARRIER.up(BARRIER.java:102)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.FD.up(FD.java:273) [jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.MERGE2.up(MERGE2.java:205)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.Discovery.up(Discovery.java:359)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2568)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.TP.passMessageUp(TP.java:1211)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1775)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1748)
[jgroups-3.1.0.Final.jar:3.1.0.Final]
... 3 more
Caused by: java.io.StreamCorruptedException: Unexpected byte found when reading an
object: 50
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:750)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
[jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:99)
[jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:935)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1205)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
[jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
at
org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:119)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:162)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:66)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:395)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at
org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
[jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
[jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
at
org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:114)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:104)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
at
org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:200)
[infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
... 29 more
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:
http://www.atlassian.com/software/jira