[jboss-user] [Clustering] - Re: Farm deployment errors with large WAR files

[bstansberry@jboss.com]

I've commented on the JIRA, but will go into a bit more detail here.

First, the logs you sent me, Adam, showed the file that was being farmed was actually "xxx.ear.filepart" not "xxx.ear". From that I believe you were using a tool (perhaps WinSCP?) to upload the file directly to the farm directory. That's a lengthy enough operation that the hot deployment scanner ran in the middle.

Hence my recommendation on the JIRA to stop the hot deployment scanner thread before doing lengthy I/O operations.

The scanner can be stopped by invoking the JMX stop() operation on the jboss.deployment:flavor=URL,type=DeploymentScanner MBean. This can be done via the jmx-console or via the twiddle utility in $JBOSS_HOME/bin.

I also you recommend you do your upload to a temp folder and then do a local copy into farm/. Otherwise if there is a failure during the upload you'll be leaving a corrupt file in farm/.

Second, looking closely at the logs tells me that a very high percentage of messages JGroups is sending from the master are not being received on the child. This makes things go very slowly. You need to determine why there is such a high rate of loss on your network. Earlier in the thread we've discussed the OS maximum read buffer setting. You can also adjust the maximum write buffer. Note also that changes made via sysctl -w are not persistent across restarts; for persistent changes you need to edit /etc/sysctl.conf.

If you can't eliminate the UDP packet losses, you might consider using a TCP stack, particularly if yours is a 2 node cluster, where there is no benefit from using IP multicast. If you can use IP multicast for the initial cluster discovery messages (which seems to be working fine) then it's very easy to configure the AS to use TCP for regular traffic, just add this to your command line arguments:

-Djboss.default.jgroups.stack=tcp

(Switching to TCP is a bit more complicated if IP multicast for discovery isn't an option.)

View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4258506#4258506

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4258506