Re: [jboss-dev] Remoting behavior on error (Re: Possible blocker with jboss-cl 2.2.0.Alpha1)
The testcase has been hung since more than 3 days, now :) So looks like
that default client timeout of 30 minutes isn't being honoured. By the
way, EJB3 uses its own config file for remoting JBOSS_HOME/server/<
servername>/deploy/ejb3-connectors-jboss-beans.xml. But that's
irrelevant since the defaults should apply to any remoting config
irrespective of which file it's configured in.
So I looked at this in a bit more detail and it now appears to be a bug
in JBoss Remoting. Here's what's happening:
- A EJB3 client interceptor
(org.jboss.aspects.remoting.InvokeRemoteInterceptor) responsible for
invoking the remote EJB container does a remoting call:
InvokerLocator locator =
(InvokerLocator)invocation.getMetaData(REMOTING, INVOKER_LOCATOR);
...
Client client = new Client(locator, subsystem);
try
{
client.connect();
...
client.invoke(invocation, null);
- Remoting internally creates a
org.jboss.remoting.transport.socket.MicroSocketClientInvoker which
creates a ServerAddress:
protected ServerAddress createServerAddress(InetAddress addr, int port)
{
return new ServerAddress(addr.getHostAddress(), port,
enableTcpNoDelay, -1, maxPoolSize);
}
Notice the hardcoded -1 being passed to the ServerAddress. That's the
"timeout". The ServerAddress internally ignores negative timeout values
and sets the timeout to 0 (resulting in infinite timeout).
- The MicroSocketClientInvoker then ultimately ends up creating a client
socket with timeout = 0.
- So when the server side remoting thread crashes (for any unrelated
reason - like the classloading issue), the client ends up with this
infinite block.
The issues i see here:
1) I don't see a way, how i can pass a timeout for the client socket.
Contrary to the comments in remoting-jboss-beans.xml, based on what i
see in the code, it's only the server side timeout which has a
corresponding param. Did i miss something?
2) Even if there was some way of passing the client socket timeout,
currently the ServerAddress creation through MicroSocketClientInvoker
hardcodes the timeout to -1.
regards,
-Jaikiran
Adrian Brock wrote:
On Tue, 2010-02-16 at 18:14 +0530, Jaikiran Pai wrote:
> Ron Sigal wrote:
>> Yeah, Carlo also pointed out that NCDFE is a java.lang.Error, which,
>> as you note, is not caught. I agree that if ServerThread is going to
>> terminate due to an error, then it should certainly close its socket,
>> so that's a bug (JBREM-1183 "ServerThread should catch
>> java.lang.Error"). But I have mixed feelings about terminating
>> ServerThread in this case. Unlike, IOException, SocketException,
>> etc., the NoClassDefFoundError doesn't suggest that the socket is no
>> longer usable. Maybe a better solution (along with fixing the
>> classloader problem, of course) is to use a positive timeout on the
>> client side.
> I think some (sensible) default timeout needs to be added so that the
> client does not end up hung. Is this something that can be added to some
> *-jboss-beans.xml in the AS?
>
It already exists, but whether 30 minutes is a "sensible" default
is a different issue. ;-)
See remoting-jboss-beans.xml
<!-- Socket read timeout. Defaults to 60000 ms (1 minute)
-->
<!-- on the server, 1800000 ms (30 minutes) on the client.
-->
<!--entry><key>timeout</key>
<value>120000</value></entry-->