[jboss-dev] Possible blocker with jboss-cl 2.2.0.Alpha1 [WAS]: Hudson Jobs Hanging

[Carlo de Wolf]

This should have nothing to do at all with jboss-cl. In essence it's the 
following scenario that fails:

- request comes in
- undeploy request comes in
- container blocks undeploy
- response goes out the container
- container unblocks and undeploys
- response tries to get out of the invoker => poof

Carlo

On 02/06/2010 06:40 PM, Brian Stansberry wrote:
> Thanks, Jaikiran. I'm forwarding this to the jboss-dev list for 
> greater visibility.
>
> The EJB3 team is seeing consistent failures where a server thread is 
> unable to load classes from rt.jar. These occur in a test where a 
> remote client makes requests against a service that is concurrently 
> being undeployed. Failure is quite similar to what is seen at 
> https://jira.jboss.org/jira/browse/JBAS-7688 and previously mentioned 
> on this list.[1]
>
> I've elevated JBAS-7688 to Blocker; we'll have to get comfortable 
> early next week that this isn't due to the jboss-cl 2.2.0.Alpha1 
> upgrade, fix jboss-cl 2.2.0, or roll back AS trunk to 2.0.8.GA.
>
> [1] 
> http://lists.jboss.org/pipermail/jboss-development/2010-February/015693.html 
>
>
> On 02/06/2010 09:24 AM, Jaikiran Pai wrote:
>> I switched one of the jobs (which i have been using for testing) on Mike
>>   to 2.0.8.GA of jbcl and haven't seen this hang in those. However, the
>> other job which is using 2.2.0.Alpha1 is consistently hanging with the
>> same exception being thrown. I have tried to reproduce this locally, but
>> haven't been able to do so (i.e. the test runs fine with 2.2.0.Alpha1 on
>> my local setup).
>>
>> The exception stacktrace (as Carlo posted earlier) seems to be
>> originating from a remoting thread which is doing some marshalling and
>> running into CNFE (all these CNFE classes so far have been the ones
>> residing in rt.jar). I took a thread dump of the server, at the time
>> when Mike was hung during this testcase - couldn't find anything useful
>> in there. Perhaps, i should be taking a thread dump of the JVM on the
>> client (the JUnit testcase), to see what it is waiting on. At the same
>> time, David mentions[1] that:
>>
>> "I hypothesize that the classloader is (sometimes) being destroyed
>> before the thread gets going, causing it to be unable to load anything
>> at all."
>>
>> I'll see if i can figure out:
>>
>> 1) Whether that statement applies to this testcase too
>> 2) Why this happens specifically in this testcase
>>
>>
>>
>> [1]
>> http://lists.jboss.org/pipermail/jboss-development/2010-February/015694.html 
>>
>>
>>
>> regards,
>> -Jaikiran
>> Brian Stansberry wrote:
>>> On 02/04/2010 12:18 PM, Jaikiran Pai wrote:
>>>> Mike is running into this again
>>>> http://mike.lab.bos.redhat.com:8380/hudson/job/EJB3_1-1.0.3-JBoss-AS-6.x-tests-ejb3/3/console 
>>>>
>>>>
>>>>
>>>>
>>>> The https://jira.jboss.org/jira/browse/EJBTHREE-1982 has linked
>>>> JBKERNEL-86 as one of the problems. JBKERNEL-86 is marked as fixed in
>>>> 2.2.0.alpha-4 of MC kernel. The current AS trunk has 2.2.0-alpha-5. 
>>>> But
>>>> we are still seeing the same hang. So i guess there's more to it.
>>>> Looking at the logs (attached) i don't see an explicit OOM, but i 
>>>> do see
>>>> signs of it (like CNFE on a com.sun.* class after numerous deploys).
>>>
>>> On the face of it that sounds like a different problem. Occam's razor
>>> say's it's not, but a CNFE seems like a different beast than a memory
>>> leak.
>>>
>>> Possibly the CNFE is due to the jboss-cl upgrade to 2.2.0.Alpha1? See
>>> https://jira.jboss.org/jira/browse/JBAS-7688 for another issue where
>>> after that change classes in rt.jar weren't available.
>>>
>>> The AS should run fine w/ it's jboss-cl moved back to 2.0.8.GA, so
>>> this theory is testable.
>>
>
>