Change in locking behavior in MainDeployer [WAS: Port conflict in tx manager - cluster-udp-SYNC-0
by Brian Stansberry
The reason the cluster-udp-SYNC-0 server is failing to start is because
a server (cluster-udp-1) from the previous test set is not shutting down
because threads are deadlocking. See attached thread dump.
What's changed here is what used to be a read lock acquisition in
MainDeploymentImpl.process() has changed into a write lock acquisition
in a jboss-deployers snapshot deployed a couple days ago. r84308 [1]
The root of the problem is thread AsynchKeyChangeHandler is
asynchronously responding to a notification of an undeploy on the other
node by 1) synchronizing on an HASingletonImpl and 2) calling into
MainDeployerImpl.addDeployment(). The addDeployment call needs to
acquire a read lock on a ReentrantReadWriteLock.
Meanwhile the test driver has moved on and is trying to undeploy the
HASingletonImpl via an RMI call. The RMI call has called
MainDeployerImpl.process() and acquired a write lock on the
ReentrantReadWriteLock. It then needs to synchronize on the HASingletonImpl.
Deadlock.
Does this lock need to be a write lock?
[1]
http://fisheye.jboss.org/browse/JBossAS/projects/jboss-deployers/trunk/de...
Brian Stansberry wrote:
> The servers from the previous set of tests are failing to shut down;
> I'll take a look why:
>
> [server:stop] Shutting down server: cluster-udp-0
> [server:stop] shutdownTimeout will be=45
> [server:stop] Writing server thread dump to
> /mnt/hudson_workspace/workspace/JBoss-AS-5.0.x-testSuite-sun15-sun16/JBossAS_5_0/build/output/jboss-5.0.1.GA/server/cluster-udp-0/log/threadDump.log
>
> [server:stop] Failed to shutdown server "cluster-udp-0" before timeout.
> Destroying the process.
> [server:stop] Unable to shutdown server properly:
> org.jboss.jbossas.servermanager.ServerShutdownException: Failed to
> shutdown server before timeout.Process was destroyed.
> [server:stop] Shutting down server: cluster-udp-1
> [server:stop] shutdownTimeout will be=45
> [server:stop] Writing server thread dump to
> /mnt/hudson_workspace/workspace/JBoss-AS-5.0.x-testSuite-sun15-sun16/JBossAS_5_0/build/output/jboss-5.0.1.GA/server/cluster-udp-1/log/threadDump.log
>
> [server:stop] Failed to shutdown server "cluster-udp-1" before timeout.
> Destroying the process.
> [server:stop] Unable to shutdown server properly:
> org.jboss.jbossas.servermanager.ServerShutdownException: Failed to
> shutdown server before timeout.Process was destroyed.configuration
> (SYNC, no BR)
>
> Dimitris Andreadis wrote:
>> For some reason the cluster-udp-SYNC-0 test config in the
>> JBoss-AS-5.0.x-testSuite-sun15-sun16 run shows a port conflict (don't
>> appear up in the other runs thought, eg. testSuite-sun16):
>>
>> http://hudson.qa.jboss.com/hudson/view/JBoss%20AS/job/JBoss-AS-5.0.x-test...
>>
>>
>> http://hudson.qa.jboss.com/hudson/view/JBoss%20AS/job/JBoss-AS-5.0.x-test...
>>
>>
>> Some server failed to start.
>> 06:01:19,991 INFO [TransactionManagerService] JBossTS Transaction
>> Service (JTA version) - JBoss Inc.
>> 06:01:19,992 INFO [TransactionManagerService] Setting up property
>> manager MBean and JMX layer
>> 06:01:20,314 WARN [arjLoggerI18N]
>> [com.arjuna.ats.arjuna.recovery.TransactionStatusManager_14] - Failed
>> to create server socket on address 10.16.93.89 and port: 4,713
>> 06:01:20,316 ERROR [AbstractKernelController] Error installing to
>> Create: name=TransactionManager state=Configured
>> com.arjuna.ats.arjuna.exceptions.FatalError:
>> [com.arjuna.ats.arjuna.recovery.TransactionStatusManager_9] - Could
>> not get unique port.
>> at
>> com.arjuna.ats.arjuna.recovery.TransactionStatusManager.start(TransactionStatusManager.java:185)
>>
>> at
>> com.arjuna.ats.arjuna.recovery.TransactionStatusManager.<init>(TransactionStatusManager.java:72)
>>
>> at
>> com.arjuna.ats.arjuna.coordinator.TxControl.<clinit>(TxControl.java:355)
>> at
>> com.arjuna.ats.jbossatx.jta.TransactionManagerService.create(TransactionManagerService.java:178)
>>
>>
>
>
--
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
brian.stansberry(a)redhat.com
16 years, 3 months
Hack for Continuous Integration in AS
by Andrew Lee Rubinger
Had a thought regarding the lack of testability for non-released
components in AS.
The chicken-egg dilemna is:
* We can't validate a release until it passes AS and TCK TestSuites
* We can't integrate into AS until a component is released
So we work around this by releasing, crossing our fingers, and
re-releasing if a problem pops up.
Some hack idea is:
* Create "component-matrix-continuous" with parent of "component-matrix"
* c-m-c may define versioned SNAPSHOTs in the depMgt section, overriding
those defined by c-m.
* Make a property to set the artifactId of the parent of thirdparty POM
to the continuous build:
> ./build.sh -Dcontinuous=true
...then we can set up Hudson runs to test versioned SNAPSHOTs to catch
errors that may be coming down the line without affecting the normal build.
WWJD?
S,
ALR
--
Andrew Lee Rubinger
Sr. Software Engineer
JBoss, a division of Red Hat, Inc.
http://exitcondition.alrubinger.com
16 years, 3 months
JBossWeb as foundation for OSGi HttpService
by Thomas Diesler
Hi Folks,
I trying to optimize the JBoss OSGi HttpService integration with
JBossWeb. Currently this generates JBossWebMetaData in memory and
deploys that through the MainDeployer.
What I am looking for however is an embedded Http server that I can use
in the OSGi service. Most other HttpService implementations use Jetty
for that reason.
Is it possible to use JBossWeb embedded and setup Http endpoints through
an API?
cheers
-thomas
16 years, 3 months
JBoss AS build remoting and jmx-remoting
by Paul Gier
In the app server build in trunk, I'd like to change the modules
jbossas/remoting and jbossas/jmx-remoting so that they follow the same flat
directory structure as the other modules.
So instead of the nested directories jbossas/remoting and jbossas/jmx-remoting,
these would be moved to jbossas-remoting and jbossas-jmx-remoting. Is there any
reason why I shouldn't change this?
Thanks!
16 years, 3 months
VM arguments finally coming to boot.log
by Galder Zamarreno
Hi,
I've just implemented https://jira.jboss.org/jira/browse/JBAS-5240 using
java.lang.management.RuntimeMXBean.getInputArguments();
Example:
13:44:43,226 INFO [ServerInfo] VM arguments: -Dprogram.name=run.sh
-Xms128m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true
-Dsun.rmi.dgc.client.gcInterval=3600000
-Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.net.preferIPv4Stack=true
-Djava.endorsed.dirs=/home/galder/jboss/code/as/trunk/build/output/jboss-6.0.0.Alpha1/lib/endorsed
This is a much better solution than the one originally suggested in the
jira. This is also a much needed information from a support point of
view and the fact that from now on it's logged in boot.log reduces the
pieces of information we need to get from customer (before we needed
either console output to see these options or for the customer to spell
it out for us).
Hence, the following line in all of our run scripts should be removed as
it's obsolete:
echo " JAVA_OPTS: $JAVA_OPTS"
Thoughts?
--
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat
16 years, 4 months
Re: [jboss-dev] How to fix vfs?
by Shelly McGowan
Ales,
The packaging of this .ear is testing the following assertion defined in the Persistence Specification:
One or more jar files may be specified using the jar-file elements instead of, or in addition to the mapping files specified in the mapping-files elements. If specified, these JAR files will be searched for managed persistence classes and any mapping metadata annotations found on them will be processed or they will be mapped using the mapping annotation defaults defined by this specification. **Such JAR files are specified relative to the root of the persistence unit.**
persistence.xml:
<jar-file>lib/common.jar</jar-file>
The persistence unit is at the root of the .ear.
Shelly
----- Original Message -----
From: "Ales Justin" <ales.justin(a)gmail.com>
To: "JBoss.org development list" <jboss-development(a)lists.jboss.org>
Sent: Tuesday, February 17, 2009 11:56:36 AM GMT -05:00 US/Canada Eastern
Subject: Re: [jboss-dev] How to fix vfs?
>> This change
>> http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbossas/projects/vfs/trunk/src...
>> and its current usage within the VFS make all that for nothing.
>>
>
> It was easy to figure out the problem was due to broken vfs paths
> by looking at the error message and then working back through the
> stacktrace:
>
> java.io.IOException: Child not found /lib/common.jar/
> <snip/>
> at org.jboss.virtual.plugins.registry.DefaultVFSRegistry.findHandler(DefaultVFSRegistry.java:107)
> at
> org.jboss.virtual.plugins.registry.DefaultVFSRegistry.getFile(DefaultVFSRegistry.java:81)
> at
> org.jboss.virtual.plugins.registry.DefaultVFSRegistry.getFile(DefaultVFSRegistry.java:119)
>
> You can even see that adding the / wouldn't be valid if that path was
> the URI/URL because from the previous log statement, it is not unpacked
> and therefore not a directory.
>
> "
> Searching mapped entities in jar/par:
> vfszip:snipped/ejb3_pkg_scope.jar/lib/common.jar
> "
I don't think this is the problem.
"vfszip:snipped/ejb3_pkg_scope.jar/lib/common.jar" is broken in the
first place.
lib/common.jar is in .ear not .jar.
_______________________________________________
jboss-development mailing list
jboss-development(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-development
16 years, 4 months
Re: [jboss-dev] AS 5.1 and JBoss Messaging 2
by Tim Fox
Resending
Tim Fox wrote:
> Jason T. Greene wrote:
>> I really wish we could. Tim said theres pretty much no was JBM2 can
>> make Beta1, but it could make CR1. The question is, would it be
>> stable and TCK compliant enough that it doesn't end up delaying GA?
> No, JBM 2.0 is a major release and is not compatible with JBM 1.4.
> That's why we can't replace JBM 1.4 in a minor release since AS minor
> versions need to be compatible, and why it's being offered as a
> technology preview, not replacement for default JMS provider.
>>
>> Dimitris Andreadis wrote:
>>> Why not keep it simple and ship JBM2 in AS 5.1 Beta/CR? I suppose
>>> it's a matter of dates, right?
>>>
>>> Jason T. Greene wrote:
>>>> Because that is just configuration. Shipping just 1.4 in the
>>>> community, and 1.4 and 2.0 in EAP is completely backwards from the
>>>> whole upstream concept.
>>>>
>>>> David Ward wrote:
>>>>> Why would this be weird? .com/EAP 4.x has a "production" profile
>>>>> that .org/AS 4.x does not...
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>> Jason T. Greene wrote:
>>>>>> Part of the problem is that EAP is planning on including a JBM2
>>>>>> profile / option. IMO it would be very weird to ship something in
>>>>>> EAP but not in AS. Besides creating a profile, another option we
>>>>>> have is to include JBM2 with an install script in docs/examples.
>>>>>> Then someone could switch any AS profile to JBM2.
>>>>>>
>>>>>> Dimitris Andreadis wrote:
>>>>>>> We have enough configurations so far, so I don't think it's a
>>>>>>> good idea adding any new ones until we come up with with a way
>>>>>>> to share most of the jars/configuration.
>>>>>>>
>>>>>>> In your case it's better to offer the preview as a separate
>>>>>>> download.
>>>>>>>
>>>>>>> Andy Taylor wrote:
>>>>>>>> As part of the AS 5.1 release we will be shipping JBM 2.0 as a
>>>>>>>> technology preview. This will require 1 or more new profiles to
>>>>>>>> be created. I have created a branch (Branch_5_x_JBM2) to use
>>>>>>>> for now which i will merge in once JBM 2.0 is released and
>>>>>>>> there is something concrete and working.
>>>>>>>>
>>>>>>>> What new profiles we need to add is open to discussion. We'll
>>>>>>>> definitely need a copy of default as a start which could be
>>>>>>>> called messaging-2. However, from a JBM2 point of view, it
>>>>>>>> would be good to also demonstrate clustering which would mean a
>>>>>>>> copy of all, say messaging-2-all, and since JBM2 can run with a
>>>>>>>> minimal set of services maybe a messaging-2-minimal as well.
>>>>>>>>
>>>>>>>> I appreciate that adding all 3 might be too many new profiles
>>>>>>>> to add so I'd like to hear peoples views!
>>>>>>>>
>>>>>>>> Andy Taylor Core Developer
>>>>>>>> JBoss Messaging
>>>>>>>> _______________________________________________
>>>>>>>> jboss-development mailing list
>>>>>>>> jboss-development(a)lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/jboss-development
>>>>>>> _______________________________________________
>>>>>>> jboss-development mailing list
>>>>>>> jboss-development(a)lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/jboss-development
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> jboss-development mailing list
>>>>> jboss-development(a)lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-development
>>>>
>>>>
>>
>>
>
>
--
Tim Fox
Messaging Lead
JBoss - a division of Red Hat
http://jbossfox.blogspot.com/
irc.freenode.net#jbossmessaging
fox(a)redhat.com
16 years, 4 months
JBAS-6518 - Writing a test
by Adrian Brock
I'm trying to write a test that reproduces
https://jira.jboss.org/jira/browse/JBAS-6518
I've got such a test:
./build.sh one-test
-Dtest=org.jboss.test.deployers.spaces.test.SpacesUnitTestCase
But it doesn't work since this test framework uses the
profile service which doesn't seem to like the spaces
in the deployment either.
For that reason, I've only committed the test to trunk for now.
Emanuel, I think this has something to do with what I mentioned
about not using the "short name" to identify a deployment,
but I tried using that to get the ManagementView, it still didn't work.
I did get some bizarre warning (probably unrelated?):
20:46:33,948 WARN [ManagementViewImpl] Failed to merged the following
runtime ManagedObjects:
{jboss.system:type=ServerInfo/jboss.system:type=ServerInfo=ManagedObject{jboss.system:type=ServerInfo}}
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
16 years, 4 months
Re: [jboss-dev] AS 5.1 and JBoss Messaging 2
by Tim Fox
Resending
Tim Fox wrote:
> Andy Taylor wrote:
>>
>>
>> Jason T. Greene wrote:
>>> Well, as an example, we have multiple server dirs, but only one
>>> client dir, and everything in there is part of the same classloader.
>>> So, will an app writen against the 1.4 API work with the 2.0 API jar?
>> Applications written using pure JMS should work as the Connection
>> Factories in JBM 2.0 have different package names.
>>> Is there any client specific code in 1.4 and 2.0 that shares the
>>> same packages (if so that will be a conflict)?
>> I don't think there is but we can easily repackage any clashes.
>>>
>>> Is the MDB implementation in EJB3 compatible with both 1.4 and 2.0?
>> I guess this is something we need to test but as far as i understood
>> it would just use the new JMS RA implementation that is currently
>> being written.
>>>
>>> Do the dependency sets conflict between 1.4 and 2.0 (i.e. different
>>> jgroups requirements etc)?
>> JBM2 no longer uses JGroups or Remoting which may have caused
>> problems. We would make sure that JBM2 uses the appropriate versions
>> of any other conflicts.
> We don't use anything else right now, so there is zero chance of
> conflicts.
>
> However, we'll re-introduce JGroups dependency at a later date, but
> we'll make sure we use the same one as AS.
>>>
>>> Is the JMX API in 2.0 BC with 1.4?
>>>
>>> Andy Taylor wrote:
>>>> Jesper has already developed JCA resource adapter for JBM 2.0, Is
>>>> there anything apart from that we would need?
>>>>
>>>> Jason T. Greene wrote:
>>>>> Yeah thats a good point. Andy how are you guys planning on
>>>>> handling AS components which are developed agains the JBM 1.4 API
>>>>> being compatible with the JBM2 API. Were you going to add some
>>>>> kind of compatibility layer?
>>>>>
>>>>> Dimitris Andreadis wrote:
>>>>>> This is an other option, however, it has other implications in
>>>>>> how this get included. You'll have to import 2 different version
>>>>>> of the same component (and possibly other dependencies).
>>>>>>
>>>>>> Not sure how our existing build system would support this.
>>>>>>
>>>>>> Jason T. Greene wrote:
>>>>>>> Part of the problem is that EAP is planning on including a JBM2
>>>>>>> profile / option. IMO it would be very weird to ship something
>>>>>>> in EAP but not in AS. Besides creating a profile, another option
>>>>>>> we have is to include JBM2 with an install script in
>>>>>>> docs/examples. Then someone could switch any AS profile to JBM2.
>>>>>>>
>>>>>>> Dimitris Andreadis wrote:
>>>>>>>> We have enough configurations so far, so I don't think it's a
>>>>>>>> good idea adding any new ones until we come up with with a way
>>>>>>>> to share most of the jars/configuration.
>>>>>>>>
>>>>>>>> In your case it's better to offer the preview as a separate
>>>>>>>> download.
>>>>>>>>
>>>>>>>> Andy Taylor wrote:
>>>>>>>>> As part of the AS 5.1 release we will be shipping JBM 2.0 as a
>>>>>>>>> technology preview. This will require 1 or more new profiles
>>>>>>>>> to be created. I have created a branch (Branch_5_x_JBM2) to
>>>>>>>>> use for now which i will merge in once JBM 2.0 is released and
>>>>>>>>> there is something concrete and working.
>>>>>>>>>
>>>>>>>>> What new profiles we need to add is open to discussion. We'll
>>>>>>>>> definitely need a copy of default as a start which could be
>>>>>>>>> called messaging-2. However, from a JBM2 point of view, it
>>>>>>>>> would be good to also demonstrate clustering which would mean
>>>>>>>>> a copy of all, say messaging-2-all, and since JBM2 can run
>>>>>>>>> with a minimal set of services maybe a messaging-2-minimal as
>>>>>>>>> well.
>>>>>>>>>
>>>>>>>>> I appreciate that adding all 3 might be too many new profiles
>>>>>>>>> to add so I'd like to hear peoples views!
>>>>>>>>>
>>>>>>>>> Andy Taylor Core Developer
>>>>>>>>> JBoss Messaging
>>>>>>>>> _______________________________________________
>>>>>>>>> jboss-development mailing list
>>>>>>>>> jboss-development(a)lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/jboss-development
>>>>>>>> _______________________________________________
>>>>>>>> jboss-development mailing list
>>>>>>>> jboss-development(a)lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/jboss-development
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> jboss-development mailing list
>>>> jboss-development(a)lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/jboss-development
>>>
>>>
>> _______________________________________________
>> jboss-development mailing list
>> jboss-development(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jboss-development
>
>
--
Tim Fox
Messaging Lead
JBoss - a division of Red Hat
http://jbossfox.blogspot.com/
irc.freenode.net#jbossmessaging
fox(a)redhat.com
16 years, 4 months
