[jboss-as7-dev] NPE in POST_MODULE processors
Thomas Diesler
thomas.diesler at jboss.com
Thu Aug 16 11:42:39 EDT 2012
Regarding ...
/This sounds very non-deterministic. Just to clarify, are you saying
that if the user has a complex bundle deployment with lots of
inter-dependencies on startup some may be resolved and some won't,
and this may change on subsequent startups depending on the order in
which they start?/
With a complex set of bundle deployments the user will have to deploy
them in a known order (which is a problem in itself). There is pull
request #2790 <https://github.com/jbossas/jboss-as/pull/2790> waiting
that will allow the management client to have control over the auto
start behaviour. So a user could first install the complete set in
multiple operations and later explicitly start a selected set of
bundles. This would overcome the order issue on first deploy.
Once the bundles are installed and activated the framework records their
respective state. On server restart these persistent bundles are
deployed in an arbitrary order but there is a guarantee backed into the
Framework integration layer that ensures that the first resolve attempt
is made after all persistent bundles have been installed. From the
resolve perspective order also matters - you might get different wiring
results depending on the order you resolve the bundles. One possible
approach might be to resolve the full set of persistent bundles at once,
but the guarantee for an identical wiring is still weak. A better
approach would be to always resolve in a known order (i.e. sort by
bundle id). The still better solution would be to persist the last known
wiring graph and restore that on startup. Currently, the persistent
bundles are resolved in the order they hit the BundleResolveProcessor
which is arbitrary AFAIK.
I have written up the complete subsystem activation process in this
article
<https://community.jboss.org/wiki/OSGiSubsystemActivationProcess>. It
contains the known issues and ideas for possible solutions. Perhaps we
can start from there to find a more consistent solution.
cheers
--thomas
On 08/15/2012 01:32 PM, Thomas Diesler wrote:
>
> On 08/15/2012 11:20 AM, Stuart Douglas wrote:
>>
>> On 15/08/2012, at 6:59 PM, Thomas Diesler <thomas.diesler at jboss.com
>> <mailto:thomas.diesler at jboss.com>> wrote:
>>
>>> > Why would the OSGI bundle not be able to resolve, is it because is
>>> waiting for another OSGI bundle to be installed?
>>>
>>> This is by virtue of the API - BundleContext.install() does not
>>> resolve the bundle. As the method name suggests, it just installs
>>> the bundle.
>>>
>>> In the hot-deployment case it is debatable whether bundle resolution
>>> and later bundle activation should be attempted or not. By design,
>>> the order of bundle deployment is not the responsibility of the user
>>> but that of the framework. For a complex graph of interdependent
>>> bundles the user cannot possibly be asked to deploy them in the
>>> "right order". Instead the API allows to INSTALL the complete set
>>> (i.e. make the metadata available to the resolver) and later
>>> activate the bundles as needed. There are other triggers for bundle
>>> resolution too (e.g. resource access)
>>>
>>> We currently do resolve/activate during DUP processing on a trial
>>> basis. For a bundle that only has dedependencies on already
>>> installed bundles the resolve/activation works fine and the services
>>> become available. I guess this is the expected hot-deploy behaviour.
>>> A bundle that cannot resolve - for various reasons, one being the
>>> user says so - we dont attempt to start the bundle either. It would
>>> still run through all remaining DUPs but does not have a module
>>> attached.
>>
>> This sounds very non-deterministic. Just to clarify, are you saying
>> that if the user has a complex bundle deployment with lots of
>> inter-dependencies on startup some may be resolved and some won't,
>> and this may change on subsequent startups depending on the order in
>> which they start?
> Yes, this is a long outstanding issue [AS7-378
> <https://issues.jboss.org/browse/AS7-378>]. I still have no guarantee
> that all bundles in a given set have been INSTALLED (in OSGi
> terminology) / have completed the Phase.REGISTER phase (in AS7
> terminology) when the one bundle hits the BundleResolveProcessor. The
> framework records the persistent bundle state and on restart it is a
> requirement that all persistent bundles reach their respective target
> state for successful framework initialization. There is a little more
> detail to it and I'd be more than happy to work with you to find a
> consistent solution. We can take up this topic in another osgi
> specific thread if you like.
>>
>>>
>>> Non-OSGi deployments that use jboss-modules metadata to define their
>>> dependencies (i.e. Dependencies clause in the manifest) have that
>>> problem too, but worse. A complex system of interdependent module
>>> deployments is likely not manageable because of this ordering issue.
>>> Even if the user gets the ordering right the first time, on server
>>> restart the notion of deployment order is lost and very likely
>>> initial deployments will fail with no osgi involved. Granted that
>>> this describes a use case that is not intended to be used for user
>>> deployments.
>>
>> No, JBoss modules uses MSC services to resolve the dependencies. At
>> container start all deployments are now run as part of the boot ops,
>> so as long as all deployments are present this will always work. We
>> do need a more specified way of saying "Don't start this deployment
>> until another deployment is done", but this is mainly for things like
>> EJB's, not for class loading.
> Considering use case: moduleA depends on moduleB. On restart both
> deployments are processed in parallel. Even with 100 other deployments
> in between it is guaranteed that moduleA wont run into "missing
> service on next phase" error because the module service for B has not
> been installed? If so I take back the above prediction on restart, but
> still hold the unmanageable claim because ordering is delegated to the
> user (i.e. he must get it right the first time).
>
>>
>>>
>>> > the classic one is deployment of JDBC drivers that have an OSGI
>>> manifest
>>>
>>> We already removed the hack that disables OSGi for this case. The
>>> JDBC driver *is* an OSGi bundle because it contains valid OSGi
>>> metadata. It gets processed as such and should work as expected. All
>>> DUP processing is identical as before except the way module
>>> dependencies are computed and how the Module service is created. The
>>> only case where an OSGi bundle gets treated as a library jar is when
>>> it is located in an EAR/lib directory. Bundles contained in EARs are
>>> otherwise processed as OSGi sub deployments.
>>
>> It sounds like because we have removed the hack JDBC drivers now will
>> not work if they fail to resolve?
>
> If they fail to resolve it would be because a requirement specified by
> the JDBC driver cannot be satisfied (e.g. wrong execution environment,
> missing package wire). I'd say the deployment of that driver should
> fail at resolve time because it would not work anyway because of the
> missing wire to a valid capability. Please don't forget that the
> requirements given by author should be honoured and satisfied if you
> want the driver to work - they should not be ignored or replaced by
> some made up hard wires that happen to work. In this respect a JDBC
> driver is no different to any other OSGi bundle.
>
>>
>>>
>>> > we should not be allowing the presence of the OSGI subsystem to
>>> provide a different experience for users that are only after EE
>>> functionality
>>>
>>> Agreed, EE deployments should not be effected - and I don't think
>>> they are. The OSGi subsystem is not activated unless #1 you do so by
>>> management op #2 you deploy a bundle #3 some component is an
>>> injection target for the system BundleContext
>>>
>>> > We remove OSGI from the default profile, and provide a
>>> standalone-osgi.xml for users that wish to use OSGI
>>>
>>> AFAICS this would remove a few services that are already lazy and a
>>> few DUPs that deal with bundle deployments. We already have the
>>> configuration for a pure OSGi runtime as you suggest. Removing the
>>> OSGi subsystem from the default profile would not solve the need for
>>> DUP authors to be aware of OSGi deployments and code for the case of
>>> unresolved bundle deployments.
>>
>> Even if we resolve the module issue I still think that it would be
>> worth making this a separate profile. Like Jaikiran I really don't
>> like the idea of other subsystems having to code around OSGI. Another
>> possibility we could potentially explore is a separate deployment
>> chain for OSGI, so these DUP's do not even run if it is an OSGI
>> deployment.
> The purpose of OSGi integration in AS7 is to make middleware services
> that come with AS7 available to modular applications that use the OSGi
> standard and vice versa (i.e. make OSGi services available to EE
> components). We are not trying to build a standalone OSGi runtime and
> compete with Virgo <http://www.eclipse.org/virgo/>, Karaf
> <http://karaf.apache.org/>, etc. Instead, we are competing against
> WebSphere, WebLogic, Glassfish - which AFAIK all use OSGi as their
> bottom most layer and increasingly so make this tech available to user
> deployments. From the business perspective the ability to architect
> non-trivial modular applications in a standard way is a requirement on
> the product sheet.
>>
>> Do we have any usage data on how many of our users actually use OSGI?
>> The more I think about it the more I think it makes sense to leave it
>> out of the default profile. Even though you say 'it is not active
>> unless you deploy a bundle', the thing is that many JDBC driver have
>> OSGI metadata, so users that simply want to setup a datasource will
>> still have OSGI activating, which is usually not what they would want.
> I have download stats on sourceforge for the jbosgi umbrella which are
> around 3000/month
> <http://sourceforge.net/projects/jboss/files/JBossOSGi/stats/timeline?dates=2012-01-01+to+2012-08-15>.
> I also know of a few large EAP accounts that are using this tech or
> have it as a decision maker for EAP yes/no. The reason that many JDBC
> drivers have OSGi metadata is because they *are* OSGi bundles and want
> their requirements to be honoured in a given runtime. OSGi subsystem
> startup should be quick and flawless and those driver bundles should
> work seamless in AS7. They currently do AFAIK - if not I'd be
> interested in the details.
>>
>> Stuart
>>
>>>
>>> > OSGI deployment that cannot be resolved pause the deployment
>>> process until such time as they can be
>>>
>>> Yes, this is very much in line with what I think how it should work.
>>> The management API should allow the user to specify whether a
>>> deployment should get resolved/activated too. As a desired side
>>> effect this could introduce life cycle for any AS7 deployment (i.e.
>>> start/stop decoupled from deploy/undeploy). I already did some work
>>> in this direction related to in "Add notion of start/stop for
>>> deployments <https://issues.jboss.org/browse/AS7-2777>". It builds
>>> on top of "Allow management client to associate metadata with
>>> DeploymentUnit <https://issues.jboss.org/browse/AS7-3694>", which is
>>> waiting to get pulled <https://github.com/jbossas/jboss-as/pull/2790>.
>>>
>>> > which means that there will always be a Module available
>>>
>>> YES ;-)
>>>
>>> cheers
>>> --thomas
>>>
>>> On 08/15/2012 07:26 AM, Stuart Douglas wrote:
>>>> Why would the OSGI bundle not be able to resolve, is it because is waiting for another OSGI bundle to be installed? And if so, wouldn't it make more sense to pause the deployment process until the bundle can be resolved? Otherwise the behaviour will be different depending on when the bundle is resolved (e.g. if a bundle is resolved late it will not have EJB's deployed, if it is resolved early it will).
>>>>
>>>> I really hate the way that OSGI takes over and prevents the module being created, I am pretty sure that the number of users that this has caused problems for is larger than the number of users that actually use OSGI (the classic one is deployment of JDBC drivers that have an OSGI manifest).
>>>>
>>>> I think we really need a solution for this for AS 7.2, because as it currently stands we are primarily an EE app server, and we should not be allowing the presence of the OSGI subsystem to provide a different experience for users that are only after EE functionality.
>>>>
>>>> To this end, I propose the following:
>>>>
>>>> - We remove OSGI from the default profile, and provide a standalone-osgi.xml for users that wish to use OSGI, this way OSGI cannot affect users that simply want EE functionality
>>>> - OSGI deployment that cannot be resolved pause the deployment process until such time as they can be, by making the POST_MODULE DeploymentUnitPhaseService passive, which means that there will always be a Module available.
>>>>
>>>> What do you think?
>>>>
>>>> Stuart
>>>>
>>>> On 15/08/2012, at 3:05 PM, Thomas Diesler<thomas.diesler at jboss.com> wrote:
>>>>
>>>>> Folks,
>>>>>
>>>>> a quick reminder that you cannot assume a valid Module attachment in
>>>>> Phase.POST_MODULE or after.
>>>>>
>>>>> An OSGi deployment that cannot resolve won't have a Module attached to
>>>>> the DU. There is talk about aligning the deployment phase names with
>>>>> Bundle life cycle terminology. IMHO Phase.POST_MODULE and Phase.INSTALL
>>>>> are not so lucky names because they imply meaning that may not be true.
>>>>> For suggested improvement seehttps://issues.jboss.org/browse/AS7-3585
>>>>>
>>>>> This is related to:https://issues.jboss.org/browse/AS7-5376
>>>>>
>>>>> cheers
>>>>> --thomas
>>>>>
>>>>> --
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>> Thomas Diesler
>>>>> JBoss OSGi Lead
>>>>> JBoss, a division of Red Hat
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>> jboss-as7-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>> --
>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>> Thomas Diesler
>>> JBoss OSGi Lead
>>> JBoss, a division of Red Hat
>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>
>
> --
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-as7-dev/attachments/20120816/b8d10ae9/attachment-0001.html
More information about the jboss-as7-dev
mailing list