<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<div class="moz-cite-prefix">On 08/17/2012 02:16 AM, Stuart Douglas
wrote:<br>
</div>
<blockquote
cite="mid:6780F68E-21CD-47FB-A889-5051D5B86CBE@gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<br>
<div>
<div>On 17/08/2012, at 1:42 AM, Thomas Diesler <<a
moz-do-not-send="true"
href="mailto:thomas.diesler@jboss.com">thomas.diesler@jboss.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF"> <tt>Regarding ...<br>
</tt>
<blockquote><tt><i>This sounds very non-deterministic. Just
to clarify, are you saying that if the user has a
complex bundle deployment with lots of
inter-dependencies on startup some may be resolved and
some won't, and this may change on subsequent startups
depending on the order in which they start?</i><br>
</tt></blockquote>
<tt>With a complex set of bundle deployments the user will
have to deploy them in a known order (which is a problem
in itself). There is pull request <a
moz-do-not-send="true"
href="https://github.com/jbossas/jboss-as/pull/2790">#2790</a>
waiting that will allow the management client to have
control over the auto start behaviour. So a user could
first install the complete set in multiple operations and
later explicitly start a selected set of bundles. This
would overcome the order issue on first deploy.<br>
</tt></div>
</blockquote>
<div><br>
</div>
<div>I really don't like this solution. I think that the best
solution here is passive deployments, that don't start
POST_MODULE until all their dependencies are available. In
this case it does not have to be a explicit dependency on
potential future bundles, but you you could have a 'resolved'
service that acts as a gate, once OSGI has resolved the bundle
it creates this service, which will then trigger the
deployment to continue.</div>
</div>
</blockquote>
<tt>Yes, the notion of POST_MODULE phase waiting on the
Bundle.RESOLVED service (which we already have) is the right
direction I would think.</tt><br>
<blockquote
cite="mid:6780F68E-21CD-47FB-A889-5051D5B86CBE@gmail.com"
type="cite">
<div><br>
<blockquote type="cite">
<div text="#000000" bgcolor="#FFFFFF"><tt> <br>
Once the bundles are installed and activated the framework
records their respective state. On server restart these
persistent bundles are deployed in an arbitrary order but
there is a guarantee backed into the Framework integration
layer that ensures that the first resolve attempt is made
after all persistent bundles have been installed. From the
resolve perspective order also matters - you might get
different wiring results depending on the order you
resolve the bundles. One possible approach might be to
resolve the full set of persistent bundles at once, but
the guarantee for an identical wiring is still weak. A
better approach would be to always resolve in a known
order (i.e. sort by bundle id). The still better solution
would be to persist the last known wiring graph and
restore that on startup. Currently, the persistent bundles
are resolved in the order they hit the
BundleResolveProcessor which is arbitrary AFAIK.<br>
</tt></div>
</blockquote>
<div><br>
</div>
<div>I think that this needs to be deterministic, otherwise we
will end up with a situation where deploying the same thing to
a domain results in different wirings for each server in the
domain. Persisting the wiring does not really help in this
case. IMHO any form of non-determinism is a serious bug.</div>
</div>
</blockquote>
<tt>Here is how it should work IMHO<br>
<br>
#1 The user needs to have control over whether bundle deployment
should be automatically resolved or not. It is perfectly ok for
the user to want "just install" behaviour. It is also ok, that the
user wants the bundle to resolve/start but it cannot not for one
reason or another. In which case the deployment chain would wait
on Bundle.RESOLVED. Its at the discretion of the framework to
resolve that bundle at any time - this would normally be triggered
by a class load attempt or an explicit Bundle.start() call.<br>
<br>
#2 It must be guaranteed that on restart we get the same wiring
for the persistent bundles. This could be done in two ways. #2.1
The order in which the bundles hit the resolve phase must be
deterministic (i.e. order of bundle id) and the resolver must
guarantee to produce the same result for a given bundle set and
order<br>
#2.2 Every successful resolver run records the wiring result. On
restart, that wiring result is restored given that the set of
persistent bundles is both present and not modified.<br>
<br>
I have a prototype of a deployment chain that waits in a certain
stop phase depending on a user defined StartPolicy. There are
additional start/stop management operations that make the
deployment progress or reverse DUP processing respectively.
Perhaps you like to have a look at<br>
<br>
<a class="moz-txt-link-freetext" href="https://github.com/tdiesler/jboss-as/tree/as2777">https://github.com/tdiesler/jboss-as/tree/as2777</a><br>
<br>
</tt>
<blockquote
cite="mid:6780F68E-21CD-47FB-A889-5051D5B86CBE@gmail.com"
type="cite">
<div>
<div><br>
</div>
<div>Stuart</div>
<br>
<blockquote type="cite">
<div text="#000000" bgcolor="#FFFFFF"><tt> <br>
I have written up the complete subsystem activation
process in <a moz-do-not-send="true"
href="https://community.jboss.org/wiki/OSGiSubsystemActivationProcess">this
article</a>. It contains the known issues and ideas for
possible solutions. Perhaps we can start from there to
find a more consistent solution.<br>
<br>
</tt></div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
<br>
<blockquote type="cite">
<div text="#000000" bgcolor="#FFFFFF"><tt> cheers<br>
--thomas <br>
</tt><br>
<div class="moz-cite-prefix">On 08/15/2012 01:32 PM, Thomas
Diesler wrote:<br>
</div>
<blockquote cite="mid:502B88C0.9060307@jboss.com"
type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<br>
<div class="moz-cite-prefix">On 08/15/2012 11:20 AM,
Stuart Douglas wrote:<br>
</div>
<blockquote
cite="mid:7FAB89A5-233F-49DA-9177-44461AE7F8DA@gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<br>
<div>
<div>On 15/08/2012, at 6:59 PM, Thomas Diesler <<a
moz-do-not-send="true"
href="mailto:thomas.diesler@jboss.com">thomas.diesler@jboss.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF"> <tt>> Why
would the OSGI bundle not be able to resolve, is
it because is waiting for another OSGI bundle to
be installed?<br>
<br>
This is by virtue of the API -
BundleContext.install() does not resolve the
bundle. As the method name suggests, it just
installs the bundle. <br>
<br>
In the hot-deployment case it is debatable
whether bundle resolution and later bundle
activation should be attempted or not. By
design, the order of bundle deployment is not
the responsibility of the user but that of the
framework. For a complex graph of interdependent
bundles the user cannot possibly be asked to
deploy them in the "right order". Instead the
API allows to INSTALL the complete set (i.e.
make the metadata available to the resolver) and
later activate the bundles as needed. There are
other triggers for bundle resolution too (e.g.
resource access)<br>
<br>
We currently do resolve/activate during DUP
processing on a trial basis. For a bundle that
only has dedependencies on already installed
bundles the resolve/activation works fine and
the services become available. I guess this is
the expected hot-deploy behaviour. A bundle that
cannot resolve - for various reasons, one being
the user says so - we dont attempt to start the
bundle either. It would still run through all
remaining DUPs but does not have a module
attached.<br>
</tt></div>
</blockquote>
<div><br>
</div>
<div>This sounds very non-deterministic. Just to
clarify, are you saying that if the user has a
complex bundle deployment with lots of
inter-dependencies on startup some may be resolved
and some won't, and this may change on subsequent
startups depending on the order in which they start?</div>
</div>
</blockquote>
<tt>Yes, this is a long outstanding issue [<a
moz-do-not-send="true"
href="https://issues.jboss.org/browse/AS7-378">AS7-378</a>].
I still have no guarantee that all bundles in a given
set have been INSTALLED (in OSGi terminology) / have
completed the Phase.REGISTER phase (in AS7 terminology)
when the one bundle hits the BundleResolveProcessor. The
framework records the persistent bundle state and on
restart it is a requirement that all persistent bundles
reach their respective target state for successful
framework initialization. There is a little more detail
to it and I'd be more than happy to work with you to
find a consistent solution. We can take up this topic in
another osgi specific thread if you like.<br>
</tt>
<blockquote
cite="mid:7FAB89A5-233F-49DA-9177-44461AE7F8DA@gmail.com"
type="cite">
<div><br>
<blockquote type="cite">
<div text="#000000" bgcolor="#FFFFFF"><tt> <br>
Non-OSGi deployments that use jboss-modules
metadata to define their dependencies (i.e.
Dependencies clause in the manifest) have that
problem too, but worse. A complex system of
interdependent module deployments is likely not
manageable because of this ordering issue. Even
if the user gets the ordering right the first
time, on server restart the notion of deployment
order is lost and very likely initial
deployments will fail with no osgi involved.
Granted that this describes a use case that is
not intended to be used for user deployments. <br>
</tt></div>
</blockquote>
<div><br>
</div>
<div>No, JBoss modules uses MSC services to resolve
the dependencies. At container start all deployments
are now run as part of the boot ops, so as long as
all deployments are present this will always work.
We do need a more specified way of saying "Don't
start this deployment until another deployment is
done", but this is mainly for things like EJB's, not
for class loading. <br>
</div>
</div>
</blockquote>
<tt>Considering use case: moduleA depends on moduleB. On
restart both deployments are processed in parallel. Even
with 100 other deployments in between it is guaranteed
that moduleA wont run into "missing service on next
phase" error because the module service for B has not
been installed? If so I take back the above prediction
on restart, but still hold the unmanageable claim
because ordering is delegated to the user (i.e. he must
get it right the first time).<br>
<br>
</tt>
<blockquote
cite="mid:7FAB89A5-233F-49DA-9177-44461AE7F8DA@gmail.com"
type="cite">
<div><br>
<blockquote type="cite">
<div text="#000000" bgcolor="#FFFFFF"><tt> <br>
> the classic one is deployment of JDBC
drivers that have an OSGI manifest<br>
<br>
We already removed the hack that disables OSGi
for this case. The JDBC driver *is* an OSGi
bundle because it contains valid OSGi metadata.
It gets processed as such and should work as
expected. All DUP processing is identical as
before except the way module dependencies are
computed and how the Module service is created.
The only case where an OSGi bundle gets treated
as a library jar is when it is located in an
EAR/lib directory. Bundles contained in EARs are
otherwise processed as OSGi sub deployments.</tt><br>
</div>
</blockquote>
<div><br>
</div>
<div>It sounds like because we have removed the hack
JDBC drivers now will not work if they fail to
resolve?</div>
</div>
</blockquote>
<tt><br>
If they fail to resolve it would be because a
requirement specified by the JDBC driver cannot be
satisfied (e.g. wrong execution environment, missing
package wire). I'd say the deployment of that driver
should fail at resolve time because it would not work
anyway because of the missing wire to a valid
capability. Please don't forget that the requirements
given by author should be honoured and satisfied if you
want the driver to work - they should not be ignored or
replaced by some made up hard wires that happen to work.
In this respect a JDBC driver is no different to any
other OSGi bundle.<br>
<br>
</tt>
<blockquote
cite="mid:7FAB89A5-233F-49DA-9177-44461AE7F8DA@gmail.com"
type="cite">
<div><br>
<blockquote type="cite">
<div text="#000000" bgcolor="#FFFFFF"> <br>
<tt>> we should not be allowing the presence of
the OSGI subsystem to provide a different
experience for users that are only after EE
functionality <br>
<br>
Agreed, EE deployments should not be effected -
and I don't think they are. The OSGi subsystem
is not activated unless #1 you do so by
management op #2 you deploy a bundle #3 some
component is an injection target for the system
BundleContext<br>
<br>
> We remove OSGI from the default profile,
and provide a standalone-osgi.xml for users that
wish to use OSGI<br>
<br>
AFAICS this would remove a few services that are
already lazy and a few DUPs that deal with
bundle deployments. W</tt><tt>e already have t</tt><tt>he
configuration for a pure OSGi runtime as you
suggest. Removing the OSGi subsystem from the
default profile would not solve the need for DUP
authors to be aware of OSGi deployments and code
for the case of unresolved bundle deployments.<br>
</tt></div>
</blockquote>
<div><br>
</div>
<div>Even if we resolve the module issue I still think
that it would be worth making this a separate
profile. Like Jaikiran I really don't like the idea
of other subsystems having to code around OSGI.
Another possibility we could potentially explore is
a separate deployment chain for OSGI, so these DUP's
do not even run if it is an OSGI deployment. <br>
</div>
</div>
</blockquote>
<tt>The purpose of OSGi integration in AS7 is to make
middleware services that come with AS7 available to
modular applications that use the OSGi standard and vice
versa (i.e. make OSGi services available to EE
components). We are not trying to build a standalone
OSGi runtime and compete with <a moz-do-not-send="true"
href="http://www.eclipse.org/virgo/">Virgo</a>, <a
moz-do-not-send="true" href="http://karaf.apache.org/">Karaf</a>,
etc. Instead, we are competing against WebSphere,
WebLogic, Glassfish - which AFAIK all use OSGi as their
bottom most layer and increasingly so make this tech
available to user deployments. From the business
perspective the ability to architect non-trivial modular
applications in a standard way is a requirement on the
product sheet. <br>
</tt>
<blockquote
cite="mid:7FAB89A5-233F-49DA-9177-44461AE7F8DA@gmail.com"
type="cite">
<div>
<div><br>
</div>
<div>Do we have any usage data on how many of our
users actually use OSGI? The more I think about it
the more I think it makes sense to leave it out of
the default profile. Even though you say 'it is not
active unless you deploy a bundle', the thing is
that many JDBC driver have OSGI metadata, so users
that simply want to setup a datasource will still
have OSGI activating, which is usually not what they
would want.</div>
</div>
</blockquote>
<tt>I have download stats on sourceforge for the jbosgi
umbrella which are <a moz-do-not-send="true"
href="http://sourceforge.net/projects/jboss/files/JBossOSGi/stats/timeline?dates=2012-01-01+to+2012-08-15">around
3000/month</a>. I also know of a few large EAP
accounts that are using this tech or have it as a
decision maker for EAP yes/no. The reason that many JDBC
drivers have OSGi metadata is because they *are* OSGi
bundles and want their requirements to be honoured in a
given runtime. OSGi subsystem startup should be quick
and flawless and those driver bundles should work
seamless in AS7. They currently do AFAIK - if not I'd be
interested in the details. <br>
</tt>
<blockquote
cite="mid:7FAB89A5-233F-49DA-9177-44461AE7F8DA@gmail.com"
type="cite">
<div>
<div><br>
</div>
<div>Stuart</div>
<br>
<blockquote type="cite">
<div text="#000000" bgcolor="#FFFFFF"><tt><br>
</tt><tt>> OSGI deployment that cannot be
resolved pause the deployment process until such
time as they can be <br>
<br>
Yes, this is very much in line with what I think
how it should work. The management API should
allow the user to specify whether a deployment
should get resolved/activated too. As a desired
side effect this could introduce life cycle for
any AS7 deployment (i.e. start/stop decoupled
from deploy/undeploy). I already did some work
in this direction related to in "<a
moz-do-not-send="true"
href="https://issues.jboss.org/browse/AS7-2777">Add
notion of start/stop for deployments</a>". It
builds on top of "<a moz-do-not-send="true"
href="https://issues.jboss.org/browse/AS7-3694">Allow
management client to associate metadata with
DeploymentUnit</a>", which is waiting to get <a
moz-do-not-send="true"
href="https://github.com/jbossas/jboss-as/pull/2790">pulled</a>.<br>
<br>
> which means that there will always be a
Module available <br>
<br>
YES ;-)<br>
<br>
cheers<br>
--thomas<br>
</tt><br>
<div class="moz-cite-prefix">On 08/15/2012 07:26
AM, Stuart Douglas wrote:<br>
</div>
<blockquote
cite="mid:2ABFD913-B778-4469-83B8-607BE9BDC902@gmail.com"
type="cite">
<pre wrap="">Why would the OSGI bundle not be able to resolve, is it because is waiting for another OSGI bundle to be installed? And if so, wouldn't it make more sense to pause the deployment process until the bundle can be resolved? Otherwise the behaviour will be different depending on when the bundle is resolved (e.g. if a bundle is resolved late it will not have EJB's deployed, if it is resolved early it will).
I really hate the way that OSGI takes over and prevents the module being created, I am pretty sure that the number of users that this has caused problems for is larger than the number of users that actually use OSGI (the classic one is deployment of JDBC drivers that have an OSGI manifest).
I think we really need a solution for this for AS 7.2, because as it currently stands we are primarily an EE app server, and we should not be allowing the presence of the OSGI subsystem to provide a different experience for users that are only after EE functionality.
To this end, I propose the following:
- We remove OSGI from the default profile, and provide a standalone-osgi.xml for users that wish to use OSGI, this way OSGI cannot affect users that simply want EE functionality
- OSGI deployment that cannot be resolved pause the deployment process until such time as they can be, by making the POST_MODULE DeploymentUnitPhaseService passive, which means that there will always be a Module available.
What do you think?
Stuart
On 15/08/2012, at 3:05 PM, Thomas Diesler <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:thomas.diesler@jboss.com"><thomas.diesler@jboss.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Folks,
a quick reminder that you cannot assume a valid Module attachment in
Phase.POST_MODULE or after.
An OSGi deployment that cannot resolve won't have a Module attached to
the DU. There is talk about aligning the deployment phase names with
Bundle life cycle terminology. IMHO Phase.POST_MODULE and Phase.INSTALL
are not so lucky names because they imply meaning that may not be true.
For suggested improvement see <a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://issues.jboss.org/browse/AS7-3585">https://issues.jboss.org/browse/AS7-3585</a>
This is related to: <a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://issues.jboss.org/browse/AS7-5376">https://issues.jboss.org/browse/AS7-5376</a>
cheers
--thomas
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
jboss-as7-dev mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:jboss-as7-dev@lists.jboss.org">jboss-as7-dev@lists.jboss.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/jboss-as7-dev">https://lists.jboss.org/mailman/listinfo/jboss-as7-dev</a>
</pre>
</blockquote>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
</pre>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
</pre>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
</pre>
</body>
</html>