In addition to checking "all" boot time, could we also check the memory
in use after booting "all". with and without. This is important as
customers will be unhappy if more memory is consumed by AS, leaving less
room for their applications. This is already a problem and needs to
improve (not regress further).
On Fri, 2010-04-09 at 13:38 +0200, Thomas Diesler wrote:
FYI, here a transcript of the chat that I had with Dimitris
Friday 09 April 2010
...
[12:38:37] Thomas Diesler: The discussion is however relevant for the
upcoming M3 release. ;-)
[12:45:25] Thomas Diesler: the beauty about smoke-tests is that folks
actually run them often
[12:45:45] Dimitris Andreadis: i know, but your test target will also
show up in hudson runs
[12:45:45] Thomas Diesler: thats why I wanted osgi coverage in there out
of the box
...
[12:46:28] Thomas Diesler: again, whats your concern against osgi being
part of 'all'? is it just boot time?
[12:46:42] Dimitris Andreadis: yea, boot time
...
[12:51:20] Dimitris Andreadis: I think the 'osgi' config is a good first
step to show that "we do OSGi", too. And we can take it from there
depending on user interest.
[12:53:59] Thomas Diesler: my main concern with that approach is that
other services i.e. ejb3, web, hornetq, etc. would not work as expected
when a user installs jbosgi using the installer. Because there is no
test coverage that tests these services with osgi in place.
[12:54:07] Thomas Diesler: Here is a prime example
http://community.jboss.org/thread/150453?tstart=0
[12:54:54] Thomas Diesler: Simple EJB3 SLSB breaks when osgi is installed
[12:55:13] Thomas Diesler: Issues like this I'd like to see in the AS
hudson run
[12:55:37] Thomas Diesler: which I would not if its (only) in a separate
profile
[12:55:53] Dimitris Andreadis: but we could test for that with your
custom osgi-test config, no?
[12:57:01] Thomas Diesler: only if we run all-tests against an
'all+osgi' configuration
[12:57:18] Thomas Diesler: I claim taht it would be better to make osgi
part of all
[12:57:35] Thomas Diesler: so that all test coverage must work with osgi
in place
[12:58:01] Dimitris Andreadis: i see your point
[13:04:34] Thomas Diesler: Here is another example:
https://jira.jboss.org/jira/browse/JBAS-7909
[13:04:51] Thomas Diesler: the 'default' profile would not even boot
with osgi in place
[13:05:17] Thomas Diesler: because the AOP subsystems makes certain
assumptions that osgi bundle deployments do not sattisfy
[13:05:59] Thomas Diesler: stuff like this I'd like to catch early
before I want to ship osgi as part of AS
[13:06:07] Dimitris Andreadis: in that case the only way to satisfy
everybody is to copy over osgi for all tests and remove it afterwards.
[13:06:20] Dimitris Andreadis: i.e. make all-test a custom config
[13:06:38] Thomas Diesler: that would work
...
[13:07:47] Dimitris Andreadis: maybe you can throw the idea/reasoning on
the dev-list thread
[13:08:26] Thomas Diesler: sure, let me add the boot time delta
[13:08:38] Dimitris Andreadis: yes, thats imporant
[13:19:17] Thomas Diesler: on my machine I get 'all' config boot times of
osgi excluded: ~ 1:17min
osgi included: ~ 1:37min
This is an increase of ca. 29%
[13:19:50] Dimitris Andreadis: 1:17 is a lot already
...
[13:28:05] Dimitris Andreadis: all+osgi: 1:02
[13:29:52] Dimitris Andreadis: all-osgi: 49
[13:30:33] Dimitris Andreadis: +26.5%
[13:31:26] Thomas Diesler: same result
[13:31:31] Thomas Diesler: so what do you say?
[13:33:13] Dimitris Andreadis: do the isolated testing anyway (custom
config with smoke tests + osgi tests anyway) and discuss if we should
alter the all testing
[13:33:36] Thomas Diesler: ok
Full transcript
---------------
Friday 09 April 2010
[12:38:04] Thomas Diesler: Hi Dimitries, shall we continue this
discussion for a short while here? Nobody else seems to be concerned
anyway. :-)
[12:38:37] Thomas Diesler: The discussion is however relevant for the
upcoming M3 release. ;-)
[12:39:00] Dimitris Andreadis: Jason will be online in a couple of
hours, I suppose he'll reply, too.
[12:39:28] Dimitris Andreadis: I think it's ok as it is now, with the
osgi config
[12:39:33] Thomas Diesler: perhaps we can reach a consensus until then.
it could safe him some time
[12:40:13] Dimitris Andreadis: But needs better testing, maybe a special
test target with a custom config if you want to test combinations with
other components not included in the osgi config
[12:42:06] Thomas Diesler: smoke-tests would also run against 'default',
right?
[12:42:25] Dimitris Andreadis: i think it runs against all
[12:42:51] Thomas Diesler: I could try to create an osgi-test profile on
top of default and rune the existing smoke tests + some osgi smoke tests
against that
[12:43:30] Thomas Diesler: yes, it runs against 'all', but I mean
'default' would be sufficient for the smoke tests
[12:44:03] Dimitris Andreadis: i think there is one or two tests run by
the smoke test target that need clustering (i.e. all) but you could
leave those 2 out.
[12:44:18] Dimitris Andreadis: but that sounds like a plan
[12:44:48] Dimitris Andreadis: i.e. osgi-test with the smoke test (minus
a couple) plus your own osgi tests
[12:45:25] Thomas Diesler: the beauty about smoke-tests is that folks
actually run them often
[12:45:45] Dimitris Andreadis: i know, but your test target will also
show up in hudson runs
[12:45:45] Thomas Diesler: thats why I wanted osgi coverage in there out
of the box
[12:46:25] Dimitris Andreadis: osgi might be deployed in the default
config, but let's give it some time, I think.
[12:46:28] Thomas Diesler: again, whats your concern against osgi being
part of 'all'? is it just boot time?
[12:46:42] Dimitris Andreadis: yea, boot time
[12:47:10] Dimitris Andreadis: I tell you, it will be much more
"advertised" if you leave it in its own profile for now.
[12:47:32] Dimitris Andreadis: ..and it looks good, i must say, I've
just tested it.
[12:47:41] Thomas Diesler: sure, but I would have an separate 'osgi'
profile anyway
[12:47:48] Dimitris Andreadis: ok
[12:48:03] Thomas Diesler: I just think it might be sensible to have it
in 'all' as well
[12:48:14] Thomas Diesler: let me have a look at the boot time
difference ...
[12:49:02] Dimitris Andreadis: you could also make an ant script in
/docs/examples that just copies over whatever is need from the 'osgi'
config to the 'all' or 'default' config, if users want it integrated.
[12:50:17] Thomas Diesler: I have an installer ;-)
http://jbmuc.dyndns.org/jboss-osgi/userguide/html/ChapGettingStarted.html...
[12:50:27] Dimitris Andreadis: ok :)
[12:51:20] Dimitris Andreadis: I think the 'osgi' config is a good first
step to show that "we do OSGi", too. And we can take it from there
depending on user interest.
[12:53:59] Thomas Diesler: my main concern with that approach is that
other services i.e. ejb3, web, hornetq, etc. would not work as expected
when a user installs jbosgi using the installer. Because there is no
test coverage that tests these services with osgi in place.
[12:54:07] Thomas Diesler: Here is a prime example
http://community.jboss.org/thread/150453?tstart=0
[12:54:54] Thomas Diesler: Simple EJB3 SLSB breaks when osgi is installed
[12:55:13] Thomas Diesler: Issues like this I'd like to see in the AS
hudson run
[12:55:37] Thomas Diesler: which I would not if its (only) in a separate
profile
[12:55:53] Dimitris Andreadis: but we could test for that with your
custom osgi-test config, no?
[12:57:01] Thomas Diesler: only if we run all-tests against an
'all+osgi' configuration
[12:57:18] Thomas Diesler: I claim taht it would be better to make osgi
part of all
[12:57:35] Thomas Diesler: so that all test coverage must work with osgi
in place
[12:58:01] Dimitris Andreadis: i see your point
[13:04:34] Thomas Diesler: Here is another example:
https://jira.jboss.org/jira/browse/JBAS-7909
[13:04:51] Thomas Diesler: the 'default' profile would not even boot
with osgi in place
[13:05:17] Thomas Diesler: because the AOP subsystems makes certain
assumptions that osgi bundle deployments do not sattisfy
[13:05:59] Thomas Diesler: stuff like this I'd like to catch early
before I want to ship osgi as part of AS
[13:06:07] Dimitris Andreadis: in that case the only way to satisfy
everybody is to copy over osgi for all tests and remove it afterwards.
[13:06:20] Dimitris Andreadis: i.e. make all-test a custom config
[13:06:38] Thomas Diesler: that would work
[13:07:47] Dimitris Andreadis: maybe you can throw the idea/reasoning on
the dev-list thread
[13:08:26] Thomas Diesler: sure, let me add the boot time delta
[13:08:38] Dimitris Andreadis: yes, thats imporant
[13:19:17] Thomas Diesler: on my machine I get 'all' config boot times of
osgi excluded: ~ 1:17min
osgi included: ~ 1:37min
This is an increase of ca. 29%
[13:19:50] Dimitris Andreadis: 1:17 is a lot already
[13:20:38] Thomas Diesler: slow machine (yawn)
[13:21:00] Dimitris Andreadis: just trying out mine (another slow machine)
[13:21:23] Dimitris Andreadis: all: 1:10
[13:21:53] Thomas Diesler: 1011 cp -r
server/osgi/deployers/osgi.deployer server/all/deployers/
1012 cp -r server/osgi/deploy/osgi server/all/deploy/
[13:22:31] Thomas Diesler:
cp server/osgi/conf/bootstrap/deployers.xml
server/all/conf/bootstrap/deployers.xml
[13:25:02] Dimitris Andreadis: trying it out with osgi...
[13:26:11] Dimitris Andreadis: all+osgi: 1:04
[13:26:14] Dimitris Andreadis: ?
[13:26:29] Thomas Diesler: we are all set then :D
[13:26:42] Dimitris Andreadis: am on WinXP
[13:26:44] Dimitris Andreadis: :)
[13:27:15] Dimitris Andreadis: i'll make a 2nd try, it's probably cached
at the filesystem level
[13:28:05] Dimitris Andreadis: all+osgi: 1:02
[13:29:52] Dimitris Andreadis: all-osgi: 49
[13:30:33] Dimitris Andreadis: +26.5%
[13:31:26] Thomas Diesler: same result
[13:31:31] Thomas Diesler: so what do you say?
[13:33:13] Dimitris Andreadis: do the isolated testing anyway (custom
config with smoke tests + osgi tests anyway) and discuss if we should
alter the all testing
[13:33:36] Thomas Diesler: ok
On 04/09/2010 12:24 PM, Dimitris Andreadis wrote:
> What I've seen is most users taking the existing configs as-is and
> adding their deployments, with small modifications. If they want
> cluster support they just pick the 'all' config. Advanced users might
> try to trim down unused services but many will just leave what is
> already there.
>
> I definitely think osgi support should be tested as part of the
> testsuite to spot regression and test osgi in combination with other
> subsystems. But this can always be done by just creating and booting a
> custom config based off of 'default', as many other tests do, e.g. a
> lot of security tests. So that could make a separate testing target.
>
> In any case, this whole discussion will not be much relevant as jboss
> AS evolves and gives you more freedom to define your configuration, or
> just have everything available and on-demand activated.
>
> Thomas Diesler wrote:
>> As of yesterday we had osgi part of the 'all' profile and a separate
>> 'osgi' profile. The idea was that integration issues with osgi would
>> show up early in the 'all' testsuite and I can have osgi smoke tests
>> as part of the smoke-tests target. The osgi profile would only
>> contain services that are (at least to some degree) integrated with
>> osgi. i.e. TM is, EJB3 is not
>>
>> For a better understanding, are users really running the 'all'
>> profile? I thought this was more a 'pick and choose' repository of
>> stuff.
>>
>> My own QA installs osgi in 'default' and runs 90+ tests on it. It
>> would however not detect that the osgi installation breaks EJB3 tests.
>>
>> I guess we have two options
>>
>> #1 install osgi in 'all' and integrate it in smoke-tests and
>> all-tests. So we always know that osgi works together with all the
>> other stuff. Additionally we could distribute a stripped down 'osgi'
>> profile.
>>
>> #2 install osgi in its own profile only. Only osgi functionality is
>> tested there and we move bits in there over time that actually have
>> integration and test coverage.
>>
>> Both would work work for me.
>>
>> cheers
>> -thomas
>>
>> On 04/09/2010 11:11 AM, Dimitris Andreadis wrote:
>>> Although a separate 'osgi' profile/config increases the distro size,
>>> I think too, it's better to have it separate from 'all'.
>>>
>>> If you include it in 'all', it will increase the already long boot
>>> time, and the functionality will get lost together with everything
>>> else that is already there. Also 'all' is usually the base for EAP
>>> 'production' and we might not want to offer full OSGi support, as
yet.
>>>
>>> On the other hand, having it separately, makes it stand out and will
>>> trigger more people to try it out, start experimenting with it and
>>> provide feedback. When at a later time, JBoss-OSGi becomes more
>>> integrated with the other subsystems (EJB3, etc.) we might re-consider.
>>>
>>> I would even suggest that at some point we drop the 'minimal' config
>>> in favor of 'osgi', in the sense that if people want a bare runtime
>>> to write server extensions, they might be better off going with the
>>> OSGi standard.
>>>
>>> My 0.02c
>>>
>>> Thomas Diesler wrote:
>>>> Hi Jason,
>>>>
>>>> I see that you removed OSGi from the 'all' profile. Were there
any
>>>> specific issues with it? I couldn't find pointers to why this was
>>>> necessary.
>>>>
>>>> I reopened
https://jira.jboss.org/jira/browse/JBAS-7661 where I
>>>> make a comment on why I think osgi should be part of the 'all'
>>>> profile. I am now working on osgi smoke tests that I ultimate want
>>>> to run against 'all' as part of the smoke-tests target.
>>>>
>>>> cheers
>>>> -thomas
>>>>
>>