I have spoken with David Lloyd about this, and he has provided me with some additional information that leads me to believe that this is still a good direction to pursue (certainly while we are doing side-by-side comparison with OSGi)
1. All that is left to make this successful is module dependency management.
2. The greatest benefit is that we control the programming model, and don't require any additional tools above what we already use.
3. We've found a way to run Forge from sources within Eclipse (without extra IDE tools.)
The conversation is below, for others who are interested:
lincolnthree
hey folks
question about JBM and MSC
who's around?
3:12
dmlloyd
the usual
3:14
lincolnthree
awesome
So
Question is:
What exactly does MSC provide on top of JBM, and can it be used outside of AS7?
3:15
dmlloyd
nothing, the two are wholly independent
3:15
lincolnthree
Okay… so what does MSC provide?
3:15
dmlloyd
a concurrent dependency-based service container
it starts and stops things
not likely to be useful outside of the app server
3:16
lincolnthree
Why not?
(Thinking Forge)
3:16
dmlloyd
because all an app server is, is services and management
if you pull in services to your project, you're duplicating the app server
3:16
lincolnthree
Forge is just Plugins (services) and a UI layer of some sort
And I'd hope I'm not duplicating the app-server, because I don't need 99% of what it delivers to users.
3:17
dmlloyd
are you saying you want to use MSC?
it's not really a box to check - you'd only use it if you really need asynchronous concurrent service start/stop
3:18
lincolnthree
I'm saying I want to know if it would make any sense, and if it would address any of the problems currently being discussed by Forge
I need to be able to install a plugin, have forge bring up other plugins that may depend on it
stop a plugin
3:18
dmlloyd
also, AS8 will likely use a largely rewritten version of MSC that is transactional
3:18
lincolnthree
bring down all dependent plugins
3:18
dmlloyd
so 1.x has a limited future lifespan I would expect
3:18
lincolnthree
allow plugins to depend on other plugins
in a modular environment
(Which I have working right now via a naive proxy system and some dynamic module configuration)
Basically I am comparing JBM & MSC to OSGi
We are at a crossroads here
of giving up JBM in favor of OSGi
3:19
dmlloyd
I wouldn't recommend that - OSGi is really shit
3:19
lincolnthree
And I need to know which path makes most technical sense for my project
3:20
dmlloyd
people keep blindly adopting it, and by the time it starts stabbing them in the back they have already convinced themselves that they're buzzword-compliant
or whatever it is that causes people to use it
it's like struts
everyone used it even though it's complete shit
mob psychology I guess
3:22
lincolnthree
so maybe you should be involved onthe forge dev list
because the OSGi people are gaining momentum
3:23
dmlloyd
maybe, though my time is already limited
3:23
lincolnthree
And I'm listening because it sounds like it can address a lot of our problems (in combination with weld OSG
3:24
dmlloyd
I'm not going to sell these projects to you. I can only say that OSGi is a mistake that will cost you in the mid- to long-term, and let you make your own choice
3:24
lincolnthree
I can't make an educated choice because there is no documentation about the alternatives.
Which is why I am here with you now.
3:24
dmlloyd
I'm not saying the MSC/Modules is a monolithic alternative either
the two projects are associated by their usage in AS but they're by no means connected in any technical manner
3:25
lincolnthree
You are basically the single point of knowledge for JBM
(Aside from a few others at JBoss and myself included I suppose at this point)
3:25
dmlloyd
I dunno, the docs spell out the basics pretty well I think
3:25
lincolnthree
Basics yes.
3:25
dmlloyd
modules is only basics
3:26
lincolnthree
But you're telling me "Don't use OSGi" without explaining an alternative to the service layer
Which is why I'm asking about MSC.
3:26
dmlloyd
the alternative is to do something else
even a simple start/stop with naive dependency queueing is probably better long-term
just because you'll have control over it, instead of putting your faith in the genie
3:26
lincolnthree
Hm.
3:27
dmlloyd
MSC isn't a good long-term choice just because 2.x won't be compatible with 1.x
it's really meant to be the innards of AS and that's about it
3:27
lincolnthree
Will MSC2 be any different?
3:27
dmlloyd
you could still use it as long as you don't make the API public though
well as always I hope MSC2 will be "the one that sticks"
3:28
lincolnthree
The API for none of this will ever be public
3:28
dmlloyd
I think the odds are better, as we understand the requirements a lot better now that MSC 1 has been in the wild for a while
3:28
lincolnthree
No matter what we choose, the programming model that plugin developers will see is CDI
With some annotations.
3:29
dmlloyd
it has (will have) the benefit of being transactional too, so you can "undo" whole multi-step operations
MSC 1's chief problem is that you can't really know when it's "done" with some logical operation
which is fine in an environment like AS5/6 where you just put deployments in as you go without any verification step of any sort
and that's why it is the way it is
but it turns out when you have a server control API like AS7 does, people want to know how their operations turned out, and revert them if stuff breaks
3:30
lincolnthree
Hm.
Basically here's the problem I want to solve:
Forge has plugins (services)
Right now everyone has to re-invent anything because there's no way of accessing a service from Plugin A in Plugin B
So there's no way to create building blocks, like unix applications
I need Plugin A to be able to use and depend on Plugin B's exposed APIs
And that sounds a lot like what OSGi provides
3:32
dmlloyd
well accessing their APIs is not the same as accessing their objects
3:32
lincolnthree
Explain
I'll clarify
Since API's are objects too
3:33
dmlloyd
only conceptually
physically, APIs are classes
if you want access to an API, modules can create class loading dependencies without any problems
3:33
lincolnthree
Plugin A has (MathService) - Plugin B uses (MathService from A)
3:34
dmlloyd
if you want to manage service lifecycle and manage injections and dependencies in that regard, MSC gives the low-level stuff to do that (we use it as the basis for Java EE injection for example)
3:34
lincolnthree
Yes.
3:35
dmlloyd
if you want real DI based on annotations and descriptors you need something fancier - and even CDI is pretty bumpy when it comes to DI between separate "modules" (not jboss modules, just regular modules in the EE/CDI sense)
in an AS application for example users tend to use EE injection for that
3:35
lincolnthree
This si what I am doing right now:
https://github.com/forge/core/tree/2.0/plugin-container/src/main/java/org/jboss/forge/container/services
@Service and @Remote annotations control injections
And you define your dependencies manually in a config file.
Everything from Plugin A is explosed to Plugin B, if plugin B depends on Plugin A
This could be more fine grained
but for now that's how it works
3:38
dmlloyd
for class loading dependencies, that's probably fine
3:38
lincolnthree
But only @Services are injectable
3:38
dmlloyd
I've yet to see a user legitimately need fine-grained filtering in the real world
3:38
lincolnthree
into another weld container
Each plugin has its own weld container
3:39
dmlloyd
sounds reasonable so far
3:39
lincolnthree
what actually gets injected is a proxy
which is both lazily instantiated when the dependent service becomes avaialble, and also intercepts method calls which set the TCCL to the ClassLoader of the module from which the service was injected (and resets it on completion)
3:40
dmlloyd
sounds just like EJB
3:40
lincolnthree
lol, basically
excepte i have no idea how EJBs really work
3:40
dmlloyd
you inject proxies
they have interceptor stacks which do things like set TCCL
3:41
lincolnthree
now i feel "special"
cool
well… in the end, I guess what is really missing from this whole picture is the dependency management and lifecycle stuff
bringing up dependencies in the right order, as needed
because right now all the containers come up at the same time
3:42
dmlloyd
basically that problem boils down to: a service can start when all its dependencies are satisfied
think of it as a counter
3:42
lincolnthree
and the proxies just "start working" as services come up
3:42
dmlloyd
the counter stores the number of unsatisfied dependencies
when something finishes, it ticks the counters of all its dependents
when those counters hit 0, that dependent can start
3:43
lincolnthree
right. makes sense
3:43
dmlloyd
that's basically all there is to it
MSC takes this idea and goes nuts with it
3:43
lincolnthree
i kind of wonder how much more what I have actually needs
3:43
dmlloyd
makes everything concurrent, with injection, lots of modes like ON_DEMAND and so on, complex error handling
3:43
lincolnthree
ON_DEMAND would be nice
3:44
dmlloyd
MSC 2 will be simpler but also more complex
3:44
lincolnthree
I guess I could do that too
3:44
dmlloyd
simpler in that there are fewer states for a service, but more complex in that it's transactional
3:44
lincolnthree
not sure i really need transactional
3:44
dmlloyd
so you'd open a txn, add all your services (which in turn might add more services or do other things like stopping services), then prepare and if there are no errors, commit
3:45
lincolnthree
hm
3:45
dmlloyd
if there are errors, report and roll back
3:45
jamezp
Couldn't you use to rollback a failure to load a plugin lincolnthree?
3:45
dmlloyd
well, let's not go selling vaporware just yet
3:45
lincolnthree
jamezp: well… what happens is the plugin just doesn't load
3:45
dmlloyd
I still don't actually have a working prototype
3:45
lincolnthree
nothing else blows up
unless it depends on that plugin
3:46
dmlloyd
but, I need one within about two weeks, so...
3:46
jamezp
has faith in dmlloyd
3:46
lincolnthree
but if it does, then i assume if I get this all wrking, the dependency would also not come up
3:48
dmlloyd
yeah dependents as a rule will just sit there until their dependencies appear
in AS5/6, you'd just remove broken stuff and put in the right stuff and it would (hopefully) all fix itself
that's the MSC 1 "way"
in MSC 1, things actually *do* fix themselves once you correct dependency issues
also you can "poke" failed services to make them try again
like if there's a port binding conflict, something you can fix manually
3:50
lincolnthree
interesting
one downside of JBM is that I can't run it from an eclipse launch very easily
because the module path needs to be configured
module library, etc
which is of course, sensible
3:51
dmlloyd
well, yes and no
3:51
jamezp
Sounds like you need a plugin for that
3:51
dmlloyd
you could for example have a flat-classpath application which uses jboss modules
one of the things I'm doing for modules 1.2 is making it more friendly for this kind of thing (a lot of bootstrappy stuff happens in main() and it should be factored out)
but iirc it's possible to do
iow you use modules for your loaded plugins and a flat classpath for the static stuff
mainly I'm working on that to make testing easier, but it probably applies here too
3:53
lincolnthree
that would be really nice
any idea when that might happen?
that would make forge development a LOT easier
3:53
dmlloyd
it should already be possible just by using it
you might have to hack around a quirk or two though
won't know for sure until it happens
3:55
lincolnthree
hm.
Interesting… So then I would need to
supply a module library as an argument to JBM when I started it from my class
and basically just call JBM main from some other class?
But then JBM would also need a main method to invoke ?
How would I get a handle to the BootLoader
Lots of questions
I don't have a clear picture of that in my mind
Well. I'll come back to that later
thanks for your thoughts dmlloyd
4:05
dmlloyd
JBM doesn't need a main method
think of it as a class loader factory
4:05
lincolnthree
Ahhhh …. interesting
4:05
dmlloyd
you just call reflectively in to whatever module when it's ready
like ServiceLoader does, or whatever
4:05
lincolnthree
I think the last time I looked at it … you couldn't do that
What's the starting API for which I need to use that kind of functionality?
the entry point to start exploring
4:06
dmlloyd
just create a ModuleLoader
that's really the starting point of the API
4:06
lincolnthree
Huh.
But in this case I would have no SystemModuleLoader, right?
4:06
dmlloyd
it's just that by default jboss-modules.jar creates one for you
well, all module loaders can have a system dependency
SystemModuleLoader isn't really a "thing"
you wno't have that initial filesystem-backed one though, no
4:17
lincolnthree
Huh… go figure.
ModuleLoader addonLoader = new AddonModuleLoader();
That's all it takes.
So Forge itself has a flat classpath when it is loaded
but the modules themselves do not
the addons
interesting
I guess that's the concept?
For some reason I had it in my head that you would need to be running in a modular environment for ModuleLoader to work
that makes things about 500x easier
if it is true
4:24
dmlloyd
I'd certainly love for everyone to run in a modular environment, but I recognize that it isn't always possible
but yeah that's the gist of it
4:25
lincolnthree
nice
im trying this right now
a few more hacks to make
but i think this could be the lynch-pin to OSGi
4:25
alesj left the room (quit: Quit: Leaving.).
4:27
alesj [~alesj@redhat/jboss/alesj] entered the room.
4:27
mode (+v alesj) by ChanServ
4:27
lincolnthree
also would simplify the build greatly
removing the need for modules assembly
well i'll be...
looks like it should work
gonna have to ruminate on this
thanks dmlloyd
question though
4:31
dmlloyd
np
go ahead
4:31
lincolnthree
will loaded modules also have exposure to what is on the classpath? I assume yes?
4:32
dmlloyd
yes, with the qualification that you'll have to add a dependency for that purpose
4:32
lincolnthree
or are subsequent loaded modules still truly isolated?
4:32
dmlloyd
since that dependency needs a comprehensive package list, you'll have to iterate the class path to assemble that information
4:32
lincolnthree
okay, so they are still isolated unless you add… what, the system dependencies module?
4:33
dmlloyd
they're isolated unless you add a system dependency - it's a dep type, not a module
4:33
lincolnthree
right
4:33
dmlloyd
even then you have to list the packages you want to be visible
so if isolation by default is your goal, that's probably a good thing
if you want the whole classpath exposed then like I said you have to resort to a dirty trick to do it
4:34
lincolnthree
I don't
I just want what is currently done by JBM when including the system module
4:34
dmlloyd
though there's a different sort of dirty trick that you can do which we don't use much: "fallback" dependencies
4:34
lincolnthree
which i imagine means replacing the module.xml with some programatic stuff
4:34
dmlloyd
are you using 1.0.x?
system module has been gone since 1.1.0
replaced with system dep type
4:35
lincolnthree
checking..
1.1.3.GA
4:36
dmlloyd
okay then a system dependency is identical between XML and programmatic API
basically you list the packages you wanty
4:36
lincolnthree
javax.api module.xml:
<path name="javax/accessibility" />
<path name="javax/activity" />
<path name="javax/crypto" />
<path name="javax/crypto/interfaces" />
so just build that
4:36
dmlloyd
("java." is always available though, as is "sun.reflect.")
right
4:36
lincolnthree
got it
it's only 115 paths
no problem
nothing a little VI can't solve
--
Lincoln Baxter, III
http://ocpsoft.org
"Simpler is better."