[forge-dev] Regarding the current JBoss Modules prototype of Forge 2.0

hey folks question about JBM and MSC who's around? 3:12 dmlloyd the usual 3:14 lincolnthree awesome So Question is: What exactly does MSC provide on top of JBM, and can it be used outside of AS7? 3:15 dmlloyd nothing, the two are wholly independent 3:15 lincolnthree Okay… so what does MSC provide? 3:15 dmlloyd a concurrent dependency-based service container it starts and stops things not likely to be useful outside of the app server 3:16 lincolnthree Why not? (Thinking Forge) 3:16 dmlloyd because all an app server is, is services and management if you pull in services to your project, you're duplicating the app server 3:16 lincolnthree Forge is just Plugins (services) and a UI layer of some sort And I'd hope I'm not duplicating the app-server, because I don't need 99% of what it delivers to users. 3:17 dmlloyd are you saying you want to use MSC? it's not really a box to check - you'd only use it if you really need asynchronous concurrent service start/stop 3:18 lincolnthree I'm saying I want to know if it would make any sense, and if it would address any of the problems currently being discussed by Forge I need to be able to install a plugin, have forge bring up other plugins that may depend on it stop a plugin 3:18 dmlloyd also, AS8 will likely use a largely rewritten version of MSC that is transactional 3:18 lincolnthree bring down all dependent plugins 3:18 dmlloyd so 1.x has a limited future lifespan I would expect 3:18 lincolnthree allow plugins to depend on other plugins in a modular environment (Which I have working right now via a naive proxy system and some dynamic module configuration) Basically I am comparing JBM & MSC to OSGi We are at a crossroads here of giving up JBM in favor of OSGi 3:19 dmlloyd I wouldn't recommend that - OSGi is really shit 3:19 lincolnthree And I need to know which path makes most technical sense for my project 3:20 dmlloyd people keep blindly adopting it, and by the time it starts stabbing them in the back they have already convinced themselves that they're buzzword-compliant or whatever it is that causes people to use it it's like struts everyone used it even though it's complete shit mob psychology I guess 3:22 lincolnthree so maybe you should be involved onthe forge dev list because the OSGi people are gaining momentum 3:23 dmlloyd maybe, though my time is already limited 3:23 lincolnthree And I'm listening because it sounds like it can address a lot of our problems (in combination with weld OSG 3:24 dmlloyd I'm not going to sell these projects to you. I can only say that OSGi is a mistake that will cost you in the mid- to long-term, and let you make your own choice 3:24 lincolnthree I can't make an educated choice because there is no documentation about the alternatives. Which is why I am here with you now. 3:24 dmlloyd I'm not saying the MSC/Modules is a monolithic alternative either the two projects are associated by their usage in AS but they're by no means connected in any technical manner 3:25 lincolnthree You are basically the single point of knowledge for JBM (Aside from a few others at JBoss and myself included I suppose at this point) 3:25 dmlloyd I dunno, the docs spell out the basics pretty well I think 3:25 lincolnthree Basics yes. 3:25 dmlloyd modules is only basics 3:26 lincolnthree But you're telling me "Don't use OSGi" without explaining an alternative to the service layer Which is why I'm asking about MSC. 3:26 dmlloyd the alternative is to do something else even a simple start/stop with naive dependency queueing is probably better long-term just because you'll have control over it, instead of putting your faith in the genie 3:26 lincolnthree Hm. 3:27 dmlloyd MSC isn't a good long-term choice just because 2.x won't be compatible with 1.x it's really meant to be the innards of AS and that's about it 3:27 lincolnthree Will MSC2 be any different? 3:27 dmlloyd you could still use it as long as you don't make the API public though well as always I hope MSC2 will be "the one that sticks" 3:28 lincolnthree The API for none of this will ever be public 3:28 dmlloyd I think the odds are better, as we understand the requirements a lot better now that MSC 1 has been in the wild for a while 3:28 lincolnthree No matter what we choose, the programming model that plugin developers will see is CDI With some annotations. 3:29 dmlloyd it has (will have) the benefit of being transactional too, so you can "undo" whole multi-step operations MSC 1's chief problem is that you can't really know when it's "done" with some logical operation which is fine in an environment like AS5/6 where you just put deployments in as you go without any verification step of any sort and that's why it is the way it is but it turns out when you have a server control API like AS7 does, people want to know how their operations turned out, and revert them if stuff breaks 3:30 lincolnthree Hm. Basically here's the problem I want to solve: Forge has plugins (services) Right now everyone has to re-invent anything because there's no way of accessing a service from Plugin A in Plugin B So there's no way to create building blocks, like unix applications I need Plugin A to be able to use and depend on Plugin B's exposed APIs And that sounds a lot like what OSGi provides 3:32 dmlloyd well accessing their APIs is not the same as accessing their objects 3:32 lincolnthree Explain I'll clarify Since API's are objects too 3:33 dmlloyd only conceptually physically, APIs are classes if you want access to an API, modules can create class loading dependencies without any problems 3:33 lincolnthree Plugin A has (MathService) - Plugin B uses (MathService from A) 3:34 dmlloyd if you want to manage service lifecycle and manage injections and dependencies in that regard, MSC gives the low-level stuff to do that (we use it as the basis for Java EE injection for example) 3:34 lincolnthree Yes. 3:35 dmlloyd if you want real DI based on annotations and descriptors you need something fancier - and even CDI is pretty bumpy when it comes to DI between separate "modules" (not jboss modules, just regular modules in the EE/CDI sense) in an AS application for example users tend to use EE injection for that 3:35 lincolnthree This si what I am doing right now: https://github.com/forge/core/tree/2.0/plugin-container/src/main/java/org... @Service and @Remote annotations control injections And you define your dependencies manually in a config file. Everything from Plugin A is explosed to Plugin B, if plugin B depends on Plugin A This could be more fine grained but for now that's how it works 3:38 dmlloyd for class loading dependencies, that's probably fine 3:38 lincolnthree But only @Services are injectable 3:38 dmlloyd I've yet to see a user legitimately need fine-grained filtering in the real world 3:38 lincolnthree into another weld container Each plugin has its own weld container 3:39 dmlloyd sounds reasonable so far 3:39 lincolnthree what actually gets injected is a proxy which is both lazily instantiated when the dependent service becomes avaialble, and also intercepts method calls which set the TCCL to the ClassLoader of the module from which the service was injected (and resets it on completion) 3:40 dmlloyd sounds just like EJB 3:40 lincolnthree lol, basically excepte i have no idea how EJBs really work 3:40 dmlloyd you inject proxies they have interceptor stacks which do things like set TCCL 3:41 lincolnthree now i feel "special" cool well… in the end, I guess what is really missing from this whole picture is the dependency management and lifecycle stuff bringing up dependencies in the right order, as needed because right now all the containers come up at the same time 3:42 dmlloyd basically that problem boils down to: a service can start when all its dependencies are satisfied think of it as a counter 3:42 lincolnthree and the proxies just "start working" as services come up 3:42 dmlloyd the counter stores the number of unsatisfied dependencies when something finishes, it ticks the counters of all its dependents when those counters hit 0, that dependent can start 3:43 lincolnthree right. makes sense 3:43 dmlloyd that's basically all there is to it MSC takes this idea and goes nuts with it 3:43 lincolnthree i kind of wonder how much more what I have actually needs 3:43 dmlloyd makes everything concurrent, with injection, lots of modes like ON_DEMAND and so on, complex error handling 3:43 lincolnthree ON_DEMAND would be nice 3:44 dmlloyd MSC 2 will be simpler but also more complex 3:44 lincolnthree I guess I could do that too 3:44 dmlloyd simpler in that there are fewer states for a service, but more complex in that it's transactional 3:44 lincolnthree not sure i really need transactional 3:44 dmlloyd so you'd open a txn, add all your services (which in turn might add more services or do other things like stopping services), then prepare and if there are no errors, commit 3:45 lincolnthree hm 3:45 dmlloyd if there are errors, report and roll back 3:45 jamezp Couldn't you use to rollback a failure to load a plugin lincolnthree? 3:45 dmlloyd well, let's not go selling vaporware just yet 3:45 lincolnthree jamezp: well… what happens is the plugin just doesn't load 3:45 dmlloyd I still don't actually have a working prototype 3:45 lincolnthree nothing else blows up unless it depends on that plugin 3:46 dmlloyd but, I need one within about two weeks, so... 3:46 jamezp has faith in dmlloyd 3:46 lincolnthree but if it does, then i assume if I get this all wrking, the dependency would also not come up 3:48 dmlloyd yeah dependents as a rule will just sit there until their dependencies appear in AS5/6, you'd just remove broken stuff and put in the right stuff and it would (hopefully) all fix itself that's the MSC 1 "way" in MSC 1, things actually *do* fix themselves once you correct dependency issues also you can "poke" failed services to make them try again like if there's a port binding conflict, something you can fix manually 3:50 lincolnthree interesting one downside of JBM is that I can't run it from an eclipse launch very easily because the module path needs to be configured module library, etc which is of course, sensible 3:51 dmlloyd well, yes and no 3:51 jamezp Sounds like you need a plugin for that 3:51 dmlloyd you could for example have a flat-classpath application which uses jboss modules one of the things I'm doing for modules 1.2 is making it more friendly for this kind of thing (a lot of bootstrappy stuff happens in main() and it should be factored out) but iirc it's possible to do iow you use modules for your loaded plugins and a flat classpath for the static stuff mainly I'm working on that to make testing easier, but it probably applies here too 3:53 lincolnthree that would be really nice any idea when that might happen? that would make forge development a LOT easier 3:53 dmlloyd it should already be possible just by using it you might have to hack around a quirk or two though won't know for sure until it happens 3:55 lincolnthree hm. Interesting… So then I would need to supply a module library as an argument to JBM when I started it from my class and basically just call JBM main from some other class? But then JBM would also need a main method to invoke ? How would I get a handle to the BootLoader Lots of questions I don't have a clear picture of that in my mind Well. I'll come back to that later thanks for your thoughts dmlloyd 4:05 dmlloyd JBM doesn't need a main method think of it as a class loader factory 4:05 lincolnthree Ahhhh …. interesting 4:05 dmlloyd you just call reflectively in to whatever module when it's ready like ServiceLoader does, or whatever 4:05 lincolnthree I think the last time I looked at it … you couldn't do that What's the starting API for which I need to use that kind of functionality? the entry point to start exploring 4:06 dmlloyd just create a ModuleLoader that's really the starting point of the API 4:06 lincolnthree Huh. But in this case I would have no SystemModuleLoader, right? 4:06 dmlloyd it's just that by default jboss-modules.jar creates one for you well, all module loaders can have a system dependency SystemModuleLoader isn't really a "thing" you wno't have that initial filesystem-backed one though, no 4:17 lincolnthree Huh… go figure. ModuleLoader addonLoader = new AddonModuleLoader(); That's all it takes. So Forge itself has a flat classpath when it is loaded but the modules themselves do not the addons interesting I guess that's the concept? For some reason I had it in my head that you would need to be running in a modular environment for ModuleLoader to work that makes things about 500x easier if it is true 4:24 dmlloyd I'd certainly love for everyone to run in a modular environment, but I recognize that it isn't always possible but yeah that's the gist of it 4:25 lincolnthree nice im trying this right now a few more hacks to make but i think this could be the lynch-pin to OSGi 4:25 alesj left the room (quit: Quit: Leaving.). 4:27 alesj [~alesj@redhat/jboss/alesj] entered the room. 4:27 mode (+v alesj) by ChanServ 4:27 lincolnthree also would simplify the build greatly removing the need for modules assembly well i'll be... looks like it should work gonna have to ruminate on this thanks dmlloyd question though 4:31 dmlloyd np go ahead 4:31 lincolnthree will loaded modules also have exposure to what is on the classpath? I assume yes? 4:32 dmlloyd yes, with the qualification that you'll have to add a dependency for that purpose 4:32 lincolnthree or are subsequent loaded modules still truly isolated? 4:32 dmlloyd since that dependency needs a comprehensive package list, you'll have to iterate the class path to assemble that information 4:32 lincolnthree okay, so they are still isolated unless you add… what, the system dependencies module? 4:33 dmlloyd they're isolated unless you add a system dependency - it's a dep type, not a module 4:33 lincolnthree right 4:33 dmlloyd even then you have to list the packages you want to be visible so if isolation by default is your goal, that's probably a good thing if you want the whole classpath exposed then like I said you have to resort to a dirty trick to do it 4:34 lincolnthree I don't I just want what is currently done by JBM when including the system module 4:34 dmlloyd though there's a different sort of dirty trick that you can do which we don't use much: "fallback" dependencies 4:34 lincolnthree which i imagine means replacing the module.xml with some programatic stuff 4:34 dmlloyd are you using 1.0.x? system module has been gone since 1.1.0 replaced with system dep type 4:35 lincolnthree checking.. 1.1.3.GA 4:36 dmlloyd okay then a system dependency is identical between XML and programmatic API basically you list the packages you wanty 4:36 lincolnthree javax.api module.xml: <path name="javax/accessibility" /> <path name="javax/activity" /> <path name="javax/crypto" /> <path name="javax/crypto/interfaces" /> so just build that 4:36 dmlloyd ("java." is always available though, as is "sun.reflect.") right 4:36 lincolnthree got it it's only 115 paths no problem nothing a little VI can't solve