Re: [forge-dev] Forge 2.0 and OSGi

Hi, I'd like to share my point of view here as well. Before doing that, let me state some things: * I don't like OSGi, but I will try to be as objective as possible * I have some (though little) experience with OSGi and with some of its Eclipse's extensions (tycho, p2, equinox) In my opinion, if you are an application developer (or in our case, plugin developer), you should take care about OSGi as little as possible. You just need to [@]Inject your dependencies to the core and forget. No nasty import packages declarations in manifest files, not even nastier [declarative] services declaration via OSGI-INF.XML or even worse - looking them up from the service tracker (I think the JNDI was forgotten in Java EE 5?). Yes, I also heard that there were some efforts that DI for Java, or even CDI annotations were brought to OSGi. But are you sure that it will work right now? That was about plugin development. As for the server (or in our case Forge core) development, I am sure we definitely need some kind of a modular runtime. Its basic idea is to take the burden of classloading and version management from us. It doesn't matter whether it is OSGi, JBoss Modules or whatever you say. Just take this burden out of Forge. So we have to now decide which solution better suites us. If we decide to choose OSGi, then let's speculate on how we build and assemble Forge. We are apparently using Maven for that. But Maven understands its own dependencies, not those in OSGi's manifests. So we have three options here: 1) Duplicate the dependencies declarations in pom.xml and in manifest files. Stupid I would say... 2) Use Maven's bnd plugin to create our manifest files based on the dependencies declared in the pom.xml. I'm not sure though whether this plugin would allow us to import and export only the packages that we want to. By default it imports everything and (which is worse) exports everything 3) Use Maven's tycho plugin. It teaches Maven how to use your manifest files, so you don't need to declare dependencies in pom.xml in parallel to importing packages. However with tycho you enter the Eclipse world of OSGi. Which means that we have to create a special target platform maven project. And should rewrite our dist project from scratch. And we'll have to find special repositories for all our dependencies (standard Nexus repositories don't work to my knowledge, they have to support the p2 protocol). And will have to pray to God every time we run a build that our dependencies are found. Because guessing the reason for a failure to find a dependency is sometimes harder than guessing the numbers of the lottery. My opinion: the worst option! So, after this long post, my final question is: what is that you don't like in JBoss modules so that you want to replace it with OSGi? Because I also think that having them both working together is not necessary and is a bit overkill. Cheers, Ivan On Tue, Sep 25, 2012 at 1:05 PM, Paul Bakker <paul.bakker.nl(a)gmail.com&gt; wrote: > Let's make sure I understand your question. The scenario would be as follows: > -Forge core services have some state shared with plugins > -There are plugins using this service > -A core service gets unregistered (software update for example) > -What happens with the plugin? > > I would say, the plugin should get unregistered as well, and register again as soon as the core service comes back. This might sound tricky (it is if you do service registration by hand), but is trivial when using something like Felix Dependency Manager. > If the shared data is in-memory it will be lost when the service re-registers, but you could serialize it to the bundle cache and deserialize when the service comes back. > The important concept is that you don't HAVE to deal with services not being available, the most used scenario is to just deregister all dependent services as well, which will degrade application functionality (at least temporally), but that is no problem for things like updates. > > As another concrete example, in the app I'm working on we have a lot of JAX-RS services. Those services often use other services to get to data (e.g. mongo services). Both the JAX-RS services and mongo DOAs are OSGi services. When a Mongo DAO bundle gets updated it's service get unregistered. At that moment, the JAX-RS service will be automatically unregistered as well (making the resource unavailable). When the update is complete, the Mongo service is registered again, and the JAX-RS resource will be registered again as well. > > Paul > > > On Sep 25, 2012, at 10:21 , Max Rydahl Andersen <max.andersen(a)redhat.com&gt; wrote: > > > > > On 25 Sep 2012, at 09:39, Paul Bakker <paul.bakker.nl(a)gmail.com&gt; wrote: > > > >> Semantic versioning is a lot easier when using OSGi I think, so in that aspect it will help with the versioning problem. > >> I don't agree on the services part; I'm working on large OSGi applications right now and that's ALL services (hundreds of them), and that works fine (as well a "normal" DI solution would). > > > > yeah, i'm not saying its not useful for some systems. > > > > I haven't seen a good example of a plugin system that allows extensions to access its datastructures and still be replacable dynamically ....you got examples of that ? > > > >> Eclipse is the best known example, but also the worst example of using OSGi. There is a lot of things in Eclipse that are considered anti-patterns in the rest of the OSGi world. Often with good (sometimes historic) reasons, but don't look at it as the ideal OSGi application ;-) > > > > neither do I - but its at least one data point in how even if you use osgi it doesn't really help you if your app has type data sharing needs. > > > >> Good point about JSR-330. On top of that; there is a new OSGi spec in development (spec lead is a JBoss guy I believe) that makes CDI available in OSGi. There is already a prototype in Weld. > > > > yes, and just for kicks, here is a footnote about JSR-330 in eclipse4. > > > > This was around the time google guice/Seam was popular and the JSR-299 / 330 "battle" was on. When I asked why they chose JSR-330 over JSR-299 (since IMO JSR-330 is not sufficient for what you need inside an UI app) there answer back then (2years?) was that Google and Spring supported JSR-330 and JSR-299 wasn't possible to use because it has no official backing and was tied to JEE. > > > > Then the conversation moved to how they were hitting the limits of JSR-330 not having an event model, not having a way to define boundaries of the injection and so forth. > > > > It was then fun to point out JSR-299 actually had more backing than JSR-330 and that JSR-299 actually had an even model and that at least Weld could be looked at to see if it could help to their "injection boundaries".....but yeah, the tied to JEE was at that time not easy to hide even though Weld could load on JavaSE. > > > > In the end they ended up doing their own "mutated" version of JSR-330 + the "missing features" > > > > I hope to see CDI available on eclipse equionox one day - would be fun to see it compare. > > /max > > > > Paul > >> > >> > >> On Sep 25, 2012, at 9:31 , Max Rydahl Andersen <max.andersen(a)redhat.com&gt; wrote: > >> > >>> btw. Eclipse Osgi and the Eclipse 4 platform uses JSR-330 style injection for their stuff. worth taking a look at if you are going this way. > >>> > >>> does osgi solves tooling multiple version problems ? nope - osgi's notions aren't really helpful in having a tool that can work reliable against multiple runtimes. > >>> since osgi is used in eclipse tooling it allows install/uninstalls ? Nope - since state sharing is needed in java code they cannot do this cleanly - and moving it all to services - the code would be super tricky to get right in my experience (Paul hints at this in his "theoretically speaking") > >>> does osgi help users ? nope - they don't get their features faster because of this. > >>> > >>> /max > >>> > >>> On 25 Sep 2012, at 09:16, Paul Bakker <paul.bakker.nl(a)gmail.com&gt; wrote: > >>> > >>>> Let's start by explaining something about OSGi. When designing towards modularity there are two parts of OSGi that are important: > >>>> 1) the classloading mechanism. > >>>> Bundles (modules) declare their requirements on other packages (imports) in a manifest file. Bundles can also export packages by declaring them in the manifest so that they become available to other bundles. When the OSGi container is started the resolver will check if all declared imports are available (exported by another bundle), if not, the bundle will not start. The resolver supports versions and ranges of version, and it's possible to have several versions of the same package installed. This mechanism makes sure that you will never get classnotfoundexceptions because the resolver checks if all requirements are fulfilled, and makes sure that implementation classes can simply be hidden by not exporting them; perfect for making sure others only use your public APIs. Also note that you never write the import headers yourself (this would be a lot of work), they should be calculated by tools (bnd) instead. Exports are defined yourself however. > >>>> This mechanism is similar to what JBoss Modules is doing; although it's doing a little more (which makes the resolver a little slower). Because startup speed is on the most important aspects of AS7 the resolver overhead during startup was not acceptable. I didn't see the numbers so I can judge on this, but I have seen many other very large applications that work fine (and start up A LOT quicker then Forge...) > >>>> > >>>> Also note that just having separate classloaders doesn't give you real modularity yet, for that we need services... > >>>> > >>>> 2) dynamic services > >>>> A service is simply a class registered as a service; compare this to a class registered/picked up to the CDI bean manager. A service can be injected in other services, and in practice the model works very similar with other dependency injection solutions. Implementation code that should be available to others (using an API) should be published as a service; the client uses the service by it's API and simply injects the service. What makes services different from other dependency injection solutions is that they can be dynamically registered and de-registered; they can come and go during runtime. Of course this just happens randomly, there are good reasons to do this. For example, a bundle is re-installed (new version), or uninstalled. The services of this bundle will also become unavailable. Or you might pick up new configuration somewhere that should start new services for example. > >>>> What happens to clients of a service that is de-registered though? In most cases you would want to de-register the client (a service itself as well) as well. In other cases you might just switch to another service that publishes the same API, or just continue work without the service. This sounds way more complex than it is; you solve these things by simply configuring this in your dependency injection framework, and everything will be handled automatically. > >>>> > >>>> So theoretically speaking, a Forge implementation in OSGi would require the following: > >>>> -Define all Forge APIs in one or more API bundles > >>>> -Implement the core as OSGi services. > >>>> -A plugin is also an OSGi service, and implements a plugin interface. They are registered "whiteboard style". > >>>> -A plugin injects core features by injecting OSGi services (remember, this is just DI similar as we do now) > >>>> -Plugins can be dynamically installed/de-installed/reloaded by simply reloading the bundle and registering the service > >>>> > >>>> So yes, the architecture of Forge fits OSGi perfectly. > >>>> > >>>> In my opinion OSGi is not too complex. The spec is large, but that is because it's a spec that is being used over 10 years now! It supports many corner cases, and if you would have to understand each corner case you could definitely say it's complex. You don't have to however, in most cases you really only deal with the basics and they are easy. Compare it to the CDI spec. The CDI spec has many tricky corner cases as well, but most users don't even know about those corner cases, and this is fine. If you would implement your own OSGi container you would have to deal with the complexity, but by just using OSGi you don't. > >>>> > >>>> So should we start using OSGi? Well this is a difficult question... In my opinion we should have started that way, but it might be a little late for that. OSGi shouldn't be hacked on top of Forge, that's just going to bring more problems. Forge itself should be build in OSGi, but this will of course change things. The programming model would be very similar, but the exact APIs etc. would be considerably different. I'm not sure this is feasible at this point. > >>>> > >>>> If you want to see the basics of programming in OSGi (with dynamic services etc.) take a look at this talk: http://parleys.com/#st=5&id=3361&sl=0. Skip to the part where I start coding :-) > >>>> > >>>> Paul > >>>> > >>>> > >>>> > >>>> On Sep 25, 2012, at 2:23 , Dan Allen <dan.j.allen(a)gmail.com&gt; wrote: > >>>> > >>>>> ...which is pretty consistent with my point as well. It's too over-engineered (as are the arguments as to why it isn't) for low-level containers. But the need (to live in this world) is nothing a layer of abstraction can't satiate :) > >>>>> > >>>>> -Dan > >>>>> > >>>>> On Mon, Sep 24, 2012 at 6:13 PM, JFlower <fiorenzino(a)gmail.com&gt; wrote: > >>>>> i remember this post: > >>>>> > >>>>> I've rarely in my life met a more overengineered, overcomplex spec than OSGi. Compatibility with OSGi should be an anti-requirement in any sane world. > >>>>> Unfortunately we live in this world :-( > >>>>> > >>>>> Gavin King > >>>>> > >>>>> [http://in.relation.to/22155.lace] > >>>>> > >>>>> Fiorenzo > >>>>> > >>>>> PS i never used osgi in my jee apps, and i hate my eclipse when i need to restart it after a plugin installation... > >>>>> > >>>>> 2012/9/25 <ggastald(a)redhat.com&gt; > >>>>> Hello, > >>>>> > >>>>> I've been done some thinking and researching for Forge 2.0 based on the last forge meeting we had and the current code in the 2.0 branch, and it seems that the architecture we're looking for is very close to OSGi architecture itself (regarding to plugability and modularity). > >>>>> > >>>>> I'm also afraid that we'll face the same problems that OSGi tries to solve. As my current experience with OSGi is next to minimal (and probably to better understand why and have some arguments if someone asks me about it), I would like some opinions about the advantages/disadvantages of why not having the Forge container as an OSGi compliant solution. > >>>>> > >>>>> Also, don't get me wrong: I am not trying to convince anyone of using OSGi into the forge core, just want to understand better why this architecture is not a viable solution so far. I know Lincoln is against using it, but I just want some arguments in case someone asks me in conferences and stuff :) > >>>>> > >>>>> Of course, we need to keep using CDI and annotations as well. So if it's possible to have that and at the same time the modularity (and plugability) offered by OSGi, it would be awesome. > >>>>> > >>>>> Looking forward for your answers ! > >>>>> > >>>>> Best Regards, > >>>>> > >>>>> -- > >>>>> George Gastaldi | Senior Software Engineer > >>>>> JBoss Forge Team > >>>>> Red Hat > >>>>> > >>>>> _______________________________________________ > >>>>> forge-dev mailing list > >>>>> forge-dev(a)lists.jboss.org > >>>>> https://lists.jboss.org/mailman/listinfo/forge-dev > >>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> forge-dev mailing list > >>>>> forge-dev(a)lists.jboss.org > >>>>> https://lists.jboss.org/mailman/listinfo/forge-dev > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Dan Allen > >>>>> Principal Software Engineer, Red Hat | Author of Seam in Action > >>>>> Registered Linux User #231597 > >>>>> > >>>>> http://google.com/profiles/dan.j.allen > >>>>> http://mojavelinux.com > >>>>> http://mojavelinux.com/seaminaction > >>>>> > >>>>> _______________________________________________ > >>>>> forge-dev mailing list > >>>>> forge-dev(a)lists.jboss.org > >>>>> https://lists.jboss.org/mailman/listinfo/forge-dev > >>>> > >>>> _______________________________________________ > >>>> forge-dev mailing list > >>>> forge-dev(a)lists.jboss.org > >>>> https://lists.jboss.org/mailman/listinfo/forge-dev > >>> > >>> > >>> _______________________________________________ > >>> forge-dev mailing list > >>> forge-dev(a)lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/forge-dev > >> > >> > >> _______________________________________________ > >> forge-dev mailing list > >> forge-dev(a)lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/forge-dev > > > > > > _______________________________________________ > > forge-dev mailing list > > forge-dev(a)lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/forge-dev > > > _______________________________________________ > forge-dev mailing list > forge-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/forge-dev _______________________________________________ forge-dev mailing list forge-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/forge-dev