[14:54] sebersole: smarlow im around just lately waste a great deal of my time on conference calls :) [14:55] sebersole: smarlow i dont see how anything can be done with modules to address the issue, unless you are able to make it psychic [14:55] :) [14:56] --> spagop has joined this channel (~spagop@ppp-93-104-15-160.dynamic.mnet-online.de). [14:56] Nihility: well one thing that keeps rolling around in my head is the notion of exporting for the sake of integration [14:57] *** rcernich is now known as rcernich_away. [14:57] since really, all this is spi-level stuff [14:57] sebersole: I'm available now if you want to talk [14:57] smarlow: ^^ [14:57] dmlloyd: thanks :) [14:58] that email chain was somewhat vague in terms of actual requirements [14:58] so I threw out a bunch of possible solutions to common situations [14:58] but the book is not exhausted by any means [14:58] i am really thinking that it is best to to not have each jar/module handle this itself [14:58] dmlloyd: i saw some of those [14:58] i just joined that mailing list though [14:59] like 5 minutes ago [14:59] the best approach for a given situation really depends on what it is you need to accomplish and why [14:59] dmlloyd: sure [14:59] so lets take the first one [15:00] Integrator, which is the service loader thing emmanuel asked about specifically [15:00] so Integrator is a contract defined in hibernate orm jars [15:00] other jars would contain impls [15:00] and have their serevice loader locator file [15:01] META-INF/services/org.hibernate... [15:01] the original suggestion was a bi-directional dep [15:01] are the implementations typically scoped to a be system-wide, per-application, or what? [15:01] which i think is awful (imho) [15:02] --> barreiro has joined this channel (~barreiro@2.83.128.38). [15:02] well thats a good question that i can't sum up in a single word :) [15:02] its per application's choice of orm and integrator combo [15:02] so per-application maybe [15:03] not sure how that fits in your terms [15:03] an example would be hibernate ogm [15:03] a complicating factor is that a module system is a general-purpose system which you could run any app on, but *our* module distribution is really just an AS distribution [15:03] which has one of these Integrators [15:03] so things you'd do in a general sense you'd maybe do differently for that reason [15:03] well i think the problems are general [15:04] i dont think they are specific to "this particular" module sustem [15:04] yeah but the solutions can be more pragmatic rather than more idealistic [15:04] --> mbabacek has joined this channel (~karm@redhat/jboss/mbabacek). [15:04] *** ChanServ gives mbabacek permission to talk. [15:04] though you obviously have more exp [15:04] right, I get what you're saying [15:04] like i was saying above.. [15:04] the elephant in the room is that there isn't a general-purpose implementation mechanism/registry [15:04] one general pattern i seem to see is that of a set of "intergation exports" [15:05] as oppsed to api exports [15:05] this is exactly the reason i decided to split hibernate into api/spi/internal [15:05] in terms of packaging [15:05] <-- ttarrant has left this server (Ping timeout: 265 seconds). [15:05] to help facilitate this [15:06] the spis need to be exposed for communication between "modules" [15:06] <-- mdobozy has left this server (Quit: Leaving.). [15:06] but they should not be exported to the app itself [15:06] if that makes sense [15:06] sure [15:07] the real problem i have with defining dep on both sides... [15:07] is custom modules, or customizing modules [15:07] now we have users mucking with AS provided modules [15:07] right [15:07] quick question, do you ever want users to be able to deploy say a different ogm or ORM? [15:08] but yet work with the containers opposite [15:08] Nihility: smarlow does :) [15:08] --> rhusar has joined this channel (~rhusar@ip-89-103-39-182.net.upcbroadband.cz). [15:08] i personally dont :) [15:08] well, what we have today is based on the fundamental assumption that we currently only have two practical/performant options for finding implementations: the API class loader and an implicit (TCCL) or explicit classloader parameter [15:09] so, everything has grown from there [15:09] <-- jcacek_wfh has left this server (Quit: Ex-Chat). [15:09] theres two ways you can go about that which have a large impact [15:09] --> oskutka has joined this channel (~ony@212-96-177-49.cust.selfnet.cz). [15:09] --> jjaggars has joined this channel (~jjaggars@12.237.225.202). [15:09] either you consider all of the hibernate stuff to be one fixed group [15:09] Nihility: if i were doing that, i'd personally rather have an explicit module file [15:09] (e.g. orm version A with ogm version B only) [15:09] and it would name orm version x and ogm version y [15:10] Nihility: yeah there is a problem witrh that [15:10] in that they do not follow the same release cycle [15:10] and then you allow overriding the set [15:10] but not the components [15:10] so versioning that gets complicated [15:10] sure, i get what you are saying [15:10] or you let both be replaced [15:10] and let the user figure out the right combination [15:10] they are "on there own" as it were [15:11] and we provide the recommended combination by default [15:11] Nihility: like I said, if i were doing that, i'd personally rather have an explicit module file [15:11] as part of the app [15:11] naming the explicit deps [15:11] orm version x, ogm version y [15:11] but [15:12] that still doesnt solve the underlying problem [15:12] the other question is do you want to support in-deployment components [15:12] like i deploy hibernate orm x.y.z [15:12] in that orm still could not see ogm classloader [15:12] and i expect it to wire up with ogm [15:12] which is provided by the system [15:12] Nihility: the problem is that that is not the nature [15:13] the nature of these is that you have orm [15:13] and ogm, via this Integrator, reconfigures orm [15:13] but [15:13] plenty of people use orm w/o ogm [15:14] in fact thats the 80% use case [15:14] 80%+ [15:14] it's not clear what the arity is here - is any of the behavior you're talking about global, or is it per instance of ORM (if there is such a thing)? [15:14] *** qmx is now known as qmx|away. [15:15] dmlloyd: for orm+ogm in particular? [15:15] yeah [15:16] yeah basically we should describe this from a users point of view. What should they be able to do (ideally) in your opinion [15:16] currently, app deployments could have their own instance of the ORM classes or use ORM via a module [15:16] well there its a combo [15:16] (also is there anything they can do now that they shouldn't be able to?) [15:16] by instance, I mean classloader [15:16] dont worry so much about the how, just if all magic worked how could they use this stuff [15:16] an app could (1) use orm, (2) use orm+ogm [15:16] Nihility: yes, trying :) [15:17] so in that orm+ogm combo [15:17] they might (a) use the orm and ogm modules [15:17] (b) use the orm module and supply ogm themselves [15:17] (c) supply both prm and ogm themselves [15:17] (d) supply custom module(s) [15:18] at least thats how i understand the possibilities in AS [15:18] --> rbalent has joined this channel (~rbalent@ip-62-245-108-79.net.upcbroadband.cz). [15:18] <-- rbalent has left this server (Changing host). [15:18] --> rbalent has joined this channel (~rbalent@redhat/jboss/rbalent). [15:18] *** ChanServ gives rbalent permission to talk. [15:18] git [jboss-as] push master 8c9cb5d.. James Perkins Updated/added translations [15:18] git [jboss-as] push master URL: http://github.com/jbossas/jboss-as/compare/f55e762...8c9cb5d [15:19] tbh, i personally think the app should name that it wants those modules as deps [15:19] git [jboss-as] push 7.1 8d4712f.. James Perkins Updated/added translations [15:19] git [jboss-as] push 7.1 URL: http://github.com/jbossas/jboss-as/compare/9d3beda...8d4712f [15:19] orm is somehow discovered to be needed [15:19] iiuv [15:19] iiuc [15:19] but thats not really possible with ogm [15:20] --> ALR has joined this channel (~alr@redhat/jboss/ALR). [15:20] *** ChanServ gives ALR permission to talk. [15:20] its a completely "behind the scenes" deal [15:20] <-- vjuranek has left this server (Ping timeout: 240 seconds). [15:20] --> vjuranek has joined this channel (~vjuranek@ip-89-102-30-62.net.upcbroadband.cz). [15:20] but we will certainly be doing some of what Nihility suggested [15:21] like envers really should be part of the hibernate orm module [15:21] it has this same Integrator deal [15:21] but it will be part of that module classloader, so that will be fine [15:21] --> pmuir has joined this channel (~pmuir@redhat/jboss/pmuir). [15:21] *** ChanServ gives pmuir permission to talk. [15:22] envers is released in cycle with orm, so that makes perfect sense to me [15:22] (that its part of the same module i mean) [15:23] dmlloyd: another suggestion i was told you made was to have ogm in this example register its classloader with orm [15:23] which i think is ok in the general sense [15:23] but [15:23] i think that should be done by the module itself somehow [15:24] it kinda sounds like (in the AS case at least) each deployment should be configurable with the exact combination of what pieces it wants [15:24] not siure if the module allows such "hooks" [15:24] dmlloyd: modules have just a single export correct? [15:25] yes and no [15:26] if you have a class in a module (or any other class loader), it can only ever be linked once [15:27] so this means that by extension, to give an example, an API implementation module can only ever link against one API module [15:27] ok [15:27] if you want two versions of the API module, you need two versions of the implementation module [15:27] now this has nothing to do with modules, it's just the way class loading works [15:27] now *that said* [15:28] sure [15:28] there's no reason that you cant have N implementations for a given API [15:29] dmlloyd: well more what i mean is whether a module could export its api and spi separately [15:29] i am going off osgi names [15:29] well for modules in particular, they always export everything; the filtering happens at import [15:29] sorry if that does not match up exactly with as [15:29] <-- kenfinnigan has left this server (Quit: Leaving.). [15:29] so you could introduce an intermediate module for example which filters down the full set of another module, if you wanted to [15:30] but normally I'd say if you have three JARs they should be three modules [15:30] well unfortunately we ended up having to go the other direction [15:30] to work around these problems [15:30] you have one JAR? [15:30] the hibernate orm module contains multiple jars [15:31] that's unfortunate, and actually shouldn't be necessary since you can do everything with three modules that you can do with one [15:31] hibernate orm is a project [15:31] it produces 6-7 jars [15:31] s/three/N/ [15:31] sure i knew what ya meant :) [15:32] basically it boils down to, fewer modules = fewer options [15:32] more options not always == good :) [15:32] <-- pferraro has left this server (Quit: pferraro). [15:32] often more options -> more confusion [15:33] I mean for you, not for users [15:33] whcih is where i feel we are [15:33] well [15:33] so lets explore that... [15:33] well I think the module organization is a separate question from how to interface this stuff to users, it's just that so far, messing with modules is just how it's been done [15:33] so today hibernate-envers is a separate module [15:34] and we have exactly this service loader problem there as well [15:34] we actually planned on moving it into the main orm module to work around that [15:35] we use the 1-1 bidirectional dependency currently or envers [15:35] we use the 1-1 bidirectional dependency currently *for* envers [15:35] fundamentally, using service loader by itself breaks down a bit when you've got multiple combinations of APIs and impls [15:35] which i wanted to do away with [15:35] it works well for e.g. jboss marshalling where you only have one or two implementations *ever* [15:36] dmlloyd: i disagree wrt service loader [15:36] i think it works perfect for how hibernate uses it as well [15:36] at the lowest level, if you want to be able to stitch together arbitrary pieces you have to be able to tell it what class loaders to get what pieces from [15:36] (for a given "instance" of the framework) [15:36] <-- ncross has left this server (Quit: Konversation terminated!). [15:37] you have to understand that Integrator is used to reconfigure parts of orm, prior to it filly starting up [15:37] fully [15:37] they are not used during runtime [15:38] your service loader example is a example of "supply us with some custom override behavior" [15:38] here, we truly might have multiple integrations [15:38] right well I should say "using service loader with the calling class loader" [15:38] should have said [15:39] well i dont do that either ;) [15:39] i have an extremely verbose bit of code trying to use multiple classloaders [15:39] --> ncross has joined this channel (~rnc@redhat/jboss/ncross). [15:39] *** ChanServ gives ncross permission to talk. [15:39] we built in the capability in hibernate for environment to pass in classloading behavior [15:40] generally that breaks down into it passing in multiple classloaders [15:40] so i have this capability already [15:40] it works great [15:41] the problem is lack of visibility in as7 right now between these things [15:41] the problem is lack of visibility between these things right now when deployed to as7 [15:42] that sounds better and more accurate :) [15:42] can you give a specific example? [15:42] i have been :) [15:42] orm and ogm [15:42] okay those are two JARs? [15:42] I'm just trying to understand the internal relationships [15:42] for the sake of this discussion. lets say yes :) [15:42] I assume that the ORM has an SPI portion which OGM implements directly? [15:42] there are mainly 2 jars involved [15:43] hibernate-core [15:43] hibernate-ogm [15:43] hibernate-core is a jar from orm [15:43] a core JAR it would seem :) [15:43] it contains this Integrator interface [15:43] lol [15:43] THE core jar [15:43] the ruling jar [15:44] anyway [15:44] hibernate-core knows nada about hibernate-ogm [15:44] its not dependent on it at all [15:44] hibernate-ogm however, has dep on hibernate-core [15:45] because it implements a number of spis that are defined in hibernate-core [15:45] however [15:45] hibernate-core needs to be able to "see" hibernate-ogm [15:45] to find its service locator [15:46] but only at run time right? not link time? [15:46] correct [15:46] and there could normally be other implementations? [15:47] well it also needs to be able to "see" it so it can load classes from it [15:47] dmlloyd: yep [15:47] okay, sounds like core should *not* have any kind of dep on ogm [15:47] correct [15:47] (postulating 2 modules for the moment) [15:47] dmlloyd: right or wrong, i always take this back to build depsa [15:48] in terms of visualizing [15:48] the module deps should in fact be similar to build deps [15:48] yep [15:48] i agree [15:48] so i am not far off there then... yaay! [15:48] it is true that we *do* use circular deps for certain things but nobody should think that they are *required* to for all similar situations [15:48] in this case it wouldn't be appropriate at all [15:49] but it does give rise to certain questions [15:49] i agree that circular deps have their place [15:49] infact we use that in the hibernate orm build! ;) [15:49] the obvious one being, how do you load classes from a certain implementation [15:49] yep [15:49] and I can tell you how *not* to do it: do *not* rely on the classloader of core, nor TCCL in this case [15:50] (to load classes from OGM) [15:50] we dont [15:50] to date we have been using bi-dir deps [15:50] but it sounds like we all agree those should be removed [15:50] yeah definitely [15:50] replaced with what though is the question :) [15:51] yeah and that really depends on how your SPI looks [15:51] https://github.com/hibernate/hibernate-orm/blob/master/hibernate-core/src/main/java/org/hibernate/integrator/spi/Integrator.java [15:51] thats the spi, pretty simple [15:52] ignore the second method [15:53] the basic idea is that it allows callbacks to reshape how the Hibernate SessionFactory will get built [15:53] okay, so this does happen in the context of a deployment then? [15:54] --> milestone has joined this channel (~milestone@ip-109-84-0-17.web.vodafone.de). [15:54] in context of building a SessionFactory [15:54] which *might* be deployment [15:54] aka, in the case of container managed EMF [15:55] or do you just mean in terms of scope? [15:55] <-- whitingjr has left this channel. [15:55] it will be scoped to that deployemnt that triggered the SF building [15:56] okay so we use SF for internal stuff sometimes, and deployments other times then? [15:56] so which implementations to select depend on various factors it seems [15:57] <-- rmartinelli has left this server (Quit: Leaving). [15:57] dmlloyd: an app could build a SessionFactory themselves [15:58] in the case of JPA, it could manage the EMF itself [15:58] EMF roughly equates to SessionFactiry [15:58] SessionFactory [16:00] dmlloyd: you mean which implementations to select in terms of Integrator? [16:00] if so then yeah [16:00] its really app specific [16:00] going back to build dep visualization [16:00] <-- vjuranek has left this server (Quit: Konversation terminated!). [16:00] its really up to them to name which depns they want [16:01] *** pilhuhn is now known as pil-zZzzzZZzzzz. [16:04] from a deployment perspective then, what we really need is an easy to use and concise means to spell it all out [16:04] yep [16:05] (for internal stuff we can just use whatever low-level APIs, doesn't matter as much) [16:05] org.hibernate:orm:{version}? [16:05] are modules versioned explicitly? [16:05] or just via name? [16:05] <-- adietisheim has left this server (Quit: Leaving.). [16:05] no, they're versioned by category or "slot" [16:06] I think we have a 3 and 4 category for hibernate (right smarlow?) [16:06] the categories would typically be mutually exclusive though [16:06] oh, ok [16:06] i.e. you wouldn't mix 3.x and 4.x [16:06] dmlloyd: currently yes, we have org.hibernate:main for 4.x and org.hibernate:3 for 3.x [16:06] maybe main should be orm? ;) [16:07] but I think that users probably shouldn't concern themselves with the module aspect, wouldn't you agree? [16:07] smarlow: but dmlloyd was recommending splitting that module up into one per jar [16:07] dmlloyd: just not sure of another way to do it [16:07] we can rename modules and maintain an alias for the old name (just saying, that's something that's possible if you wanted to do it) [16:07] well I mean we could have a DD or something [16:08] sure, but specifying wqhat? [16:08] *** balunasj is now known as balunasj|away. [16:08] something that says "I want hibernate 4.x. I want this OGM thing plus that thing plus that thing. Make it work" [16:08] whatever options the user has [16:08] but thats modules isnt it ;) [16:08] that's how it's implementated [16:08] implemented* [16:08] but not how it's typically specified [16:08] well you know that way better than me [16:09] currently, they could specify a PU property to indicate which provider module should be used. [16:09] and actuall smarlow has something sorta kinda similar today [16:09] I mean if what we're talking about is just giving them a choice between 3 and 4 then that's pretty simple [16:09] dmlloyd: sure [16:09] but its more than that [16:09] but if they need to be able to plug in impls, or their own accessory modules or whatever then that needs to be spelled out [16:09] it a choice between 3/4 + what integratiosn they wat [16:09] <-- galderz has left this server (Quit: Leaving). [16:10] its eclipselink/openjpa/datanucleus/other providers also [16:10] hibernate integrations? [16:10] no, he means alternate jpa providers [16:10] oh, you asking me [16:10] okay [16:10] well, yeah, ogm [16:10] we're not at JPA providers yet :) [16:10] envers [16:10] okay :) [16:11] <-- wolfc has left this server (Quit: Leaving.). [16:11] so its not just "i want hibernate 4" [16:11] its, "i want hibernate 4 and ogm" [16:11] or [16:11] "i want hibernate 4 and ogm and envers" [16:11] "i want hibernate 4 and ogm and envers and bean validation" [16:11] would things like envers or ogm have a 3.x vs 4.x variant? [16:12] yes [16:12] though [16:12] bean validation is different, we pass that in via properties [16:12] we dont have to support them i dont think [16:12] smarlow: no [16:12] okay so they'd say what major version of "stuff" they want, and then what other optional bits [16:12] you pass in the factory [16:12] the integrator still needs to run [16:13] sebersole: okay, if you mean choosing a different version, your right [16:13] and then we'd just put it into the oven and give them their SF [16:13] now we *could* do that with module names *but* the user would then have to have knowledge of what our module names are, and also of the various other flags they'd have to set on the module in order to make it so that the TCCL could be used to load all the pieces up [16:14] <-- milestone has left this server (Remote host closed the connection). [16:14] alternatively we could just set up a DD with a custom documented XSD that spells it all out in plain english (which also makes it possible to avoid "leaking" impls on to the deployment class path, if that's important to anyone) [16:14] dmlloyd: so the alt you think is (1) name the category (2) name the intg pieces? [16:15] so something like (1) i want hibernate 4 (2) I want orm, envers, ogm [16:16] as far as leaking, yeah the Integrator does have a "shutdown" piece if you notices [16:16] noticed [16:16] that should be called on undeploy, shutdown etc [16:17] <-- jfd has left this server (Ping timeout: 245 seconds). [16:18] dmlloyd: so is this DD handled explicitly by the hibernate deployer? [16:18] assuming there is still a hibernate deployer :) [16:19] --> jfd has joined this channel (jdoyle@nat/redhat/x-oajxfaslcvqnxoad). [16:19] <-- jfd has left this server (Changing host). [16:19] --> jfd has joined this channel (jdoyle@redhat/jboss/jfd). [16:19] *** ChanServ gives jfd permission to talk. [16:20] sebersole: no, we don't have a hibernate deployer [16:21] not presently [16:21] the as6 hibenate deployer is still there though :) [16:21] but yeah sebersole that seems to me like the best way to do it from the user's perspective [16:21] sebersole: i haven't rewriten it yet … this time :-) [16:21] the idea would be to make it something that is future-compatible so that we don't keep changing it on every release [16:22] we do this by avoiding specific member, class and module names and instead favor general descriptions and flags that make sense to the user [16:22] dmlloyd: what about a custom module that contains an Integrator [16:22] then saying "we will no longer support XYZ in AS 9" == deprecating a specific XML element or similar [16:23] well yeah in that case it would be allowed [16:23] I just mean for our stuff [16:23] our DDs should refrain from exposing internal implementation details whenever possible [16:23] 5/6 was majorly guilty of this [16:23] i just mean what it looks like in DD, would it name the custom module in that case? [16:23] yeah you'd probably say "give me envers, ogm, and this module "foo.bar"" [16:23] dmlloyd: hibernate still does that today :( [16:24] new jira [AS7-4858] jconsole.sh does not start on Mac OSX [Open (Unresolved) Feature Request, Major, Jason Greene] https://issues.jboss.org/browse/AS7-4858 [16:24] lot of impl class names in config settings [16:25] *** rcernich_away is now known as rcernich. [16:26] speaking from a place of total ignorance, if I were using OGM in my application, would that imply that I need access to additional API, or is it all just behind-the-scenes/configuration stuff? [16:26] (just picking out OGM as an example, but any integrator really) [16:27] alesj: ping [16:28] ctomc: wazup? [16:28] OGM is a JPA persistence provider, so I would expect javax.persistence would be one of APIs needed [16:29] okay. maybe that's a bad example then because they should get that API anyway given that it's a JEE thing [16:29] it seems like, generally speaking, the app doesn't really need to link to these pieces as long as the SF remembers how to access them [16:31] dmlloyd: for ogm, just configuration stuff [16:31] but [16:31] in the case of say envers, that is not the case [16:31] that does expose additional apis [16:31] okay, so it'd depend on the piece then? [16:32] it really only matters in the case of a custom module since the DD parser would have direct control over what it does with the other stuff [16:32] are you asking in terms of exporting to the app? [16:32] yeah [16:32] i think it would be safe to just add them as an export [16:32] okay. limiting pollution is somewhat desirable, but not worth going nuts over [16:33] dmlloyd: well lots of them add custom hibernate event listeners [16:33] the app *may* want to interact with those [16:33] its hard to say unequivocally that the app wont [16:33] sounds reasonable [16:33] in the case of some i can say it has to be exported [16:33] envers [16:34] validator [16:34] search [16:34] ogm... just not sure [16:34] better safe than sorry i think [16:34] but i hear what you are saying [16:35] <-- jfd has left this server (Ping timeout: 245 seconds). [16:35] as long as we don't use TCCL as the single rally point for everything we have options [16:35] https://github.com/hibernate/hibernate-orm/blob/master/hibernate-core/src/main/java/org/hibernate/integrator/internal/IntegratorServiceImpl.java [16:36] which makes use of this classLoaderService.loadJavaServices( Integrator.class ) call [16:36] https://github.com/hibernate/hibernate-orm/blob/master/hibernate-core/src/main/java/org/hibernate/service/classloading/internal/ClassLoaderServiceImpl.java [16:36] if you are interested [16:37] so, no, i go out of my way to avoid tccl [16:37] good. If only everyone would :) [16:37] <-- Jesmith17 has left this server (Remote host closed the connection). [16:37] sebersole, dmlloyd: I have to go offline, will catch up with you guys later... [16:37] well i take it back, i do use tccl as a fall back [16:38] but the AS integration passes me in classloader to use [16:38] so i dont get to that fallback [16:38] okay so what do you think of the DD idea? [16:39] i think its great [16:39] i think it absolutely has to be up to deployer input [16:39] deployer/developer [16:39] deployer overloaded i guess :) [16:40] develoployer! [16:40] lol [16:40] thanks for your time [16:40] <-- pmuir has left this server (Quit: Leaving). [16:40] no prob [16:41] <-- fharms has left this server (Ping timeout: 260 seconds). [16:42] <-- smarlow has left this server (Ping timeout: 244 seconds). [16:44]