[jboss-jira] [JBoss JIRA] (WFLY-618) TS: (Re-)Create test group for web profile.

TS: (Re-)Create test group for web profile. ------------------------------------------- Key: WFLY-618 URL: https://issues.jboss.org/browse/WFLY-618 Project: WildFly Issue Type: Sub-task Reporter: Ondrej Zizka (20:46:10) Nihility: OndrejZizka: got a few seconds to talk testsuite (20:46:10) smcgowan: baileyje: http://pastebin.test.redhat.com/67894 (20:46:20) OndrejZizka: Nihility: Sure (20:46:23) jharting opustil místnost (quit: Quit: Leaving.). (20:46:54) Nihility: OndrejZizka: ok so for the beta release i need to resurect the old default profile "the web profile" and make the existing one the "full" profile (20:47:17) Nihility: OndrejZizka: i had some long conversations with various folks about how to do that without screwing up the progress that has already been made with the testsuite (20:48:06) Nihility: OndrejZizka: it seems that the best solution was to have a property which tells the testsuite to run against the "web" profile, using the different config, and also disabling tests using a pattern match (20:48:27) Nihility: OndrejZizka: so basically we would have to manually kick off a web run, or have a hudson run or something (20:49:04) Nihility: OndrejZizka: did you have any thoughts on that? I had originally hoped we would use different configs for different tests, but that seems to be a nightmare in our current maven configuration (20:49:28) Nihility: OndrejZizka: or any other ideas would be welcome (20:49:32) OndrejZizka: Nihility: Have you seen my last post in jboss-as7-dev? (20:49:59) OndrejZizka: Nihility: I have modified the testsuite a bit, so I believe this task is easier to do with it now too, (20:50:07) OndrejZizka: I only need someone to fix OSGi tests (20:50:41) Nihility: OndrejZizka: all i have is "printing project number" (20:50:42) OndrejZizka: After that, web profile vs. full profile can be simply a profile activating different modules (20:51:35) OndrejZizka: Nihility: Forwarded (20:51:46) Nihility: OndrejZizka: i mean do you think the different config per test module is reasonable? (20:51:59) Nihility: OndrejZizka: it seemed like we would have to do a BIG resturcturing for that (20:52:23) OndrejZizka: Nihility: Depends on what are the differences, and for which modules, (20:52:45) OndrejZizka: Inside the integration module, the differences are not significant, (20:53:01) OndrejZizka: the modules are used to separate tests virtually (20:53:07) Nihility: OndrejZizka: yeah thats the sisue the integration module mixes full and non-full (20:53:59) OndrejZizka: Nihility: Could you send me a list of which modules are non-full? And also, the config for web AS profile (20:54:15) OndrejZizka: or a list of differences (20:54:40) Nihility: OndrejZizka: im working under the assumption we have to run twice (i figured there was no time to restructure to that level) (20:55:23) Nihility: OndrejZizka: yeah sure, its basically standalone but without jacorb, hornetq, and jaxr (20:55:34) Nihility: OndrejZizka: and webservices (20:55:55) OndrejZizka: Twice? There are two executions of surefire, which is because some tests test some persistent stuff (20:56:43) Nihility: OndrejZizka: no i dont mean automatically run twice, i mean the user has to say something like ./build.sh -Dweb-profile or something (20:57:01) Nihility: OndrejZizka: to be honest i dont like that solution as much as doing a config per module (20:57:28) OndrejZizka: Nihility: I think this could be solved too (20:57:31) Nihility: OndrejZizka: but the config per module sounds like there is just not enough time, we need to get teh beta out this week (20:58:20) smcgowan opustil místnost (quit: Quit: Leaving). (20:59:02) OndrejZizka: Nihility: Check this (20:59:03) OndrejZizka: https://github.com/OndraZizka/jboss-as/tree/TS-modules-tmp/testsuite/inte... (20:59:29) OndrejZizka: It's basically done, only OSGi tests fail because there's some assumption about relative paths (20:59:38) Nihility: ah intersting (20:59:47) Nihility: so you use a maven module to activate profiles? (20:59:51) OndrejZizka: If someone fixes that, I think we could use that for the requirements you stated nickarls Nihility (21:00:13) OndrejZizka: Nihility: and vice versa - this way is the most flexible (21:00:47) OndrejZizka: Using profiles, I activate sets of modules, which define -DallTests, or a specific group; (21:01:17) OndrejZizka: then, in these modules, there are profiles which can be disabled, and also "global" profiles which change database config etc. (21:02:35) Nihility: ah ok so in theory (21:02:44) Nihility: we would have a group of "full" tests (21:02:47) Nihility: or something (21:03:48) OndrejZizka: Nihility: yes (21:04:16) OndrejZizka: Either as a module, or as a profile. (21:04:24) OndrejZizka: in the integration/basic module (21:04:28) OndrejZizka: I'd prefer the second, (21:04:43) OndrejZizka: but if it turns out there's some problem with maven, we can fall back to module-based selection (21:04:52) OndrejZizka: and they can run both in one run (21:05:35) OndrejZizka: I mean, running two executions of same tests using profiles is sometimes troublesome (21:05:52) Nihility: yeah and its time consuming nickarls Nihility (21:08:52) OndrejZizka: Nihility: So, do we need to run the web tests twice? (21:09:01) OndrejZizka: in this full/non-full scenario (21:09:16) OndrejZizka: Or in full, only the complement would be run? (21:09:42) OndrejZizka: * complementary tests (21:11:12) ctomc [~ctomc(a)upc.si.94.140.92.246.dc.cable.static.telemach.net] vstoupil do místnosti. (21:11:19) Nihility: OndrejZizka: well IMO we only need to run the "full-only" tests against standalone-full.xml (21:12:25) OndrejZizka: Nihility: Ok, that should be okay. So I'll prepare a PoC and ping you, okay? (21:12:33) OndrejZizka: Which TZ is thomas.diesler(a)jboss.com in? (21:13:03) Nihility: OndrejZizka: ok, hes in munich which i think is GMT+1 (21:13:33) Nihility: OndrejZizka: we should chat with stuartdouglas about this as well (21:16:11) maxandersen [~Adium@redhat/jboss/maxandersen] vstoupil do místnosti. (21:16:11) režim (+v maxandersen) od ChanServ (21:16:18) Nihility: OndrejZizka: so according to that email the issue he brings up is that we have duplicate sources? (21:17:50) stuartdouglas: I really don't like the 1 big ball of sources thing (21:18:14) stuartdouglas: I think it overcomplicates things, requires additional configuration, and may cause dependency and other problems down the road stalep stuartdouglas nickarls Nihility (21:22:56) OndrejZizka: stuartdouglas, Nihility, when such problems arise, there's no problem in separating them (21:23:31) OndrejZizka: cd testsuite/integration; mv src/test/java/foo/bar module/src/test/java/foo/bar (21:23:38) OndrejZizka: ;-) (21:23:57) OndrejZizka: Not counting resources (21:24:19) OndrejZizka: But I hope that tests keep their resources well separated (21:24:33) OndrejZizka: As per Jason's recent post to dev list (21:26:40) darranl [~darranl@redhat/jboss/darranl] vstoupil do místnosti. (21:26:40) režim (+v darranl) od ChanServ (21:27:47) OndrejZizka: stuartdouglas: Either way - pushing everything to a single module doesn't work, so the question is not whether to stay with current or split, (21:28:08) OndrejZizka: but whether to split the modules and keeps tests together, or split them "physically" (21:28:27) OndrejZizka: I have a) done, stuart likes b), can be done too (21:28:31) stuartdouglas: I think we should go with splitting them up into different modules, with the relevant source in each module (21:28:38) stuartdouglas: it just seems like the simplest approach (21:28:49) stuartdouglas: and it is also the 'standard' maven way (21:29:12) stuartdouglas: From your point of view, what are the downsides to doing it this way? (21:31:48) CheffPJ [~pmcdonou(a)219.sub-75-248-111.myvzw.com] vstoupil do místnosti. (21:32:26) Nihility: stuartdouglas: OndrejZizka assuming we did the split into smaller finer grained modules, would the activation thing still work (e.g. we create dummy modules that activate the other ones)? (21:33:02) Nihility: stuartdouglas: OndrejZizka or would we just use the shell scripts? navssurtani1 nickarls Nihility nickarls Nihility (21:33:53) OndrejZizka: Nihility: The separation needs was the original reason why I started this change nickarls Nihility (21:34:25) OndrejZizka: Nihility: So, the activation would be done by activating different profiles, which would include (21:34:42) OndrejZizka: <modules> <module>...</module> <module>...</module> <module>...</module> ... </modules>