Hi, we would like to do some refactoring of PicketLink integration testsuite for easier QE testing with new versions of EAP. We have some ideas how to improve it and I want to ask you what do you think about that. Here is the list, we would like to: 1) change of configuration which running specified tests for specified container. It is currently set by @TargetContainers annotation. We prefer to using better usage of maven profiles and for instance Java subpackages according to profiles. Every subpackage will contain tests which will run only in that profile (for example org.picketlink.test.trust.tests.eap6 will contain tests for EAP6 profile). Tests which run in every profile will stay in current packages. Tests which run in more profiles (but not in all of them) will be added by include/exclude parameters of maven-failsafe-plugin. Why? It will be easier to configurate it for QE testing. We need some easy way how to see each test of some profile. Currently we have all tests together and it's quite uncomfortable. It will be simpler to add a new container too. 2) avoid use of Ant and try to rewrite it to maven (using Maven Resource Plugin etc.). Why? We want to have ability of setting properties from command line (which is not handled correctly by maven-antrun-plugin). Also we want to have only one type of configuration files. 3) create Arquillian's ServerSetupTasks for setting containters (setting security domains for testing etc.). Why? We want to avoid XSLT because it is sometimes out of work in diffrent type of JDKs. 4) remove "dist" folders from every container (/integration-tests/CONTAINER/dist) and remove distributions from dist. We will use only one dist folder which will be located in integration-tests folder. Why? If anybody is cloning picketlink-integration-tests from git, he have to clone distributions, but it take a lot of unnecessary time. We think that better way is have one empty folder for all distributions. User can input nedded distributions. What do you think about that? Does anybody have any idea about improvement of picketlink-integration-tests testsuite? Thanks, Ondrej Lukas

From: "Anil Saldhana" <Anil.Saldhana(a)redhat.com&gt; To: security-dev(a)lists.jboss.org Sent: Thursday, May 23, 2013 4:17:12 PM Subject: Re: [security-dev] Refactoring of picketlink-integration-tests It should be ok as long as we test against the various versions. On 05/23/2013 02:10 AM, Ondra Lukas wrote: > Hi, > we would like to do some refactoring of PicketLink integration testsuite > for easier QE testing with new versions of EAP. We have some ideas how to > improve it and I want to ask you what do you think about that. Here is the > list, we would like to: > > 1) change of configuration which running specified tests for specified > container. It is currently set by @TargetContainers annotation. We prefer > to using better usage of maven profiles and for instance Java subpackages > according to profiles. Every subpackage will contain tests which will run > only in that profile (for example org.picketlink.test.trust.tests.eap6 > will contain tests for EAP6 profile). Tests which run in every profile > will stay in current packages. Tests which run in more profiles (but not > in all of them) will be added by include/exclude parameters of > maven-failsafe-plugin. > > Why? > It will be easier to configurate it for QE testing. We need some easy way > how to see each test of some profile. Currently we have all tests together > and it's quite uncomfortable. It will be simpler to add a new container > too. > > 2) avoid use of Ant and try to rewrite it to maven (using Maven Resource > Plugin etc.). > > Why? > We want to have ability of setting properties from command line (which is > not handled correctly by maven-antrun-plugin). Also we want to have only > one type of configuration files. > > 3) create Arquillian's ServerSetupTasks for setting containters (setting > security domains for testing etc.). > > Why? > We want to avoid XSLT because it is sometimes out of work in diffrent type > of JDKs. > > 4) remove "dist" folders from every container > (/integration-tests/CONTAINER/dist) and remove distributions from dist. We > will use only one dist folder which will be located in integration-tests > folder. > > Why? > If anybody is cloning picketlink-integration-tests from git, he have to > clone distributions, but it take a lot of unnecessary time. We think that > better way is have one empty folder for all distributions. User can input > nedded distributions. > > What do you think about that? Does anybody have any idea about improvement > of picketlink-integration-tests testsuite? > > Thanks, > Ondrej Lukas _______________________________________________ security-dev mailing list security-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/security-dev

From: "Pedro Igor Silva" <psilva(a)redhat.com&gt; To: "Ondra Lukas" <olukas(a)redhat.com&gt; Cc: security-dev(a)lists.jboss.org Sent: Wednesday, June 26, 2013 5:27:56 PM Subject: Re: [security-dev] Refactoring of picketlink-integration-tests Very nice. I have only a few questions for now: - Today we're able to execute a specific group of tests (saml, ws-trust, xacml, etc) for a specific container or all of them. Is this still possible ?

- Is there any profile to run arquillian using a remote container ? That would make a lot easier to add and debug things.

- The ServerSetup is only for SecurityDomains or it can be used to any other configuration in standalone.xml ?

- Is it possible to run the test against a specific PicketLink Version ? Without having to change poms or any config file ?

I would like to start using this new repo for some PicketLink 2.5 tests. That way I can give more feedbacks. Thanks. Pedro Igor

----- Original Message ----- From: "Ondra Lukas" <olukas(a)redhat.com&gt; To: security-dev(a)lists.jboss.org Sent: Wednesday, June 26, 2013 9:25:10 AM Subject: Re: [security-dev] Refactoring of picketlink-integration-tests Hi, I have some progress in refactoring of PicketLink testsuite. Most important part for us is EAP 6 and for this reason all changes are connected with eap-6 profile meanwhile. It's still working version. Take a look at https://github.com/jboss-security-qe/picketlink-integration-tests/tree/pl.... What's new? 1) I copied ServerSetup from Wildfly testsuite and modified it for using in PicketLink testsuite. It is used for adding security domains instead of XSLT in jbas7/eap6 (addSecurityDomain.xslt). Every test which use some security domains adds that domains by annotation @ServerSetup({names_of_domains}). Security domains are defined in SecurityDomainServerSetupTask - there are domains which are currently used by tests and new domain can be added later. However ServerSetup is compatible only with EAP 6.1 (there is little difference in adding security domain via Management API in EAP 6.0 and EAP 6.1), using with EAP 6.0 will be added later. 2) Tests are divided into packages according to containers. We have 5 types of packages: jbas7, eap6, jbas5, eap5 and common. Every profile runs test in some package according to: - profile jbas7 - runs jbas7 and some includes from commons - eap6 - eap6, jbas7 and some includes from commons - jbas5 - jbas5, commons - eap5 - eap5, jbas5, commons - jbas6 - commons and some excludes - tomcat6 - commons and some excludes Which tests are actually running is defined in maven-failsafe-plugin in integration-tests/CONTAINER/pom.xml. 3) Using of Ant file is removed from eap6 profile. Downloading EAP6 is provided by maven-download-plugin and copying PicketLink libraries is provided by maven-resources-plugin. However it used maven-antrun-plugin for unzip container - it's not satisfying but I didn't find any other way how to unzip something by maven. Due to copying PicketLink libraries there are three new profiles: - copy-picketlink-from-maven - it is activated by default and it copies PicketLink from maven.repo.local - copy-picketlink-from-lib - it copies PicketLink from jboss.as.picketlink.lib.dir and it work only if jboss.as.picketlink.lib.dir is set - skip-copy-picketlink - it doesn't copy anything, original PicketLink from EAP module is used Property picketlink.skip.lib.copy was removed from eap6 profile because it is useless now. 4) Folder dist was added to integration-tests directory - it's only little change, all containers will be located at one place instead of dist folder in every container. You can use your own EAP6 distribution if you copy it to dist folder and running testsuite with property -Deap6-dist-zip=name_of_file_from_dist.zip (and with -Deap6-dir-structure=structure if it isn't use default jboss-eap-6.1) I want to ask you, is anybody have any idea what can be improved in these changes. I'll be happy for every feedback. Thanks, Ondra ----- Original Message ----- > From: "Anil Saldhana" <Anil.Saldhana(a)redhat.com&gt; > To: security-dev(a)lists.jboss.org > Sent: Thursday, May 23, 2013 4:17:12 PM > Subject: Re: [security-dev] Refactoring of picketlink-integration-tests > > It should be ok as long as we test against the various versions. > > On 05/23/2013 02:10 AM, Ondra Lukas wrote: > > Hi, > > we would like to do some refactoring of PicketLink integration testsuite > > for easier QE testing with new versions of EAP. We have some ideas how to > > improve it and I want to ask you what do you think about that. Here is > > the > > list, we would like to: > > > > 1) change of configuration which running specified tests for specified > > container. It is currently set by @TargetContainers annotation. We prefer > > to using better usage of maven profiles and for instance Java subpackages > > according to profiles. Every subpackage will contain tests which will run > > only in that profile (for example org.picketlink.test.trust.tests.eap6 > > will contain tests for EAP6 profile). Tests which run in every profile > > will stay in current packages. Tests which run in more profiles (but not > > in all of them) will be added by include/exclude parameters of > > maven-failsafe-plugin. > > > > Why? > > It will be easier to configurate it for QE testing. We need some easy way > > how to see each test of some profile. Currently we have all tests > > together > > and it's quite uncomfortable. It will be simpler to add a new container > > too. > > > > 2) avoid use of Ant and try to rewrite it to maven (using Maven Resource > > Plugin etc.). > > > > Why? > > We want to have ability of setting properties from command line (which is > > not handled correctly by maven-antrun-plugin). Also we want to have only > > one type of configuration files. > > > > 3) create Arquillian's ServerSetupTasks for setting containters (setting > > security domains for testing etc.). > > > > Why? > > We want to avoid XSLT because it is sometimes out of work in diffrent > > type > > of JDKs. > > > > 4) remove "dist" folders from every container > > (/integration-tests/CONTAINER/dist) and remove distributions from dist. > > We > > will use only one dist folder which will be located in integration-tests > > folder. > > > > Why? > > If anybody is cloning picketlink-integration-tests from git, he have to > > clone distributions, but it take a lot of unnecessary time. We think that > > better way is have one empty folder for all distributions. User can input > > nedded distributions. > > > > What do you think about that? Does anybody have any idea about > > improvement > > of picketlink-integration-tests testsuite? > > > > Thanks, > > Ondrej Lukas > _______________________________________________ > security-dev mailing list > security-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/security-dev > _______________________________________________ security-dev mailing list security-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/security-dev