Re: [gatein-dev] Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509)

----- Original Message ----- From: "Dimitri BAELI" <dbaeli(a)gmail.com&gt; To: "Prabhat Jha" <pjha(a)redhat.com&gt; Cc: &quot;gatein-dev(a)lists.jboss.org gatein-dev(a)lists.jboss.org&quot; <gatein-dev(a)lists.jboss.org&gt;, "Michal Vančo" <mvanco(a)redhat.com&gt;, "myarboro" <myarboro(a)redhat.com&gt;, "Arthur Peltier" <arthur.peltier(a)exoplatform.com&gt;, "Aristote Philippe" <philippe.aristote(a)exoplatform.com&gt; Sent: Thursday, February 25, 2010 2:15:56 PM GMT -06:00 US/Canada Central Subject: Re: Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509) Sorry for the late response, the QA Team will soon have the commit right to fix directly the issues in the GateIn code base. ko/ should be cleaned in the comming days, and we are working with the portal team to have the issues fixed (3 issues registered). I hope by "coming days" you man early next week. ;-) unstable/ tests are not guaranteed to be fixed as it is a Selenium issue with GateIn (some wait instructions are not respected). We had this problem with richfaces but we figured a work around. Please have your team describe the problem and I will have him connected to right person. The exectuion speed is configurable so that we can run the tests slowly to be sure to have the right result, but the goal is to have all the tests capable to run full speed (with wait where needed). Yes, this has helped and now all passable tests are passing. Next time, the Portal Team tester should be able to give that kind of feed back. I'm out of the daily work on Selenium Tests implementation but still on GateIn QA management on eXo side. Lucky you. ;-) For priority II test planning, are you going to be involved with this or is it going to be Arthur or Aristote? Dimitri On 23 févr. 2010, at 22:59, Prabhat Jha wrote: > Hi Dimitri, > > As far as 5 tests that are in selenium/issue, I only see 2 jira issues mentioned in gatein-trunk/testsuite/GateIn_v3.0_SniffTests.ods namely for SNF_PRL_06 and SNF_PRL_13? Do you have jira ids for the rest of 3. May be you did not get a chance to update the sheet. I am asking so that I can see if they can be bumped up in priority. Sorry for sounding like an auditor. ;-) > I will ask Michal to verify 2 tests in selenium/candidate and selenium/unstable. Do you have a ETA on selenium/ko? > > I would also like to take a stab at priority II tests that we had talked before now that priority I is "almost" completed. Please let me know when you want to do that. > > Thanks, > Prabhat > > > ----- Original Message ----- > From: "Dimitri BAELI" <dbaeli(a)gmail.com&gt; > To: "Prabhat Jha" <pjha(a)redhat.com&gt; > Cc: &quot;gatein-dev(a)lists.jboss.org gatein-dev(a)lists.jboss.org&quot; <gatein-dev(a)lists.jboss.org&gt;, "Michal Vančo" <mvanco(a)redhat.com&gt;, "myarboro" <myarboro(a)redhat.com&gt;, "Arthur Peltier" <arthur.peltier(a)exoplatform.com&gt;, "Aristote Philippe" <philippe.aristote(a)exoplatform.com&gt; > Sent: Monday, February 22, 2010 5:55:29 PM GMT -06:00 US/Canada Central > Subject: Re: Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509) > > Hello, > Today I've worked with Michal to have the same results on both sides for the Selenium tests. > Michal will run the tests again tomorrow and will give his feedback. > > A new organisation is taking place for the Selenium Tests: > > Inside the testsuite/selenium-snifftests/ module not all the tests are run by default, only those who should not fail. > > /selenium (21) > /selenium/candidate/ (2) : The tests ready to be integrated in the suite, but should be verified on both sides. > /selenium/design/ (0) : New test being designed > /selenium/issue/ (5) : Tests failing due to a GateIn known issue (with Jira Issue) > /selenium/ko (6) : Tests broken or not ready for full automation, under study to determine if it is a GateIn issue or unstable test to be fixed > /selenium/unstable/ (1) : Tests ok in SeleniumIDE but not always passing at full speed. > > So current status is 21 (+2) tests ok, 5 blocked by a GateIn issue, 6 under study. > To be confirmed by Michal, for sure. > > Next step is to let the Portal Team (with part of the eXo QA Team) finalize and extend those tests. > > Dimitri > > On 18 févr. 2010, at 16:56, Prabhat Jha wrote: > >> Hi Dimitri, >> >> Any update on this? I was hoping to have these taken care of this week so that we can focus on new tests starting next week. >> >> Thanks, >> Prabhat >> >> >> ----- Original Message ----- >> From: "Dimitri" <dbaeli(a)gmail.com&gt; >> To: "Michal Vančo" <mvanco(a)redhat.com&gt; >> Cc: gatein-dev(a)lists.jboss.org, "Prabhat Jha" <pjha(a)redhat.com&gt; >> Sent: Monday, February 15, 2010 9:50:53 AM GMT -06:00 US/Canada Central >> Subject: Re: Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509) >> >> Tests order is not controled, I'll have a look on that. >> A test failure should not affect the others (browser is restarted and >> session also), this is annoying, a better tearDown of the tests will >> be studied. >> >> Dimitri BAELI >> VP Quality -- eXo Platform >> >> Le 15 févr. 2010 à 16:41, Michal Vančo <mvanco(a)redhat.com&gt; a écrit : >> >>> Hi, >>> >>> I've tried the Selenium tests on trunk version of GateIn and almost >>> all >>> the tests are failing due to the unreachable actions -> timeout. >>> There are links to hudson: >>> http://hudson.jboss.org/hudson/view/GateIn/job/gatein_portal_ui_selenium_... >>> http://hudson.jboss.org/hudson/view/GateIn/job/gatein_portal_ui_selenium_... >>> >>> Seems like the tests are failing, because in some test case, there was >>> set a french language to signed user, than the test fails and french >>> language remains. Then the other tests can't find the elements >>> present, >>> because they are in other language. >>> >>> I have also checked the tests on localhost on beta5 (tomcat) and >>> have a >>> same problem, but only at last 14 tests. Reports from this testing are >>> attached. >>> >>> I would prefer to place the tests which are changing languages (of >>> signed user) to the end, because if some of them fails, all the >>> following tests fails as well. >>> >>> Also, please run the tests on trunk version of GateIn to be able to >>> compare results. >>> >>> Thanks and regards, >>> >>> Michal Vanco >>> >>> And btw. thanks Dimitri for configuration on JBoss AS (selenium port), >>> now we can run the tests much easier. >>> >>> Dimitri BAELI wrote at 12. 02. 2010 v 18:21 +0700: >>>> Hello, >>>> Today a new version of the Selenium Tests have been commited with >>>> some tooling to generate them from the html version (easier to record >>>> and maintain for such basic scripts). >>>> >>>> >>>> They are now located in testsuite/selenium-snifftests >>>> To perform the tests mvn install -Pselenium >>>> The JUnit classes are generated in target/generated/test from the >>>> src/suite content. >>>> Those tests have been tested on the GateIn beta5 and should ensure >>>> there is no major regression on the UI part. >>>> 3 tests are currently failing due to GTNPORTAL known issues, some >>>> instability (when running Selenium full speed) are under study. >>>> >>>> >>>> We'll soon provide a documentation on how to record some new >>>> scripts (using Selenium IDE) and integrate then in the testsuite, the >>>> Portal Tester will be in charge for that. >>>> >>>> >>>> Warning for Selenium on Snowleopard, FF crashes, and for the moment >>>> the fix is to patch and recompile selenium-rc and the >>>> selenium-maven-plugin (documented in the readme >>>> of testsuite/selenium-snifftests). >>>> >>>> >>>> Dimitri >>>> >>>> >>>> Le 21 janv. 2010 à 20:50, Dimitri BAELI a écrit : >>>> >>>>> >>>>> >>>>> Début du message réexpédié : >>>>> >>>>>> De : Michal Vančo <mvanco(a)redhat.com&gt; >>>>>> >>>>>> Date : 20 janvier 2010 17:44:47 HNEC >>>>>> >>>>>> À : gatein-dev(a)lists.jboss.org >>>>>> >>>>>> Cc : dbaeli(a)gmail.com >>>>>> >>>>>> Objet : Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509) >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> I work on running ui tests for gatein from exo >>>>>> (https://jira.jboss.org/jira/browse/JBQA-3004 ). >>>>>> >>>>>> I send some comments about selenium tests that I run on localhost >>>>>> (I >>>>>> used the list of cases from >>>>>> https://jira.jboss.org/jira/browse/GTNPORTAL-509 ). >>>>>> >>>>>> Selenium tests comments (tested on beta04 with Tomcat): >>>>>> >>>>>> * SNF_PRL_01 WC3 check for pages in public - OK >>>>>> * SNF_PRL_02 Create new account - OK >>>>>> * SNF_PRL_03 Change displaying language - OK >>>>>> * SNF_PRL_04 Sign in/out - OK >>>>>> * SNF_PRL_05 Remember my login - FAIL - remember me funcion >>>>>> doesn't work >>>>>> * SNF_PRL_06 Users Management - FAIL - changes in profile editting >>>>>> (mary) didn't reproduce to User Management page, but after Edit >>>>>> again - >>>>>> edited values are displayed >>>>>> * SNF_PRL_07 Groups Management - OK >>>>>> * SNF_PRL_08 Memberships Management - OK >>>>>> * SNF_PRL_09 Import application - OK >>>>>> * SNF_PRL_10 Create/Edit/Delete category - OK >>>>>> * SNF_PRL_11 Add application into category - FAIL - trying to set >>>>>> permission on application which is not selected, I think that the >>>>>> application should be add into "category test edit" >>>>>> * SNF_PRL_12 Portlet - ISN'T PRESENT >>>>>> * SNF_PRL_13 Add gadget(remote/create new) - FAIL - not correctly >>>>>> started (Sign in->Dashboard...) >>>>>> * SNF_PRL_14 Show or not Import Application icon - ISN'T PRESENT >>>>>> * SNF_PRL_15 Link to pages - OK >>>>>> * SNF_PRL_16 Expand All - ISN'T PRESENT >>>>>> * SNF_PRL_17 Collapse All - ISN'T PRESENT >>>>>> * SNF_PRL_18 Create new portal - OK >>>>>> * SNF_PRL_19 Edit portal - OK >>>>>> * SNF_PRL_20 Delete portal - FAIL - >>>>>> waitForElementPresent //div[@class='CollapseIcon'], before this >>>>>> there is >>>>>> twice clickAt "Platform", I think that there should be clickAt >>>>>> "Platform" and then clickAt "Administrators" (or >>>>>> waitForElementPresent >>>>>> "Platform", clickAt "Platform"), then there twice clickAt Save >>>>>> (should >>>>>> be waitForElementPresent "Save", clickAt "Save")) >>>>>> * SNF_PRL_21 Change using portal -OK >>>>>> * SNF_PRL_22 Create/Edit page/Edit site layout for portal - OK >>>>>> * SNF_PRL_23 Add Navigation - OK >>>>>> * SNF_PRL_24 Edit navigation/Edit properties/Delete/ in Group >>>>>> Navigation >>>>>> - OK >>>>>> * SNF_PRL_25 Edit page of node - OK >>>>>> * SNF_PRL_26 Move up/down a node - ISN'T PRESENT >>>>>> * SNF_PRL_27 Create/Edit page/Edit site layout for group - (1,2) >>>>>> OK >>>>>> * SNF_PRL_28 Add/drag-drop - OK >>>>>> * SNF_PRL_29 Add/Edit?Delete tab - OK >>>>>> * SNF_PRL_30 Create/Edit page/Edit site layout for user - OK >>>>>> * SNF_PRL_31 Change language - OK >>>>>> * SNF_PRL_32 Change skin - OK >>>>>> * SNF_PRL_33 Setting account - OK (but has a wrong name - >>>>>> LogoPortletAccSetting and code SNF_PRL_34) >>>>>> * SNF_PRL_34 Change logo - INS'T PRESENT >>>>>> >>>>>> There is also SNF_PRL_35 under which the advanced page >>>>>> adding/editing - >>>>>> OK >>>>>> >>>>>> I can also reproduce the result file from test cases if needed. >>>>>> >>>>>> Thanks a lot for for your replies. >>>>>> >>>>>> Regards >>>>>> Michal >>>>>> >>>>>> >>>>> >>>>> Dimitri BAELI >>>>> VP Quality - eXo Platform >>>>> >>>>> >>>> >>>> Dimitri BAELI >>>> VP Quality - eXo Platform >>>> >>>> >>> >>> <report> >>> <surefire-reports.zip> > > Dimitri BAELI > VP Quality - eXo Platform > Dimitri BAELI VP Quality - eXo Platform