[gatein-dev] Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509)
Michal Vančo
mvanco at redhat.com
Wed Feb 24 11:22:22 EST 2010
Hi,
I tried to run tests in candidate folder and get following results:
- test 22 - pass with speed more than 1000ms, but I think there is
mistake in testcase, portal language is changed to French, but after
verifying it's not set back to English - it could cause a failure of
other tests
- test 28 fails even with speed 3000ms (problem when try to add
Calendar gadget on Dashboard)
Test in unstable folder (20) fails as well.
Regards,
Michal
Prabhat Jha píše v Út 23. 02. 2010 v 16:59 -0500:
> Hi Dimitri,
>
> As far as 5 tests that are in selenium/issue, I only see 2 jira issues mentioned in gatein-trunk/testsuite/GateIn_v3.0_SniffTests.ods namely for SNF_PRL_06 and SNF_PRL_13? Do you have jira ids for the rest of 3. May be you did not get a chance to update the sheet. I am asking so that I can see if they can be bumped up in priority. Sorry for sounding like an auditor. ;-)
>
> I will ask Michal to verify 2 tests in selenium/candidate and selenium/unstable. Do you have a ETA on selenium/ko?
>
> I would also like to take a stab at priority II tests that we had talked before now that priority I is "almost" completed. Please let me know when you want to do that.
>
> Thanks,
> Prabhat
>
>
> ----- Original Message -----
> From: "Dimitri BAELI" <dbaeli at gmail.com>
> To: "Prabhat Jha" <pjha at redhat.com>
> Cc: "gatein-dev at lists.jboss.org gatein-dev at lists.jboss.org" <gatein-dev at lists.jboss.org>, "Michal Vančo" <mvanco at redhat.com>, "myarboro" <myarboro at redhat.com>, "Arthur Peltier" <arthur.peltier at exoplatform.com>, "Aristote Philippe" <philippe.aristote at exoplatform.com>
> Sent: Monday, February 22, 2010 5:55:29 PM GMT -06:00 US/Canada Central
> Subject: Re: Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509)
>
> Hello,
> Today I've worked with Michal to have the same results on both sides for the Selenium tests.
> Michal will run the tests again tomorrow and will give his feedback.
>
> A new organisation is taking place for the Selenium Tests:
>
> Inside the testsuite/selenium-snifftests/ module not all the tests are run by default, only those who should not fail.
>
> /selenium (21)
> /selenium/candidate/ (2) : The tests ready to be integrated in the suite, but should be verified on both sides.
> /selenium/design/ (0) : New test being designed
> /selenium/issue/ (5) : Tests failing due to a GateIn known issue (with Jira Issue)
> /selenium/ko (6) : Tests broken or not ready for full automation, under study to determine if it is a GateIn issue or unstable test to be fixed
> /selenium/unstable/ (1) : Tests ok in SeleniumIDE but not always passing at full speed.
>
> So current status is 21 (+2) tests ok, 5 blocked by a GateIn issue, 6 under study.
> To be confirmed by Michal, for sure.
>
> Next step is to let the Portal Team (with part of the eXo QA Team) finalize and extend those tests.
>
> Dimitri
>
> On 18 févr. 2010, at 16:56, Prabhat Jha wrote:
>
> > Hi Dimitri,
> >
> > Any update on this? I was hoping to have these taken care of this week so that we can focus on new tests starting next week.
> >
> > Thanks,
> > Prabhat
> >
> >
> > ----- Original Message -----
> > From: "Dimitri" <dbaeli at gmail.com>
> > To: "Michal Vančo" <mvanco at redhat.com>
> > Cc: gatein-dev at lists.jboss.org, "Prabhat Jha" <pjha at redhat.com>
> > Sent: Monday, February 15, 2010 9:50:53 AM GMT -06:00 US/Canada Central
> > Subject: Re: Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509)
> >
> > Tests order is not controled, I'll have a look on that.
> > A test failure should not affect the others (browser is restarted and
> > session also), this is annoying, a better tearDown of the tests will
> > be studied.
> >
> > Dimitri BAELI
> > VP Quality -- eXo Platform
> >
> > Le 15 févr. 2010 à 16:41, Michal Vančo <mvanco at redhat.com> a écrit :
> >
> >> Hi,
> >>
> >> I've tried the Selenium tests on trunk version of GateIn and almost
> >> all
> >> the tests are failing due to the unreachable actions -> timeout.
> >> There are links to hudson:
> >> http://hudson.jboss.org/hudson/view/GateIn/job/gatein_portal_ui_selenium_tests_jboss5/24/
> >> http://hudson.jboss.org/hudson/view/GateIn/job/gatein_portal_ui_selenium_tests_tomcat6/22/
> >>
> >> Seems like the tests are failing, because in some test case, there was
> >> set a french language to signed user, than the test fails and french
> >> language remains. Then the other tests can't find the elements
> >> present,
> >> because they are in other language.
> >>
> >> I have also checked the tests on localhost on beta5 (tomcat) and
> >> have a
> >> same problem, but only at last 14 tests. Reports from this testing are
> >> attached.
> >>
> >> I would prefer to place the tests which are changing languages (of
> >> signed user) to the end, because if some of them fails, all the
> >> following tests fails as well.
> >>
> >> Also, please run the tests on trunk version of GateIn to be able to
> >> compare results.
> >>
> >> Thanks and regards,
> >>
> >> Michal Vanco
> >>
> >> And btw. thanks Dimitri for configuration on JBoss AS (selenium port),
> >> now we can run the tests much easier.
> >>
> >> Dimitri BAELI wrote at 12. 02. 2010 v 18:21 +0700:
> >>> Hello,
> >>> Today a new version of the Selenium Tests have been commited with
> >>> some tooling to generate them from the html version (easier to record
> >>> and maintain for such basic scripts).
> >>>
> >>>
> >>> They are now located in testsuite/selenium-snifftests
> >>> To perform the tests mvn install -Pselenium
> >>> The JUnit classes are generated in target/generated/test from the
> >>> src/suite content.
> >>> Those tests have been tested on the GateIn beta5 and should ensure
> >>> there is no major regression on the UI part.
> >>> 3 tests are currently failing due to GTNPORTAL known issues, some
> >>> instability (when running Selenium full speed) are under study.
> >>>
> >>>
> >>> We'll soon provide a documentation on how to record some new
> >>> scripts (using Selenium IDE) and integrate then in the testsuite, the
> >>> Portal Tester will be in charge for that.
> >>>
> >>>
> >>> Warning for Selenium on Snowleopard, FF crashes, and for the moment
> >>> the fix is to patch and recompile selenium-rc and the
> >>> selenium-maven-plugin (documented in the readme
> >>> of testsuite/selenium-snifftests).
> >>>
> >>>
> >>> Dimitri
> >>>
> >>>
> >>> Le 21 janv. 2010 à 20:50, Dimitri BAELI a écrit :
> >>>
> >>>>
> >>>>
> >>>> Début du message réexpédié :
> >>>>
> >>>>> De : Michal Vančo <mvanco at redhat.com>
> >>>>>
> >>>>> Date : 20 janvier 2010 17:44:47 HNEC
> >>>>>
> >>>>> À : gatein-dev at lists.jboss.org
> >>>>>
> >>>>> Cc : dbaeli at gmail.com
> >>>>>
> >>>>> Objet : Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509)
> >>>>>
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I work on running ui tests for gatein from exo
> >>>>> (https://jira.jboss.org/jira/browse/JBQA-3004 ).
> >>>>>
> >>>>> I send some comments about selenium tests that I run on localhost
> >>>>> (I
> >>>>> used the list of cases from
> >>>>> https://jira.jboss.org/jira/browse/GTNPORTAL-509 ).
> >>>>>
> >>>>> Selenium tests comments (tested on beta04 with Tomcat):
> >>>>>
> >>>>> * SNF_PRL_01 WC3 check for pages in public - OK
> >>>>> * SNF_PRL_02 Create new account - OK
> >>>>> * SNF_PRL_03 Change displaying language - OK
> >>>>> * SNF_PRL_04 Sign in/out - OK
> >>>>> * SNF_PRL_05 Remember my login - FAIL - remember me funcion
> >>>>> doesn't work
> >>>>> * SNF_PRL_06 Users Management - FAIL - changes in profile editting
> >>>>> (mary) didn't reproduce to User Management page, but after Edit
> >>>>> again -
> >>>>> edited values are displayed
> >>>>> * SNF_PRL_07 Groups Management - OK
> >>>>> * SNF_PRL_08 Memberships Management - OK
> >>>>> * SNF_PRL_09 Import application - OK
> >>>>> * SNF_PRL_10 Create/Edit/Delete category - OK
> >>>>> * SNF_PRL_11 Add application into category - FAIL - trying to set
> >>>>> permission on application which is not selected, I think that the
> >>>>> application should be add into "category test edit"
> >>>>> * SNF_PRL_12 Portlet - ISN'T PRESENT
> >>>>> * SNF_PRL_13 Add gadget(remote/create new) - FAIL - not correctly
> >>>>> started (Sign in->Dashboard...)
> >>>>> * SNF_PRL_14 Show or not Import Application icon - ISN'T PRESENT
> >>>>> * SNF_PRL_15 Link to pages - OK
> >>>>> * SNF_PRL_16 Expand All - ISN'T PRESENT
> >>>>> * SNF_PRL_17 Collapse All - ISN'T PRESENT
> >>>>> * SNF_PRL_18 Create new portal - OK
> >>>>> * SNF_PRL_19 Edit portal - OK
> >>>>> * SNF_PRL_20 Delete portal - FAIL -
> >>>>> waitForElementPresent //div[@class='CollapseIcon'], before this
> >>>>> there is
> >>>>> twice clickAt "Platform", I think that there should be clickAt
> >>>>> "Platform" and then clickAt "Administrators" (or
> >>>>> waitForElementPresent
> >>>>> "Platform", clickAt "Platform"), then there twice clickAt Save
> >>>>> (should
> >>>>> be waitForElementPresent "Save", clickAt "Save"))
> >>>>> * SNF_PRL_21 Change using portal -OK
> >>>>> * SNF_PRL_22 Create/Edit page/Edit site layout for portal - OK
> >>>>> * SNF_PRL_23 Add Navigation - OK
> >>>>> * SNF_PRL_24 Edit navigation/Edit properties/Delete/ in Group
> >>>>> Navigation
> >>>>> - OK
> >>>>> * SNF_PRL_25 Edit page of node - OK
> >>>>> * SNF_PRL_26 Move up/down a node - ISN'T PRESENT
> >>>>> * SNF_PRL_27 Create/Edit page/Edit site layout for group - (1,2)
> >>>>> OK
> >>>>> * SNF_PRL_28 Add/drag-drop - OK
> >>>>> * SNF_PRL_29 Add/Edit?Delete tab - OK
> >>>>> * SNF_PRL_30 Create/Edit page/Edit site layout for user - OK
> >>>>> * SNF_PRL_31 Change language - OK
> >>>>> * SNF_PRL_32 Change skin - OK
> >>>>> * SNF_PRL_33 Setting account - OK (but has a wrong name -
> >>>>> LogoPortletAccSetting and code SNF_PRL_34)
> >>>>> * SNF_PRL_34 Change logo - INS'T PRESENT
> >>>>>
> >>>>> There is also SNF_PRL_35 under which the advanced page
> >>>>> adding/editing -
> >>>>> OK
> >>>>>
> >>>>> I can also reproduce the result file from test cases if needed.
> >>>>>
> >>>>> Thanks a lot for for your replies.
> >>>>>
> >>>>> Regards
> >>>>> Michal
> >>>>>
> >>>>>
> >>>>
> >>>> Dimitri BAELI
> >>>> VP Quality - eXo Platform
> >>>>
> >>>>
> >>>
> >>> Dimitri BAELI
> >>> VP Quality - eXo Platform
> >>>
> >>>
> >>
> >> <report>
> >> <surefire-reports.zip>
>
> Dimitri BAELI
> VP Quality - eXo Platform
>
More information about the gatein-dev
mailing list