[gatein-dev] Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509)
Prabhat Jha
pjha at redhat.com
Tue Feb 23 09:15:09 EST 2010
Yes, you can run them outside of Hudson. :-) Michal will send instructions on how to run tests.
----- Original Message -----
From: "Thomas Heute" <theute at redhat.com>
To: gatein-dev at lists.jboss.org
Sent: Tuesday, February 23, 2010 3:50:53 AM GMT -06:00 US/Canada Central
Subject: Re: [gatein-dev] Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509)
Good news :)
We will need instructions to run those (Can we run them outside of
Hudson ?), and also how to add tests.
Thanks !
On 02/23/2010 10:31 AM, Michal Vančo wrote:
> Hi,
>
> today I tested described version of tests and get the following results:
> - localhost:
> - Tomcat: Tests run: 21, Failures: 0, Errors: 0, Skipped: 0
> - JBoss AS: Tests run: 21, Failures: 0, Errors: 0, Skipped: 0
> - Hudson:
> - Tomcat: Tests run: 21, Failures: 0, Errors: 0, Skipped: 0
> (more details at: http://hudson.jboss.org/hudson/view/GateIn/job/gatein_portal_ui_selenium_tests_tomcat6/31/ )
> - JBoss: Tests run: 21, Failures: 0, Errors: 0, Skipped: 0
> (more details at: http://hudson.jboss.org/hudson/view/GateIn/job/gatein_portal_ui_selenium_tests_jboss5/36/ )
>
> I used the following variables to run the tests:
> - selenium.port=6666
> - selenium.timeout=60000
> - selenium.speed=250
>
> These tests I run on yesterday trunk version of GateIn and default DB -
> HSQLDB. I'll also try to run these tests on other DBs.
>
> Good job, it is first time to see the "Success" status of UI job in
> Hudson.
>
> Best regards,
>
> Michal
>
> Dimitri BAELI wrote at Tue 23. 02. 2010 v 00:55 +0100:
>
>> Hello,
>> Today I've worked with Michal to have the same results on both sides for the Selenium tests.
>> Michal will run the tests again tomorrow and will give his feedback.
>>
>> A new organisation is taking place for the Selenium Tests:
>>
>> Inside the testsuite/selenium-snifftests/ module not all the tests are run by default, only those who should not fail.
>>
>> /selenium (21)
>> /selenium/candidate/ (2) : The tests ready to be integrated in the suite, but should be verified on both sides.
>> /selenium/design/ (0) : New test being designed
>> /selenium/issue/ (5) : Tests failing due to a GateIn known issue (with Jira Issue)
>> /selenium/ko (6) : Tests broken or not ready for full automation, under study to determine if it is a GateIn issue or unstable test to be fixed
>> /selenium/unstable/ (1) : Tests ok in SeleniumIDE but not always passing at full speed.
>>
>> So current status is 21 (+2) tests ok, 5 blocked by a GateIn issue, 6 under study.
>> To be confirmed by Michal, for sure.
>>
>> Next step is to let the Portal Team (with part of the eXo QA Team) finalize and extend those tests.
>>
>> Dimitri
>>
>> On 18 févr. 2010, at 16:56, Prabhat Jha wrote:
>>
>>
>>> Hi Dimitri,
>>>
>>> Any update on this? I was hoping to have these taken care of this week so that we can focus on new tests starting next week.
>>>
>>> Thanks,
>>> Prabhat
>>>
>>>
>>> ----- Original Message -----
>>> From: "Dimitri"<dbaeli at gmail.com>
>>> To: "Michal Vančo"<mvanco at redhat.com>
>>> Cc: gatein-dev at lists.jboss.org, "Prabhat Jha"<pjha at redhat.com>
>>> Sent: Monday, February 15, 2010 9:50:53 AM GMT -06:00 US/Canada Central
>>> Subject: Re: Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509)
>>>
>>> Tests order is not controled, I'll have a look on that.
>>> A test failure should not affect the others (browser is restarted and
>>> session also), this is annoying, a better tearDown of the tests will
>>> be studied.
>>>
>>> Dimitri BAELI
>>> VP Quality -- eXo Platform
>>>
>>> Le 15 févr. 2010 à 16:41, Michal Vančo<mvanco at redhat.com> a écrit :
>>>
>>>
>>>> Hi,
>>>>
>>>> I've tried the Selenium tests on trunk version of GateIn and almost
>>>> all
>>>> the tests are failing due to the unreachable actions -> timeout.
>>>> There are links to hudson:
>>>> http://hudson.jboss.org/hudson/view/GateIn/job/gatein_portal_ui_selenium_tests_jboss5/24/
>>>> http://hudson.jboss.org/hudson/view/GateIn/job/gatein_portal_ui_selenium_tests_tomcat6/22/
>>>>
>>>> Seems like the tests are failing, because in some test case, there was
>>>> set a french language to signed user, than the test fails and french
>>>> language remains. Then the other tests can't find the elements
>>>> present,
>>>> because they are in other language.
>>>>
>>>> I have also checked the tests on localhost on beta5 (tomcat) and
>>>> have a
>>>> same problem, but only at last 14 tests. Reports from this testing are
>>>> attached.
>>>>
>>>> I would prefer to place the tests which are changing languages (of
>>>> signed user) to the end, because if some of them fails, all the
>>>> following tests fails as well.
>>>>
>>>> Also, please run the tests on trunk version of GateIn to be able to
>>>> compare results.
>>>>
>>>> Thanks and regards,
>>>>
>>>> Michal Vanco
>>>>
>>>> And btw. thanks Dimitri for configuration on JBoss AS (selenium port),
>>>> now we can run the tests much easier.
>>>>
>>>> Dimitri BAELI wrote at 12. 02. 2010 v 18:21 +0700:
>>>>
>>>>> Hello,
>>>>> Today a new version of the Selenium Tests have been commited with
>>>>> some tooling to generate them from the html version (easier to record
>>>>> and maintain for such basic scripts).
>>>>>
>>>>>
>>>>> They are now located in testsuite/selenium-snifftests
>>>>> To perform the tests mvn install -Pselenium
>>>>> The JUnit classes are generated in target/generated/test from the
>>>>> src/suite content.
>>>>> Those tests have been tested on the GateIn beta5 and should ensure
>>>>> there is no major regression on the UI part.
>>>>> 3 tests are currently failing due to GTNPORTAL known issues, some
>>>>> instability (when running Selenium full speed) are under study.
>>>>>
>>>>>
>>>>> We'll soon provide a documentation on how to record some new
>>>>> scripts (using Selenium IDE) and integrate then in the testsuite, the
>>>>> Portal Tester will be in charge for that.
>>>>>
>>>>>
>>>>> Warning for Selenium on Snowleopard, FF crashes, and for the moment
>>>>> the fix is to patch and recompile selenium-rc and the
>>>>> selenium-maven-plugin (documented in the readme
>>>>> of testsuite/selenium-snifftests).
>>>>>
>>>>>
>>>>> Dimitri
>>>>>
>>>>>
>>>>> Le 21 janv. 2010 à 20:50, Dimitri BAELI a écrit :
>>>>>
>>>>>
>>>>>>
>>>>>> Début du message réexpédié :
>>>>>>
>>>>>>
>>>>>>> De : Michal Vančo<mvanco at redhat.com>
>>>>>>>
>>>>>>> Date : 20 janvier 2010 17:44:47 HNEC
>>>>>>>
>>>>>>> À : gatein-dev at lists.jboss.org
>>>>>>>
>>>>>>> Cc : dbaeli at gmail.com
>>>>>>>
>>>>>>> Objet : Gatein Selenium tests comment (JBQA-3004, GTNPORTAL-509)
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I work on running ui tests for gatein from exo
>>>>>>> (https://jira.jboss.org/jira/browse/JBQA-3004 ).
>>>>>>>
>>>>>>> I send some comments about selenium tests that I run on localhost
>>>>>>> (I
>>>>>>> used the list of cases from
>>>>>>> https://jira.jboss.org/jira/browse/GTNPORTAL-509 ).
>>>>>>>
>>>>>>> Selenium tests comments (tested on beta04 with Tomcat):
>>>>>>>
>>>>>>> * SNF_PRL_01 WC3 check for pages in public - OK
>>>>>>> * SNF_PRL_02 Create new account - OK
>>>>>>> * SNF_PRL_03 Change displaying language - OK
>>>>>>> * SNF_PRL_04 Sign in/out - OK
>>>>>>> * SNF_PRL_05 Remember my login - FAIL - remember me funcion
>>>>>>> doesn't work
>>>>>>> * SNF_PRL_06 Users Management - FAIL - changes in profile editting
>>>>>>> (mary) didn't reproduce to User Management page, but after Edit
>>>>>>> again -
>>>>>>> edited values are displayed
>>>>>>> * SNF_PRL_07 Groups Management - OK
>>>>>>> * SNF_PRL_08 Memberships Management - OK
>>>>>>> * SNF_PRL_09 Import application - OK
>>>>>>> * SNF_PRL_10 Create/Edit/Delete category - OK
>>>>>>> * SNF_PRL_11 Add application into category - FAIL - trying to set
>>>>>>> permission on application which is not selected, I think that the
>>>>>>> application should be add into "category test edit"
>>>>>>> * SNF_PRL_12 Portlet - ISN'T PRESENT
>>>>>>> * SNF_PRL_13 Add gadget(remote/create new) - FAIL - not correctly
>>>>>>> started (Sign in->Dashboard...)
>>>>>>> * SNF_PRL_14 Show or not Import Application icon - ISN'T PRESENT
>>>>>>> * SNF_PRL_15 Link to pages - OK
>>>>>>> * SNF_PRL_16 Expand All - ISN'T PRESENT
>>>>>>> * SNF_PRL_17 Collapse All - ISN'T PRESENT
>>>>>>> * SNF_PRL_18 Create new portal - OK
>>>>>>> * SNF_PRL_19 Edit portal - OK
>>>>>>> * SNF_PRL_20 Delete portal - FAIL -
>>>>>>> waitForElementPresent //div[@class='CollapseIcon'], before this
>>>>>>> there is
>>>>>>> twice clickAt "Platform", I think that there should be clickAt
>>>>>>> "Platform" and then clickAt "Administrators" (or
>>>>>>> waitForElementPresent
>>>>>>> "Platform", clickAt "Platform"), then there twice clickAt Save
>>>>>>> (should
>>>>>>> be waitForElementPresent "Save", clickAt "Save"))
>>>>>>> * SNF_PRL_21 Change using portal -OK
>>>>>>> * SNF_PRL_22 Create/Edit page/Edit site layout for portal - OK
>>>>>>> * SNF_PRL_23 Add Navigation - OK
>>>>>>> * SNF_PRL_24 Edit navigation/Edit properties/Delete/ in Group
>>>>>>> Navigation
>>>>>>> - OK
>>>>>>> * SNF_PRL_25 Edit page of node - OK
>>>>>>> * SNF_PRL_26 Move up/down a node - ISN'T PRESENT
>>>>>>> * SNF_PRL_27 Create/Edit page/Edit site layout for group - (1,2)
>>>>>>> OK
>>>>>>> * SNF_PRL_28 Add/drag-drop - OK
>>>>>>> * SNF_PRL_29 Add/Edit?Delete tab - OK
>>>>>>> * SNF_PRL_30 Create/Edit page/Edit site layout for user - OK
>>>>>>> * SNF_PRL_31 Change language - OK
>>>>>>> * SNF_PRL_32 Change skin - OK
>>>>>>> * SNF_PRL_33 Setting account - OK (but has a wrong name -
>>>>>>> LogoPortletAccSetting and code SNF_PRL_34)
>>>>>>> * SNF_PRL_34 Change logo - INS'T PRESENT
>>>>>>>
>>>>>>> There is also SNF_PRL_35 under which the advanced page
>>>>>>> adding/editing -
>>>>>>> OK
>>>>>>>
>>>>>>> I can also reproduce the result file from test cases if needed.
>>>>>>>
>>>>>>> Thanks a lot for for your replies.
>>>>>>>
>>>>>>> Regards
>>>>>>> Michal
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Dimitri BAELI
>>>>>> VP Quality - eXo Platform
>>>>>>
>>>>>>
>>>>>>
>>>>> Dimitri BAELI
>>>>> VP Quality - eXo Platform
>>>>>
>>>>>
>>>>>
>>>> <report>
>>>> <surefire-reports.zip>
>>>>
>> Dimitri BAELI
>> VP Quality - eXo Platform
>>
>>
>
> _______________________________________________
> gatein-dev mailing list
> gatein-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/gatein-dev
>
_______________________________________________
gatein-dev mailing list
gatein-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/gatein-dev
More information about the gatein-dev
mailing list