[aerogear-dev] Aerogear SDK E2E testing discussion

[Vojtech Sazel]

Hi all,

Wei recommended me to start further discussion about viability of making
E2E tests for regression testing of our SDKs. There was some debate around
it, so I'm putting it also to the mailing list.

I have done Appium UI test
<https://github.com/aerogear/aerogear-sdk-e2e-tests/pull/1> of the example
app included in AG Android SDK [1]
<https://github.com/aerogear/aerogear-sdk-e2e-tests/pull/1>. It has been
done for Auth functionality. This test checks happy path of login and
logout.

Is it worth doing it this way? Is it worth doing it at least for some
functionality?

My take TLDR:

Pros:
+ it tests real end-user/developer usage of the SDK in the real device
environment

In the middle:
= it requires running emulator or connected device to the test server

Cons:
- test stability (it can go wrong because of load on the test server)
- test speed
- UI change volatility can break the test - this is partially avoided by
using page objects [2] <https://martinfowler.com/bliki/PageObject.html>
pattern

I think the real environment is clear benefit, for OpenID authentication,
it's difficult to do it other way than from UI and using real browser. I
don't know if this is true for other functionality. But writing tests at
least for happy paths can be beneficial.

refs:
[1] https://github.com/aerogear/aerogear-sdk-e2e-tests/pull/1
[2] https://martinfowler.com/bliki/PageObject.html
-- 

VOJTĚCH SÁZEL

SENIOR SOFTWARE ENGINEER, RED HAT MOBILE

Red Hat

<https://www.redhat.com/>

Remote Czech Republic

vsazel at redhat.com    IM: vsazel
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/aerogear-dev/attachments/20180309/8788fa9b/attachment.html