I am trying to write some ftests for kitchensink quickstart but I am not able to write my tests. The best thing will be to show it on the example:

    @FindBy(jquery = ".invalid")
    private List<WebElement> INVALID;

    @Test
    @InSequence(1)
    public void testEmptyRegistration() {
        open();     Graphene.waitModel(browser).until().element(REGISTRATION_FORM).is().visible();
        Graphene.guardHttp(REGISTRATION_FORM).submit();

        assertTrue(contains(INVALID, "size must be between 1 and 25"));
        assertTrue(contains(INVALID, "may not be empty"));
        assertTrue(contains(INVALID, "numeric value out of bounds (<12 digits>.<0 digits> expected)"));
        assertTrue(!element(REGISTERED_MESSAGE).isPresent().apply(browser));
    }

    private boolean contains(final List<WebElement> elements, String message) {
        for (WebElement element : elements) {
            System.out.println(element.getText());
            if (element.getText().equals(message)) {
                return true;
            }
        }
        return false;
    }

The problem is with the third "contains" assert where message of "numeric value out of bounds" is expected. But that message varies completely randomly upon every test, the second version of that message is "size must be between 10 and 12".

First of all, I personally think that there should be only one version of the message. When I hit the register button on the empty form, I do not want to see any "numeric value out of bounds" message, I want to see the second version, "size must be between 10 and 12".

Secondly, it is weird that message is expected to be shown to a user ("size must be between 10 and 12") but when I deploy quickstart and I try it on my own, I have never got that kind of validation message when I write nothing into the phone number field.

This message is the result of the wrong validation or its bad combination. I would remove

@Digits(fraction = 0, integer = 12)

completely and I would check that only against some sane regexp in the

@Pattern

It is worth to say that more quickstarts which follow the same validation can be affected as well.

dependencies in pom.xml

arquillian-bom of 1.0.3.Final
arquillian-drone-bom of 1.2.0.Alpha1
arquillian-drone-webdriver-depchain
graphene-webdriver of 2.0.0.Alpha3
jboss as 7.1.1.Final