[keycloak-dev] Failing integration-arquillian tests when using -Pauth-server-wildfly

[Marko Strukelj]

There are currently many tests failing when running:

mvn clean install -Pdistribution -DskipTests
cd testsuite/integration-arquillian
mvn clean install -Pauth-server-wildfly

Failed tests:
  ProvidersTest.testClientAuthenticatorProviders:85->compareProviders:178
Providers count expected:<3> but was:<2>

ProvidersTest.testInitialAuthenticationProviders:130->compareProviders:178
Providers count expected:<20> but was:<18>
  ProvidersTest.testPerClientConfigDescriptions:93 null
  CustomFlowTest.clientAuthTest:209->grantAccessToken:235 expected:<200>
but was:<400>
  CustomFlowTest.grantTest:202->grantAccessToken:235 expected:<200> but
was:<400>
  RefreshTokenTest.refreshTokenRequest:154
Expected: (a value equal to or greater than <1799> and a value less than or
equal to <1800>)
     but: a value equal to or greater than <1799> <1798> was less than
<1799>
  RefreshTokenTest.refreshTokenReuseTokenWithRefreshTokensRevoked:261
expected:<400> but was:<200>
  RefreshTokenTest.refreshTokenUserSessionMaxLifespan:456 expected:<400>
but was:<200>
  RefreshTokenTest.testUserSessionRefreshAndIdle:398 Values should be
different. Actual: 1462968567

Tests in error:
  CustomFlowTest.loginSuccess:193 » IllegalArgument No enum constant
org.keycloa...
  CustomRegistrationFlowTest.registerUserSuccess:98 » IllegalArgument No
enum co...
  ResetPasswordTest.resetPasswordExpiredCode:386 » NotAuthorized HTTP 401
Unauth...
  ResetPasswordTest.resetPasswordExpiredCodeShort:430 » NotAuthorized HTTP
401 U...

ResetPasswordTest.resetPasswordWithPasswordHisoryPolicy:575->resetPassword:267
» NotAuthorized

OfflineTokenTest.offlineTokenAllowedWithCompositeRole:428->offlineTokenDirectGrantFlow:290
» Runtime
  OfflineTokenTest.offlineTokenBrowserFlow:210 » Runtime Failed to verify
token

OfflineTokenTest.offlineTokenDirectGrantFlow:311->testRefreshWithOfflineToken:255
» Runtime
  OfflineTokenTest.offlineTokenDirectGrantFlowWithRefreshTokensRevoked:325
» Runtime
  OfflineTokenTest.offlineTokenServiceAccountFlow:371 » Runtime Failed to
verify...

Tests run: 480, Failures: 9, Errors: 10, Skipped: 3

It's probably just server-side changes that require maintenance in the
tests themselves.

We should maybe configure Travis CI with -Pauth-server-wildfly option so
that we detect these right away.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160511/43974845/attachment-0001.html