[keycloak-dev] Failing integration-arquillian tests when using -Pauth-server-wildfly

[Vlasta Ramik]

FYI

There is 
https://keycloak-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/keycloak-deploy/

It checkouts 1.9.x branch and builds keycloak daily. If it succeeds 
artifacts are deployed to nexus staging repo and 
https://keycloak-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/keycloak-parent/ 
job is triggered. This is multi-job and one of its jobs is 
https://keycloak-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/keycloak-wildfly/. 
It run base testsuite on wildfly.

So if you want you can check test results of the job regularly to be 
kept informed.

On 05/12/2016 10:38 AM, Marko Strukelj wrote:
> I fixed the tests: https://issues.jboss.org/browse/KEYCLOAK-2993
>
> There were two basic issues:
>
> 1) Usage of Time object on the client. We added a mechanism to set 
> offset on the server - it's done by using setTimeOffset() of 
> AbstractKeycloakTest rather than Time.setOffset().
> However, we forgot that client can also sometimes directly rely on 
> Time object - for one, admin-client uses it to check if token needs to 
> be refreshed before doing a remote call. Also, JsonWebToken has 
> validation methods that rely on Time object, and is also used inside 
> tests. Thus, client-side Time.offset, and server-side Time.offset have 
> to be in sync for things to work properly. Since Undertow runs 
> embedded in the same JVM where tests execute there was no problem, but 
> for Wildfly there are two JVMs then, and any Time.offset mismatch 
> starts wreaking havoc.
>
> 2) Installation of some testing providers was not done through 
> testsuite-providers module, but instead used tests/base directly - 
> which works for embedded Undertow which loads from classpath, but not 
> for running with Wildfly which loads from modules.
>
>
>
>
> On Wed, May 11, 2016 at 2:33 PM, Marko Strukelj <mstrukel at redhat.com 
> <mailto:mstrukel at redhat.com>> wrote:
>
>     There are currently many tests failing when running:
>
>     mvn clean install -Pdistribution -DskipTests
>     cd testsuite/integration-arquillian
>     mvn clean install -Pauth-server-wildfly
>
>     Failed tests:
>     ProvidersTest.testClientAuthenticatorProviders:85->compareProviders:178
>     Providers count expected:<3> but was:<2>
>     ProvidersTest.testInitialAuthenticationProviders:130->compareProviders:178
>     Providers count expected:<20> but was:<18>
>       ProvidersTest.testPerClientConfigDescriptions:93 null
>     CustomFlowTest.clientAuthTest:209->grantAccessToken:235
>     expected:<200> but was:<400>
>       CustomFlowTest.grantTest:202->grantAccessToken:235
>     expected:<200> but was:<400>
>       RefreshTokenTest.refreshTokenRequest:154
>     Expected: (a value equal to or greater than <1799> and a value
>     less than or equal to <1800>)
>          but: a value equal to or greater than <1799> <1798> was less
>     than <1799>
>     RefreshTokenTest.refreshTokenReuseTokenWithRefreshTokensRevoked:261 expected:<400>
>     but was:<200>
>     RefreshTokenTest.refreshTokenUserSessionMaxLifespan:456
>     expected:<400> but was:<200>
>       RefreshTokenTest.testUserSessionRefreshAndIdle:398 Values should
>     be different. Actual: 1462968567
>
>     Tests in error:
>       CustomFlowTest.loginSuccess:193 » IllegalArgument No enum
>     constant org.keycloa...
>       CustomRegistrationFlowTest.registerUserSuccess:98 »
>     IllegalArgument No enum co...
>       ResetPasswordTest.resetPasswordExpiredCode:386 » NotAuthorized
>     HTTP 401 Unauth...
>       ResetPasswordTest.resetPasswordExpiredCodeShort:430 »
>     NotAuthorized HTTP 401 U...
>     ResetPasswordTest.resetPasswordWithPasswordHisoryPolicy:575->resetPassword:267
>     » NotAuthorized
>     OfflineTokenTest.offlineTokenAllowedWithCompositeRole:428->offlineTokenDirectGrantFlow:290
>     » Runtime
>       OfflineTokenTest.offlineTokenBrowserFlow:210 » Runtime Failed to
>     verify token
>     OfflineTokenTest.offlineTokenDirectGrantFlow:311->testRefreshWithOfflineToken:255
>     » Runtime
>     OfflineTokenTest.offlineTokenDirectGrantFlowWithRefreshTokensRevoked:325
>     » Runtime
>       OfflineTokenTest.offlineTokenServiceAccountFlow:371 » Runtime
>     Failed to verify...
>
>     Tests run: 480, Failures: 9, Errors: 10, Skipped: 3
>
>     It's probably just server-side changes that require maintenance in
>     the tests themselves.
>
>     We should maybe configure Travis CI with -Pauth-server-wildfly
>     option so that we detect these right away.
>
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160512/70ad39f5/attachment-0001.html