<div dir="ltr">I fixed the tests: <a href="https://issues.jboss.org/browse/KEYCLOAK-2993" target="_blank">https://issues.jboss.org/browse/KEYCLOAK-2993</a><div><br></div><div>There were two basic issues:<div><br></div><div>1) Usage of Time object on the client. We added a mechanism to set offset on the server - it's done by using setTimeOffset() of AbstractKeycloakTest rather than Time.setOffset().</div><div>However, we forgot that client can also sometimes directly rely on Time object - for one, admin-client uses it to check if token needs to be refreshed before doing a remote call. Also, JsonWebToken has validation methods that rely on Time object, and is also used inside tests. Thus, client-side Time.offset, and server-side Time.offset have to be in sync for things to work properly. Since Undertow runs embedded in the same JVM where tests execute there was no problem, but for Wildfly there are two JVMs then, and any Time.offset mismatch starts wreaking havoc.</div><div><br></div><div>2) Installation of some testing providers was not done through testsuite-providers module, but instead used tests/base directly - which works for embedded Undertow which loads from classpath, but not for running with Wildfly which loads from modules.</div><div><br></div><div><br></div><div><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 11, 2016 at 2:33 PM, Marko Strukelj <span dir="ltr"><<a href="mailto:mstrukel@redhat.com" target="_blank">mstrukel@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>There are currently many tests failing when running:</div><div><br></div><div>mvn clean install -Pdistribution -DskipTests</div><div>cd testsuite/integration-arquillian</div><div>mvn clean install -Pauth-server-wildfly</div><div><br></div>Failed tests: <br> ProvidersTest.testClientAuthenticatorProviders:85->compareProviders:178 Providers count expected:<3> but was:<2><br> ProvidersTest.testInitialAuthenticationProviders:130->compareProviders:178 Providers count expected:<20> but was:<18><br> ProvidersTest.testPerClientConfigDescriptions:93 null<br> CustomFlowTest.clientAuthTest:209->grantAccessToken:235 expected:<200> but was:<400><br> CustomFlowTest.grantTest:202->grantAccessToken:235 expected:<200> but was:<400><br> RefreshTokenTest.refreshTokenRequest:154 <div>Expected: (a value equal to or greater than <1799> and a value less than or equal to <1800>)<br> but: a value equal to or greater than <1799> <1798> was less than <1799><br> RefreshTokenTest.refreshTokenReuseTokenWithRefreshTokensRevoked:261 expected:<400> but was:<200><br> RefreshTokenTest.refreshTokenUserSessionMaxLifespan:456 expected:<400> but was:<200><br> RefreshTokenTest.testUserSessionRefreshAndIdle:398 Values should be different. Actual: 1462968567<br><br>Tests in error: <br> CustomFlowTest.loginSuccess:193 » IllegalArgument No enum constant org.keycloa...<br> CustomRegistrationFlowTest.registerUserSuccess:98 » IllegalArgument No enum co...<br> ResetPasswordTest.resetPasswordExpiredCode:386 » NotAuthorized HTTP 401 Unauth...<br> ResetPasswordTest.resetPasswordExpiredCodeShort:430 » NotAuthorized HTTP 401 U...<br> ResetPasswordTest.resetPasswordWithPasswordHisoryPolicy:575->resetPassword:267 » NotAuthorized<br> OfflineTokenTest.offlineTokenAllowedWithCompositeRole:428->offlineTokenDirectGrantFlow:290 » Runtime<br> OfflineTokenTest.offlineTokenBrowserFlow:210 » Runtime Failed to verify token<br> OfflineTokenTest.offlineTokenDirectGrantFlow:311->testRefreshWithOfflineToken:255 » Runtime<br> OfflineTokenTest.offlineTokenDirectGrantFlowWithRefreshTokensRevoked:325 » Runtime<br> OfflineTokenTest.offlineTokenServiceAccountFlow:371 » Runtime Failed to verify...<br><br>Tests run: 480, Failures: 9, Errors: 10, Skipped: 3</div><div><br></div><div>It's probably just server-side changes that require maintenance in the tests themselves.</div><div><br></div><div>We should maybe configure Travis CI with -Pauth-server-wildfly option so that we detect these right away.<br></div><div><br></div></div>
</blockquote></div><br></div>