[keycloak-dev] Testing with mocking libraries?

[Steffen Kreutz]

You are right. Managing accounts with a custom domain requires a G Suite subscription which starts at 5$/month.

> Am 06.06.2018 um 15:22 schrieb Stian Thorgersen <sthorger at redhat.com>:
> 
> I think we're in agreement here that the ideal is to do proper integration
> tests. Rather than open the doors for mocks (java based or http based) we
> should attempt to do it proper. For Google I'm not sure how realistic that
> is as the hd param probably requires an corporate account rather than a
> freebie account that we are using today for testing. We should investigate
> that first though.
> 
> On 6 June 2018 at 10:03, Hynek Mlnarik <hmlnarik at redhat.com> wrote:
> 
>> To answer the original question whether to mock or not to mock, I wouldn't
>> object allowing those in unit tests (which we have not that many though
>> until now there's no need to them and I hope we can preserve this state).
>> I'd rather not introduce those in integration tests because of maintenance
>> costs.
>> 
>> The biggest issue I see with mocks is that one needs to be extra careful to
>> return exactly what's expected from the specification/counterparty.
>> Since social
>> providers  can change without prior notice, our mocked IdPs might give us
>> false impression that we're testing them. Furthermore, such mocks would be
>> rather complex, and would require non-trivial maintenance. I'm hence biased
>> against mock IdPs and prefer the tests with real IdPs as mentioned by
>> Marek.
>> 
>> 
>> On Mon, Jun 4, 2018 at 11:22 AM, Marek Posolda <mposolda at redhat.com>
>> wrote:
>> 
>>> Dne 1.6.2018 v 14:48 Bill Burke napsal(a):
>>>> On Fri, Jun 1, 2018 at 5:18 AM, Marek Posolda <mposolda at redhat.com>
>>> wrote:
>>>>> For IDP, I am not sure how would Mocked IDP help? We already have lots
>>> of
>>>>> test coverage for Identity brokering for OIDC and SAML. The social
>>> providers
>>>>> usually implements OAuth/OIDC and there are just small differences
>>> between
>>>>> them. But those small differences and the fact that social providers
>>> are out
>>>>> of our control, is exactly the reason why we need to test with real
>>>>> providers instead of mocks IMO.
>>>>> 
>>>> IDP mocks are not a replacement for real testing.  Just something that
>>>> would run in public CI, that way at least there's something to catch
>>>> issues like when/if brokering gets refactored again.
>>> AFAIK the longer-term plan is to run social tests on every PR, to catch
>>> regressions by testing with real social providers. So maybe it's better
>>> to rather wait until we have that rather then introduce mocks now?
>>> 
>>> Also anyone from community should be able to run social tests based on
>>> our README. The important is, that someone could run it just for single
>>> provider, which he is interested in (EG. Google for this PR) and is not
>>> required to create the account+application in all social providers,
>>> which we support. But I hope the social tests already support this now.
>>> 
>>> Marek
>>>> 
>>>> Bill
>>> 
>>> 
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> 
>> 
>> 
>> 
>> --
>> 
>> --Hynek
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> 
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev