[keycloak-dev] Testing with mocking libraries?

[Anil Saldanha]

You need both sets of testing to maintain the long term quality of a
security project like Keycloak. :-)

I agree on the mocked IDP approach being more effective.

On Thu, May 31, 2018 at 1:20 PM, Stian Thorgersen <sthorger at redhat.com>
wrote:

> With regards to Identity Brokering and Social Login one potentially better
> way to do it would be to have a mocked IdP. At least then you are testing
> the actual HTTP calls and it's much closer to a real scenario than doing
> junit tests with mocks and only testing parts of the code base.
>
> On 31 May 2018 at 18:08, Anil Saldanha <asaldanha1947 at gmail.com> wrote:
>
>> I am sorry I have not looked at your integration tests.  So I may be
>> wrong here. <namaste/>
>>
>> One area that I feel that you need lots of integration tests is in the
>> "Identity Brokering" feature.  You have way too many permutations and
>> combinations (with authentication flows) that without proper integration
>> tests/mocks, you will not do proper justice to the value of KC/RH-SSO.
>>
>> On Thu, May 31, 2018 at 10:17 AM, Bill Burke <bburke at redhat.com> wrote:
>>
>>> Never liked mocks either as they are just opinionated warped
>>> perceptions of reality.   But there are cases where I think we need
>>> them off the top of my head:
>>>
>>> * Openshift integration
>>> * Social logins.
>>>
>>> Openshift we just won't have the infrastructure for it.  Social login
>>> tests require accounts which we won't want to share the logins for in
>>> a public CI.  We need something in public CI so that we can catch at
>>> least some potential problems and that we can easily try to reproduce
>>> in local IDE environments.  Mocks wouldn't be a replacement for real
>>> integration testing, just a stop gap.
>>>
>>>
>>>
>>> On Thu, May 31, 2018 at 10:44 AM, Anil Saldanha <asaldanha1947 at gmail.com>
>>> wrote:
>>> > You really have to test your core logic (with no server set up) using
>>> > integration tests with mocking libraries. One way to test out all the
>>> > possible preconditions to your logic.
>>> >
>>> > You will be doing a service to your customers and users by shielding
>>> them
>>> > from those nasty exceptions (NPEs) that happen in an integration
>>> platform
>>> > such as KeyCloak/RH-SSO. :-)
>>> >
>>> > /me wearing a user/customer hat
>>> >
>>> > On Thu, May 31, 2018 at 1:38 AM, Marek Posolda <mposolda at redhat.com>
>>> wrote:
>>> >
>>> >> Not sure. I am not strongly against it, but I afraid that if we
>>> >> introduce some mocks support, people (especially from community) will
>>> >> start to test with mocks everywhere and add them to all PRs (even to
>>> >> those which could be easily tested properly). Not sure if it's better
>>> >> alternative to skip automated test at all instead of test with mocks?
>>> >>
>>> >> But for this one, I guess we can likely extend our social test with an
>>> >> account from hosted domain and hence test it properly?
>>> >>
>>> >> Marek
>>> >>
>>> >>
>>> >> On 30/05/18 09:19, Stian Thorgersen wrote:
>>> >> > At the moment our testing strategy is only with functional or
>>> integration
>>> >> > level tests. Both with the full server up and running and primarily
>>> >> testing
>>> >> > through public APIs.
>>> >> >
>>> >> > Now here comes the question should we also allow testing through a
>>> >> mocking
>>> >> > library like Mockito?
>>> >> >
>>> >> > In general I'm against mocking libraries. At best you end up with
>>> >> something
>>> >> > that may work, but you're not guaranteed that it actually works.
>>> They
>>> >> also
>>> >> > have a very big maintenance cost when any changes are made to the
>>> >> codebase.
>>> >> >
>>> >> > However, take a look at https://github.com/keycloak/ke
>>> ycloak/pull/5215
>>> >> for
>>> >> > example. It is a contribution to add support for the hd param to the
>>> >> Google
>>> >> > login. Not sure how else we could test this without a mocking
>>> library.
>>> >> > _______________________________________________
>>> >> > keycloak-dev mailing list
>>> >> > keycloak-dev at lists.jboss.org
>>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> keycloak-dev mailing list
>>> >> keycloak-dev at lists.jboss.org
>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> >>
>>> > _______________________________________________
>>> > keycloak-dev mailing list
>>> > keycloak-dev at lists.jboss.org
>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>>
>>>
>>> --
>>> Bill Burke
>>> Red Hat
>>>
>>
>>
>