Re: [keycloak-dev] Testing with mocking libraries?

You need both sets of testing to maintain the long term quality of a security project like Keycloak. :-)

I agree on the mocked IDP approach being more effective. On Thu, May 31, 2018 at 1:20 PM, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > With regards to Identity Brokering and Social Login one potentially > better way to do it would be to have a mocked IdP. At least then you are > testing the actual HTTP calls and it's much closer to a real scenario than > doing junit tests with mocks and only testing parts of the code base. > > On 31 May 2018 at 18:08, Anil Saldanha <asaldanha1947(a)gmail.com&gt; wrote: > >> I am sorry I have not looked at your integration tests. So I may be >> wrong here. <namaste/> >> >> One area that I feel that you need lots of integration tests is in the >> "Identity Brokering" feature. You have way too many permutations and >> combinations (with authentication flows) that without proper integration >> tests/mocks, you will not do proper justice to the value of KC/RH-SSO. >> >> On Thu, May 31, 2018 at 10:17 AM, Bill Burke <bburke(a)redhat.com&gt; wrote: >> >>> Never liked mocks either as they are just opinionated warped >>> perceptions of reality. But there are cases where I think we need >>> them off the top of my head: >>> >>> * Openshift integration >>> * Social logins. >>> >>> Openshift we just won't have the infrastructure for it. Social login >>> tests require accounts which we won't want to share the logins for in >>> a public CI. We need something in public CI so that we can catch at >>> least some potential problems and that we can easily try to reproduce >>> in local IDE environments. Mocks wouldn't be a replacement for real >>> integration testing, just a stop gap. >>> >>> >>> >>> On Thu, May 31, 2018 at 10:44 AM, Anil Saldanha < >>> asaldanha1947(a)gmail.com&gt; wrote: >>> > You really have to test your core logic (with no server set up) using >>> > integration tests with mocking libraries. One way to test out all the >>> > possible preconditions to your logic. >>> > >>> > You will be doing a service to your customers and users by shielding >>> them >>> > from those nasty exceptions (NPEs) that happen in an integration >>> platform >>> > such as KeyCloak/RH-SSO. :-) >>> > >>> > /me wearing a user/customer hat >>> > >>> > On Thu, May 31, 2018 at 1:38 AM, Marek Posolda <mposolda(a)redhat.com&gt; >>> wrote: >>> > >>> >> Not sure. I am not strongly against it, but I afraid that if we >>> >> introduce some mocks support, people (especially from community) will >>> >> start to test with mocks everywhere and add them to all PRs (even to >>> >> those which could be easily tested properly). Not sure if it's better >>> >> alternative to skip automated test at all instead of test with mocks? >>> >> >>> >> But for this one, I guess we can likely extend our social test with >>> an >>> >> account from hosted domain and hence test it properly? >>> >> >>> >> Marek >>> >> >>> >> >>> >> On 30/05/18 09:19, Stian Thorgersen wrote: >>> >> > At the moment our testing strategy is only with functional or >>> integration >>> >> > level tests. Both with the full server up and running and primarily >>> >> testing >>> >> > through public APIs. >>> >> > >>> >> > Now here comes the question should we also allow testing through a >>> >> mocking >>> >> > library like Mockito? >>> >> > >>> >> > In general I'm against mocking libraries. At best you end up with >>> >> something >>> >> > that may work, but you're not guaranteed that it actually works. >>> They >>> >> also >>> >> > have a very big maintenance cost when any changes are made to the >>> >> codebase. >>> >> > >>> >> > However, take a look at https://github.com/keycloak/ke >>> ycloak/pull/5215 >>> >> for >>> >> > example. It is a contribution to add support for the hd param to >>> the >>> >> Google >>> >> > login. Not sure how else we could test this without a mocking >>> library. >>> >> > _______________________________________________ >>> >> > keycloak-dev mailing list >>> >> > keycloak-dev(a)lists.jboss.org >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >>> >> >>> >> _______________________________________________ >>> >> keycloak-dev mailing list >>> >> keycloak-dev(a)lists.jboss.org >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >>> > _______________________________________________ >>> > keycloak-dev mailing list >>> > keycloak-dev(a)lists.jboss.org >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> >>> -- >>> Bill Burke >>> Red Hat >>> >> >> >