Another option would be to use html META-tags to control robot behaviour.

http://www.metatags.info/meta_name_robots

Cheers,

Thomas

2016-04-13 10:52 GMT+02:00 Stian Thorgersen <sthorger@redhat.com>:

Maybe it should be /robots.txt and /auth/robots.txt. The first will cover the case when Keycloak is exposed directly, while the latter when a reverse proxy is used (for example auth.example.org --> kc-ip/auth).

On 13 April 2016 at 10:47, Thomas Raehalme <thomas.raehalme@aitiofinland.com> wrote:

I think it needs to be in the root folder. 

Best regards,

Thomas

On Wed, Apr 13, 2016 at 11:41 AM, Stian Thorgersen <sthorger@redhat.com> wrote:

Is it sufficient to add that to /auth/robots.txt?

On 13 April 2016 at 10:17, Thomas Raehalme <thomas.raehalme@aitiofinland.com> wrote:

I think the user is running some Bing browser plugin or desktop application which loads a page snapshot/preview of whatever links it finds in user content.

I created KEYCLOAK-2810 to add robots.txt.

https://issues.jboss.org/browse/KEYCLOAK-2810

Best regards,

Thomas

On Wed, Apr 13, 2016 at 11:11 AM, Stian Thorgersen <sthorger@redhat.com> wrote:

+1

How did it find one-time links?

On 13 April 2016 at 10:07, Thomas Raehalme <thomas.raehalme@aitiofinland.com> wrote:

Hi!

What do you think of including robots.txt in the Keycloak distribution to try to avoid Keycloak being indexed? 

We had a nasty issue with BingPreview trying to load URLs and causing one-time links to be invalidated.

Best regards,

Thomas

_______________________________________________
keycloak-dev mailing list
keycloak-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev

_______________________________________________
keycloak-dev mailing list
keycloak-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev