[keycloak-dev] robots.txt

[Thomas Darimont]

Another option would be to use html META-tags to control robot behaviour.

http://www.metatags.info/meta_name_robots

Cheers,
Thomas

2016-04-13 10:52 GMT+02:00 Stian Thorgersen <sthorger at redhat.com>:

> Maybe it should be /robots.txt and /auth/robots.txt. The first will cover
> the case when Keycloak is exposed directly, while the latter when a reverse
> proxy is used (for example auth.example.org --> kc-ip/auth).
>
> On 13 April 2016 at 10:47, Thomas Raehalme <
> thomas.raehalme at aitiofinland.com> wrote:
>
>> I think it needs to be in the root folder.
>>
>> Best regards,
>> Thomas
>>
>> On Wed, Apr 13, 2016 at 11:41 AM, Stian Thorgersen <sthorger at redhat.com>
>> wrote:
>>
>>> Is it sufficient to add that to /auth/robots.txt?
>>>
>>> On 13 April 2016 at 10:17, Thomas Raehalme <
>>> thomas.raehalme at aitiofinland.com> wrote:
>>>
>>>> I think the user is running some Bing browser plugin or desktop
>>>> application which loads a page snapshot/preview of whatever links it finds
>>>> in user content.
>>>>
>>>> I created KEYCLOAK-2810 to add robots.txt.
>>>> https://issues.jboss.org/browse/KEYCLOAK-2810
>>>>
>>>> Best regards,
>>>> Thomas
>>>>
>>>>
>>>> On Wed, Apr 13, 2016 at 11:11 AM, Stian Thorgersen <sthorger at redhat.com
>>>> > wrote:
>>>>
>>>>> +1
>>>>>
>>>>> How did it find one-time links?
>>>>>
>>>>> On 13 April 2016 at 10:07, Thomas Raehalme <
>>>>> thomas.raehalme at aitiofinland.com> wrote:
>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> What do you think of including robots.txt in the Keycloak
>>>>>> distribution to try to avoid Keycloak being indexed?
>>>>>>
>>>>>> We had a nasty issue with BingPreview trying to load URLs and causing
>>>>>> one-time links to be invalidated.
>>>>>>
>>>>>> Best regards,
>>>>>> Thomas
>>>>>>
>>>>>> _______________________________________________
>>>>>> keycloak-dev mailing list
>>>>>> keycloak-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160413/4e2efb53/attachment.html