[teiid-issues] [JBoss JIRA] (TEIID-5993) Add robots.txt to better crawl content

[Steven Hawkins (Jira)]

    [ https://issues.redhat.com/browse/TEIID-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14261266#comment-14261266 ] 

Steven Hawkins commented on TEIID-5993:
---------------------------------------

This isn't just a quick change.  You don't appear to get control over the root of the github subdomain from any one repo.  So we would probably need to update the dns records for teiid.io and start hosting the docs under that roo.

> Add robots.txt to better crawl content
> --------------------------------------
>
>                 Key: TEIID-5993
>                 URL: https://issues.redhat.com/browse/TEIID-5993
>             Project: Teiid
>          Issue Type: Quality Risk
>          Components: Documentation
>            Reporter: Steven Hawkins
>            Assignee: Steven Hawkins
>            Priority: Major
>   Original Estimate: 1 hour
>  Remaining Estimate: 1 hour
>
> We should do a better job of directing what pages are crawled.  I added a robots.txt to prevent the crawling of legacy.  Something similar should be added for teiid.github.com to allow only the crawling of master.



--
This message was sent by Atlassian Jira
(v7.13.8#713008)