summaryrefslogtreecommitdiffstats
path: root/roles/httpd/website/files/robots
Commit message (Expand)AuthorAgeFilesLines
* Disallow robots from expensive queries on koscheiMichael Simacek2017-02-241-0/+2
* Indexing /datagrepper is uselessPatrick Uiterwijk2016-02-111-0/+1
* Robots have no use in fedoracommunity as its just an aggregatorPatrick Uiterwijk2015-09-161-0/+3
* Just disallow /updates. Not much point in bodhi updates being in search engines.Kevin Fenzi2015-04-231-0/+1
* Also tell riddler we don't like themPatrick Uiterwijk2015-04-231-0/+4
* Drop crawl-delay down to 1Kevin Fenzi2015-01-292-2/+3
* Working on the proxies-websites.yml playbook.Ralph Bean2014-12-075-0/+20