dkirwan/infra-ansible
1
0
Fork 0
forked from infra/ansible
Code Pull requests Activity

proxies / wiki: add another bot to being blocked

Browse source 
This bot seems to be hitting the wiki really hard.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This commit is contained in:
Kevin Fenzi 2026-04-13 13:58:56 -07:00
commit d281e4ad31
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
roles/httpd/reverseproxy/templates/reversepassproxy.fp-wiki.conf
View file

@ -11,7 +11,7 @@ RewriteRule .* https://%{HTTP_HOST}%{REQUEST_URI} [R=301,NE,L]
# Spiders-gone-wild
# These spiders may not follow robots.txt and will
# hit admin sections which consume large amounts of CPU
RewriteCond %{HTTP_USER_AGENT} "(Bytespider|ClaudeBot|Amazonbot|YandexBot|ChatGLM-Spider|GPTBot|Barkrowler|YisouSpider|MJ12bot)" [NC]
RewriteCond %{HTTP_USER_AGENT} "(Bytespider|ClaudeBot|Amazonbot|YandexBot|ChatGLM-Spider|GPTBot|Barkrowler|YisouSpider|MJ12bot|Avengineers-Crawler)" [NC]
RewriteRule .* - [F,L]
# /wiki must come before /w due to prefix matching.