Weighs the soul of incoming HTTP requests to stop AI crawlers

chore(docs/manifest): allow common crawl to test with the team

Signed-off-by: Xe Iaso <me@xeiaso.net>

Xe Iaso 2f1e78cc 7c099644

Changed files
+1
docs
manifest
cfg
+1
docs/manifest/cfg/anubis/botPolicies.yaml
··· 11 11 ## /usr/share/docs/anubis/data or in the tarball you extracted Anubis from. 12 12 13 13 bots: 14 + - import: (data)/crawlers/commoncrawl.yaml 14 15 # Pathological bots to deny 15 16 - # This correlates to data/bots/deny-pathological.yaml in the source tree 16 17 # https://github.com/TecharoHQ/anubis/blob/main/data/bots/deny-pathological.yaml