unoffical wafrn mirror wafrn.net
atproto social-network activitypub
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

at api-load-balancing 51 lines 718 B view raw
1# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file 2 3User-agent: GPTBot 4Disallow: / 5 6User-Agent: meta-externalagent 7Disallow: / 8 9User-agent: CCBot 10Disallow: / 11 12User-agent: ChatGPT-User 13Disallow: / 14 15User-agent: Applebot-Extended 16Disallow: / 17 18User-agent: Google-Extended 19Disallow: / 20 21User-agent: anthropic-ai 22Disallow: / 23 24User-agent: Claude-Web 25Disallow: / 26 27User-agent: Claudebot 28Disallow: / 29 30User-agent: Bytespider 31Disallow: / 32 33User-agent: Omgilibot 34Disallow: / 35 36User-agent: ImagesiftBot 37Disallow: / 38 39User-agent: Meltwater 40Disallow: / 41 42User-agent: Seekr 43Disallow: / 44 45User-agent: * 46Disallow / 47Allow: /$ 48Allow: /.well-known/ 49Allow: /article/ 50Allow: /fediverse/ 51Allow: /blog/