Something went wrong. Try again.
Monorepo for Tangled forked from tangled.org/core
Something went wrong. Try again.
12345678910111213141516171819202122232425262728293031323334# Bytespider is an aggressive scraper that ignores crawl etiquetteUser-agent: BytespiderDisallow: /
# Explicitly allowed AI search & answer enginesUser-agent: GPTBotUser-agent: ChatGPT-UserUser-agent: ClaudeBotUser-agent: PerplexityBotUser-agent: Google-ExtendedUser-agent: CCBotAllow: /Disallow: /settingsDisallow: /*/settingsDisallow: /*/compareDisallow: /*/forkDisallow: /searchDisallow: /*/searchCrawl-delay: 1
# Default crawler rulesUser-agent: *Allow: /Disallow: /settingsDisallow: /*/settingsDisallow: /*/compareDisallow: /*/forkDisallow: /searchDisallow: /*/searchCrawl-delay: 1
Sitemap: https://tangled.org/sitemap.xmlSitemap: https://tangled.org/blog/sitemap-index.xml