User-agent: * Allow: / Allow: /listing/ Allow: /city/ Allow: /cities Allow: /category/ Allow: /categories Allow: /blog/ Allow: /blog Allow: /feed/ Allow: /social Allow: /search Allow: /pricing Allow: /about Allow: /contact Allow: /get-listed Allow: /add-listing Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ Disallow: /search?* # AI Search Crawlers — explicitly allowed to index business content User-agent: GPTBot Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ User-agent: ChatGPT-User Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ User-agent: Google-Extended Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ User-agent: PerplexityBot Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ User-agent: ClaudeBot Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ User-agent: anthropic-ai Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ User-agent: CCBot Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ User-agent: Amazonbot Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ User-agent: Applebot Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ # OAI-SearchBot — OpenAI web search (distinct from GPTBot training crawler) User-agent: OAI-SearchBot Allow: /listing/ Allow: /city/ Allow: /blog/ Allow: /feed/ Allow: /category/ Allow: /best-of/ Allow: /pricing Allow: /why-trailblaze Allow: /how-it-works Allow: /llms.txt Allow: /pricing.md Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /api/ Disallow: /auth/ Sitemap: https://trailblazeutah.com/sitemap.xml # AI context files # LLMs context: https://trailblazeutah.com/llms.txt # Pricing (machine-readable): https://trailblazeutah.com/pricing.md