Resource Reference

Supported AI Bots Directory

View the list of search, chat, and training crawlers tracked by AIReady.

Tracked Assistants & Chatbots

We actively identify conversational agents that fetch your URLs to construct live citations for search queries:

- **GPTBot & ChatGPT-User**: OpenAI's background and live user query search crawlers.

- **ClaudeBot**: Anthropic's primary conversational agent indexer.

- **PerplexityBot**: Crawler utilized by Perplexity AI search engine.

- **OAI-SearchBot**: OpenAI's dedicated search retrieval agent.

Tracked Training & OS Crawlers

We identify and catalog crawlers gathering web data to train future models:

- **Google-Extended**: Google's optical-out training preference verification token.

- **Applebot**: Apple Intelligence content scanner.

- **Meta AI Bot**: Scanners gathering training data for Meta's Llama models.

- **Bytespider**: Bytedance NLP translation crawler.

Still having trouble with integration?

Reach out to our developer support team directly. We are happy to help get you verified.

Call Support