Does ChatGPT crawl websites?
Quick Summary
Yes. ChatGPT crawls websites using GPTBot for training indexation and ChatGPT-User / OAI-SearchBot for real-time citations.
Detailed Explanation
OpenAI runs a dual-crawler system. GPTBot indexes public pages to train models, while ChatGPT-User and OAI-SearchBot crawl websites in response to live user search queries to provide up-to-date answers and citation links in chat interfaces.
Track crawler traffic in real-time
Find out exactly when chat models or search bots are reading your content. Try AIReady free.