PerplexityBot

PerplexityBot is Perplexity's crawler, responsible for indexing pages into the source pool its answer engine retrieves from and cites, functioning as the crawler equivalent of Perplexity's real-time, citation-first retrieval approach. Unlike some competitors, Perplexity runs one primary crawler rather than separating training and citation functions.

Why it matters

Because Perplexity operates a single crawler for both discovery and citation, blocking PerplexityBot removes a page from Perplexity entirely, with no equivalent of OpenAI's or Anthropic's split between a training-only bot and a citation-specific one.

Related terms

Perplexity

Perplexity is an AI-native search engine built around inline, numbered citations, where nearly every claim in its answers links directly to a specific source. It positions itself explicitly as a citation-first answer engine, and its retrieval favors recently published, directly responsive pages over pages with heavier accumulated editorial authority.

Real-Time Retrieval

Real-time retrieval is Perplexity's defining retrieval strategy: prioritizing recently published, directly responsive sources over pages with heavier accumulated editorial authority. Instead of leaning on a domain's long-term reputation, Perplexity's index favors whichever page answers the immediate query most precisely, updated continuously as new pages get published or existing ones get revised.

robots.txt (AI directives)

robots.txt is the long-standing standard crawler-control file at a site's root, now extended with AI-specific user-agent rules that separately allow or disallow crawlers like GPTBot, ClaudeBot, and PerplexityBot. A single robots.txt file can permit standard search crawling while blocking one or more AI crawlers, or the reverse.

Optimize your presence across Generative AI and Answer Engines.

Track your AI visibility · Get a demo