Book a demo →

PerplexityBot

Manuel Riveiro SEO & GEO consultant
Technical access 3 min · reviewed 15 Sept 2026

Definition

PerplexityBot is the crawler Perplexity uses to surface and link websites in its search results. Perplexity says it is not used to crawl content for AI foundation models. It is separate from Perplexity-User, which visits pages when a user asks a question.

In short

What Perplexity documents about PerplexityBot and Perplexity-User, which of the two robots.txt addresses, how to confirm a request with the published IP lists, and what Cloudflare and Perplexity said about each other in August 2025.

What Perplexity documents

Perplexity describes its crawlers on the page "Perplexity Crawlers" in its documentation. The page metadata gives 29 January 2026 as its last modification; the quotes here are as retrieved on 15 September 2026.

"PerplexityBot is designed to surface and link websites in search results on Perplexity. It is not used to crawl content for AI foundation models."

For a site to appear in those results, Perplexity recommends allowing PerplexityBot in robots.txt and "permitting requests from our published IP ranges". Changes are not immediate: "Each setting works independently, and it may take up to 24 hours for our systems to reflect changes."

PerplexityBot and Perplexity-User

The same page lists a second user agent with a different job:

"Perplexity-User supports user actions within Perplexity. When users ask Perplexity a question, it might visit a web page to help provide an accurate answer and include a link to the page in its response."

For robots.txt the difference is decisive. Of Perplexity-User, Perplexity writes: "Since a user requested the fetch, this fetcher generally ignores robots.txt rules." It adds that Perplexity-User "is not used for web crawling or to collect content for training AI foundation models."

A robots.txt rule like the one below therefore addresses the crawler for Perplexity's search results:

User-agent: PerplexityBot
Disallow: /

By Perplexity's own description, such a rule does not reliably keep a page away from visits that a user's question triggers.

One crawls for search results, the other fetches when someone asks.

Confirming a request, and firewall rules

Perplexity publishes the full user-agent strings, for PerplexityBot:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

Each user agent has its own IP list, https://www.perplexity.com/perplexitybot.json and https://www.perplexity.com/perplexity-user.json. On 15 September 2026 the first listed 8 addresses with a creation time of 7 February 2025, the second 4 addresses with a creation time of 17 October 2025. Perplexity calls these endpoints "the source of truth for your WAF configurations", and its advice for firewall rules is to "combine both User-Agent string matching and IP address verification".

The documentation describes such rules for letting Perplexity's bots through a web application firewall, with steps for Cloudflare and AWS WAF. A rule that blocks them would match on the same two fields.

Check the user agent and the address together, as Perplexity advises.

The dispute of August 2025

On 4 August 2025 Cloudflare published a report titled "Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives". It said:

"Although Perplexity initially crawls from their declared user agent, when they are presented with a network block, they appear to obscure their crawling identity in an attempt to circumvent the website’s preferences."

Cloudflare also wrote that it had "de-listed them as a verified bot". Perplexity answered in a post dated the same day, "Agents or Bots? Making Sense of AI on the Open Web". It argued that assistants acting on a user's request are not crawlers, rejected Cloudflare's analysis and wrote:

"It appears Cloudflare confused Perplexity with 3-6M daily requests of unrelated traffic from BrowserBase, a third-party cloud browser service that Perplexity only occasionally uses for highly specialized tasks (less than 45,000 daily requests)."

Both texts are statements by the companies involved, and this entry does not decide between them. What a site owner can use is the documentation above and their own logs.

Checking your own logs

  • Search the access log for PerplexityBot and Perplexity-User separately. Perplexity documents different robots.txt behaviour for the two.
  • Compare each IP address with the matching JSON list. A request that names PerplexityBot from another address is not confirmed as Perplexity.
  • For paths disallowed for PerplexityBot, look for confirmed PerplexityBot requests after the 24 hours Perplexity gives for changes.
  • If the site is meant to appear in Perplexity's results, check that no firewall or CDN rule blocks the published addresses.

Common mistakes

  • Expecting a robots.txt rule for PerplexityBot to stop visits triggered by a user's question. Perplexity documents that Perplexity-User generally ignores robots.txt.
  • Treating a user-agent string in the log as proof. Without the IP check it is only a claim.
  • Blocking PerplexityBot to keep content out of model training. Perplexity says the crawler is not used for that.
  • Allowing PerplexityBot in robots.txt while a firewall blocks its addresses.
  • Repeating one side of the August 2025 dispute as settled fact.
  • OAI-SearchBot: OpenAI's crawler for ChatGPT search, documented with a similar split between crawler and user-triggered visits.
  • GPTBot: OpenAI's crawler for content that may be used in model training.
  • ClaudeBot: Anthropic's crawler, controlled by its own robots.txt token.
  • robots.txt for AI Crawlers: the file in which PerplexityBot is allowed or disallowed.
  • Answer Engine: a search system built to answer a question directly rather than list links.

Source

Perplexity documentation: "Perplexity Crawlers" (retrieved 15 September 2026; page metadata dateModified 2026-01-29). Source of the quotes on PerplexityBot and Perplexity-User, the 24 hours, the user-agent string, the IP endpoints and the firewall advice. perplexity.com/perplexitybot.json and perplexity-user.json: published IP addresses (retrieved 15 September 2026; creationTime 2025-02-07 with 8 addresses and 2025-10-17 with 4 addresses). Cloudflare Blog: "Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives" (4 August 2025; retrieved 15 September 2026). Source of the Cloudflare quotes. Perplexity blog: "Agents or Bots? Making Sense of AI on the Open Web" (4 August 2025; retrieved 15 September 2026). Source of Perplexity's response.

Frequently asked questions

Does PerplexityBot follow robots.txt?

Perplexity documents robots.txt as the way to manage PerplexityBot and says changes can take up to 24 hours. The same documentation says that Perplexity-User, which fetches pages for a user’s question, generally ignores robots.txt rules.

Is PerplexityBot used to train AI models?

Perplexity says no. According to its documentation, PerplexityBot is not used to crawl content for AI foundation models, and Perplexity-User is not used to collect content for training them either.

What is the difference between PerplexityBot and Perplexity-User?

PerplexityBot surfaces and links websites in Perplexity’s search results. Perplexity-User visits a page when a user asks a question and can link that page in the answer.

How do I know a request really comes from Perplexity?

Check the IP address against the list Perplexity publishes for that user agent, perplexitybot.json or perplexity-user.json. Perplexity itself recommends combining the user-agent string with the IP address.

What was the dispute between Cloudflare and Perplexity?

On 4 August 2025 Cloudflare reported that Perplexity appeared to disguise its crawling when its declared bots were blocked. Perplexity rejected the report the same day and said Cloudflare had attributed traffic from a third-party browser service to it. This entry quotes both sides with dates and does not decide between them.