Crawlers

PerplexityBot: user agent, robots.txt, and verification

PerplexityBot discovers pages for Perplexity search results. Perplexity explicitly says it is not a crawler for training foundation models. It should not be grouped with training traffic simply because the product uses AI.

User agent and purpose

Documented purpose: Search.

Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

Perplexity’s crawler reference documents this example. Perplexity-User is a distinct user-triggered fetcher, not another spelling of PerplexityBot.

Robots.txt control

User-agent: PerplexityBot
Disallow: /

This group restricts PerplexityBot search crawling. To allow it, remove a blocking rule or deliberately configure an allow group after reviewing your existing file. Perplexity says user-requested Perplexity-User fetches generally ignore robots.txt.

Review this example in the context of your existing groups and paths. Publish it at the origin’s /robots.txt and check the actual response. A policy change can take time to be read; it does not erase earlier copies of content.

Verify origin before trusting the name

Use the PerplexityBot IP list for this token and the separate Perplexity-User list for that fetcher. A broad cloud-provider address range is not an equivalent identity check.

A User-Agent is self-reported and can be spoofed. A matching header is a claimed identity, not an authenticated bot. Use the operator’s current IP or DNS procedure for origin confidence; those checks are not cryptographic request signatures and do not authorize access to private data.

What AnalyticFast detects today

AnalyticFast does not currently classify PerplexityBot in its named crawler aggregate. This directory entry is research, not a claim of detection support. AnalyticFast does not authenticate bot identities or perform the IP/rDNS checks described here.

A useful next check

For search reachability, test a public article and confirm that your robots file and edge rules both permit the intended traffic. A crawl does not guarantee a citation; measure actual referrals and purchases separately.

Official sources and review date

• Perplexity bot purposes, robots behavior, and WAF guidance

Last checked 2026-10-31. Verify the latest policy and network data before changing production access rules.

Back to the crawler directory.