Cloudflare: ‘Perplexity Is Using Stealth, Undeclared Crawlers to Evade Website No-Crawl Directives’

Daring Fireball
By John GruberAug 5, 2025, 6:26 pm419 pts
The Cloudflare blog: We are observing stealth crawling behavior from Perplexity, an AI-powered answer engine. Although Perplexity initially crawls from their declared user agent, when they are presented with a network block, they appear to obscure their crawling identity in an attempt to circumvent the…

Read Article Share
Share Article
- email
- x.com
- facebook
- pocket
- reddit
- tumblr
- linkedin
- pinterest

Trending Today on MacHash

Woah! Two more high-profile execs set to leave Apple. 275

★ Bad Dye Job 259

Sam Altman and Jony Ive lose appeal over ‘io’ name 243

Apple Announces That Kate Adams, Its General Counsel, and Lisa Jackson, Head of Environment, Policy, and Social Initiatives, Are Retiring 241

California Governor sees Apple CEO's dealings with Trump as part of his job 226

tvOS 26.2 receives second release candidate build only a day after the first 208

tvOS 26.2 gets new RC for Apple TV 4K ahead of launch 200

Screen Protectors Without AR Coating Cancel Out iPhone 17's Anti-Reflective Display 197

About MacHash

MacHash is your real-time Apple news aggregator, delivering the latest headlines on Apple, Mac, iPhone, iPad, and iOS from top sources across the web.

As a powerful content discovery platform, MacHash continuously curates breaking news, product announcements, software updates, reviews, and industry insights related to Apple Inc. and its ecosystem.

MacHash helps you stay informed on everything from macOS and iOS developments to Apple Watch, AirPods, and the latest in tech and app innovation.

Access MacHash from your desktop or mobile device to explore, follow, and share the most trusted Apple news all in one place.