Patreon Cracks Down on AI Scraping with Cloudflare Block

Patreon, the membership platform for creators, is taking a firmer stance against AI scraping of its content for training purposes. The company is working with internet infrastructure provider Cloudflare to directly block access to AI bots designed to train their AI models on creators' work without permission. Patreon says the strengthened measures were necessary because AI scraping has become more sophisticated since it first put measures in place to deter AI crawlers in 2023.

The company's paywall has long locked much of creators' content out of reach of crawlers, but new discovery tools like a redesigned Home Feed and its tweet-like Quips may have exposed more content to crawlers. The changes come as online publishers and content creators are coming to grips with how AI is ingesting their work to make AI models smarter. Cloudflare now offers tools that allow website publishers to restrict AI bots, including a marketplace that lets websites charge AI bots for scraping, dubbed Pay Per Crawl.

Earlier this month, it changed its policies so that 'mixed-use' crawlers, meaning those that both index and train on a website's content, are blocked by default on any pages that host ads. Patreon is extending its existing work with Cloudflare to use the company's AI Crawl Control technology to update its AI policies and enforcement tools. Instead of simply asking AI crawlers not to scrape content using robots.txt files, Patreon is now actively blocking AI training bots.

'Consent shouldn't depend on whether a scraper chooses to behave,' a Patreon blog post explains, referencing the stricter measures. When testing the features, individual AI training crawlers' weekly attempts to access Patreon went from 'thousands of attempts to zero,' the post noted. That indicates that the AI scrapers were ignoring Patreon's robots.txt file and scraping the site anyway, despite its requests.

However, the company said it will allow bots that index pages and organize information that can be used to send users back to Patreon. 'As AI agents become increasingly powerful and popular, creators deserve a meaningful say in how their work is used by AI companies,' remarked Patreon's product chief Drew Rowny in the announcement. 'On most of the Internet, creators have to accept AI training on their work just to reach and grow an audience.

Patreon has a different vision: creators should be able to grow their audience and control how their work is used.' Why this matters: Patreon's move to block AI scraping sets a precedent for content creators and platforms to control how their work is used by AI companies. This shift could have significant implications for the broader industry, as more companies may follow suit to protect their content. For developers and businesses, this means they will need to adapt to new restrictions on accessing online content for AI training.