Cloudflare and OpenAI Pilot Enhances AI Discovery of Web Content

In July 2026, Cloudflare and OpenAI launched a research pilot that leveraged Cloudflare’s network signals to help ChatGPT’s search crawler find previously unknown web pages. Using a 1 % cohort of Cloudflare’s traffic, the pilot enabled OpenAI to discover and index hundreds of millions of new pages, giving ChatGPT users access to more up‑to‑date and relevant sources. The collaboration introduced filters that detect non‑existent pages, redirects, and duplicate content, reducing wasted crawl effort and lowering server load for site owners. It also added signals to flag potentially harmful pages, allowing OpenAI to keep such content out of its index while still supporting legitimate sites. The pilot’s early results show that AI systems can now surface more useful, accurate information and link users directly to original publishers.