Cloudflare has unveiled ‘Markdown for Agents,’ a new feature designed to enhance the efficiency of AI content consumption by converting HTML web pages into Markdown format. When AI agents request content, they can specify their preference for Markdown by including the <code>Accept: text/markdown</code> header in their HTTP requests. This triggers Cloudflare’s network to perform real-time conversions at the edge, substantially reducing token usage by up to 80%. For instance, a typical Cloudflare blog post can drop from 16,180 tokens in HTML to just 3,150 tokens in Markdown.
This capability not only optimizes resource usage for AI models but also provides web publishers with more effective control over their content’s distribution and utilization. Through Cloudflare’s Content Signals Policy framework, publishers can specify their AI usage preferences in <code>robots.txt</code> files via machine-readable directives for training, search, and input scenarios.
'“Markdown for Agents” is currently offered in beta for users on Pro, Business, and Enterprise plans and can be activated via a simple dashboard toggle or API, requiring no modifications to existing site templates or CMS configurations. Additionally, the feature includes response headers that reveal token counts and content-signal policies, facilitating better understanding and decision-making for AI training and indexing.
This approach is part of a broader trend to cater to the rising demand for AI-friendly content formats. While competitors like the WordPress Markdown Alternate plugin, Fasterize EdgeSEO, and Firecrawl also offer similar functionalities, Cloudflare’s solution is distinguished by its seamless integration and real-time edge processing capabilities. The feature empowers coding assistants, AI search engines, and documentation crawlers to process information more accurately, while still serving standard HTML content to human users.

Collections

Cloudflare has launched 'Markdown for Agents' in beta, a feature that converts HTML web pages to Markdown format at the edge when AI agents request it via the `Accept: text/markdown` HTTP header. This can reduce token usage by up to 80% — for example, a blog post shrinks from 16,180 HTML tokens to 3,150 Markdown tokens. Available on Pro, Business, and Enterprise plans, it can be enabled via a dashboard toggle or API with no CMS changes required. The feature also integrates with a Content Signals Policy framework, letting publishers define AI usage preferences in `robots.txt` for training, search, and input scenarios. Response headers expose token counts and content-signal policies to aid AI decision-making.

Cloudflare Enhances AI Efficiency with Markdown for Web Content