Our own build · Red Bridge tool
A Markdown generator that serves any page to AI agents
AI agents want clean Markdown, not a page full of navigation, scripts, and chrome. This is a WordPress plugin we built that hands back any page as distilled Markdown, generated on demand and cached. It's a Red Bridge tool. We run it for clients like Neo4j and SnapLogic.
Why we built it
Cloudflare's agent-readiness score, without renting the fix
Cloudflare put out a test, isitagentready.com, that scores how ready a site is for AI agents. One thing it checks: can the site serve its pages as Markdown. Cloudflare sells that as a service, their Markdown-for-agents negotiation.
We rolled our own instead, as a WordPress plugin we control. It works, and it solves the score. The code is ours, it runs on the same host as the site, and there's no third party sitting between an agent and the page.
The big idea
One page, two faces
The same URL can answer as a rich HTML page for people, or as distilled Markdown for machines. The plugin intercepts the request, fetches the rendered page, strips the navigation, scripts, and chrome, and hands back just the content with structured front-matter on top.
--- title: "How agents read your site" url: "https://yoursite.com/blog/how-agents-read/" date: "2026-07-12 14:02:11" type: "post" --- # How agents read your site Agents want the content, not the chrome. Markdown gives them the text, headings, and links with nothing in the way. ## Why it matters See the [docs](https://yoursite.com/docs) for the full walkthrough.
Entry points
Three ways a page becomes Markdown
The .md suffix
Append .md to any page, post, or video URL. /index.md and /.md resolve to the homepage.
Content negotiation
Send an Accept: text/markdown header on the normal URL and the same content comes back, no suffix required.
Archive listings
Post-type archives and the blog index resolve too, fetched by URL and labelled type: "archive".
A template_redirect hook catches the request before WordPress renders a normal page, maps the path back to a post ID, and returns Markdown with a text/markdown content type, a Vary: Accept header, and X-Robots-Tag: noindex.
Under the hood
The conversion pipeline
Click a stage to see what happens. The work is split across three small classes plus the theme router.
Route the request
lib/markdown-routing.php A template_redirect hook catches the request before WordPress renders a normal page. It strips the .md suffix, maps the path back to a post ID with url_to_postid(), and confirms the post is published and not password-protected. Anything it can't match returns a clean 404.
- Homepage special-cased (page_on_front)
- Archives matched against registered post types
- Bails to 404 if the generator isn't loaded
Fetch the rendered HTML
PageFetcher Rather than rebuild the content from the database, it requests the live, fully-rendered permalink with wp_remote_get() on a 10s timeout. That way the Markdown reflects exactly what a visitor sees: Gutenberg blocks, dynamic templates, and all.
- Real HTTP fetch of the published page
- Returns empty on any request error, which becomes a 404
Clean and normalize
ContentParser The heavy lifting. Everything above the content start marker is dropped, then scripts, styles, footers, modals, screen-reader-only nodes, and in-page nav lists are removed. Site-specific block patterns get rewritten into something an agent can actually read.
- Relative links and images made absolute
- Video modal blocks become plain video URLs
- PDF embeds become direct links
- Accordion and card wrappers unwrapped
Convert to Markdown
MarkdownGenerator · league/html-to-markdown The cleaned HTML runs through the League HTML-to-Markdown converter (ATX headings, table support, hard breaks). A series of regex passes then polish the output, preserving code blocks, tidying link syntax, and collapsing excess whitespace.
- iframe embeds become canonical video links
- br inside pre becomes real newlines
- Bare-URL links normalized
- Empty and alt-less images stripped
Add front-matter
MarkdownGenerator Each document is topped with a YAML block so an agent knows what it's reading: title, canonical URL, last-modified date, and content type. Video posts get a video_url; archive pages are tagged type: "archive".
- Title from og:title, falling back to the title tag
- Machine-parseable YAML
Cache and serve
plugin bootstrap Generating Markdown is expensive, since it re-fetches the live page, so each result is stored in a WordPress transient for a week and returned with caching headers. Generation only happens on a cache miss, so repeat requests are nearly free.
- Cache-Control set for the CDN
- noindex, nofollow, so it never competes with the HTML page in search
Smart cleanup
Built for messy, real-world pages
A generic HTML-to-Markdown library chokes on a marketing site full of interactive blocks. The parser handles the awkward cases so the output stays readable.
block + shortcode "Watch video" buttons that open a lightbox become a plain video link an agent can actually follow.
unwrap Anchors wrapping a whole heading plus paragraph are unwrapped and given a clean link, instead of broken multi-line Markdown.
href="#" Toggle triggers become bold question text followed by their answer, instead of dead (#) links.
strip Lists that only hold on-page #anchor jump-links are dropped. They're navigation, not content.
data-src Inline PDF viewers are replaced with a direct link to the document.
UTF-8 Targeted entity fixes avoid corrupting quotes and dashes that a broad decode would mangle.
Stays fresh, stays fast
Generate once, serve for a week
Markdown is expensive to produce, since it re-fetches the live page, so each result is cached in a transient for a week. When an editor saves or deletes a post, the plugin hooks save_post and before_delete_post and clears the matching cache, including the relevant archive and the blog index, so stale content never lingers.
- save_post → purge
- before_delete_post → purge
- archive transient → purge
- blog index → purge
Try it
Request Markdown for any page
# Or via content negotiation, no .md suffix needed curl -H "Accept: text/markdown" https://yoursite.com/blog/how-agents-read/ # Response HTTP/1.1 200 OK Content-Type: text/markdown; charset=utf-8 Vary: Accept Cache-Control: public, max-age=86400, s-maxage=86400 X-Robots-Tag: noindex, nofollow
Where it runs
In production for our clients
We built it once and run it across accounts. It's live for these clients now, with more to follow. Pre and post agent-readiness scores live on each client's own case study.
The stack
- WordPress plugin (PSR-4)
- league/html-to-markdown
- DOMDocument + XPath
- WordPress transients
- template_redirect hook
Want your WordPress site readable by AI agents?
We build and run WordPress for funded technology and SaaS companies. This is one of the tools we bring to that work.