Guillaume Duvernay

Open source

astro-md-twins

Every page served twice: HTML for people, Markdown for machines.

An Astro template. It builds a static site, and for each page it also builds a Markdown file at the same path plus .md. An optional Cloudflare Worker serves the same Markdown from the page URL when a client asks for it by header.

Role
Author
Year
2026
Stack
Astro, Cloudflare Workers
Visit
Source code

HTML is expensive for a machine to read

A page carries navigation, footers, banners, scripts and class attributes. A crawler downloads all of it to get three paragraphs. Cloudflare measured one blog post at 16,180 tokens as HTML and 3,150 as Markdown, so about 80% less. Commerce pages have been measured at 95% less.

Crawlers do take the Markdown

These are from my own server logs on the AI Glot docs, 19 August to 14 September 2026, counting only Cloudflare-verified bots so no user agent is trusted. GoogleOther took 46.8% of its documents as Markdown, GPTBot 42.7%, ClaudeBot 34.4%. The numbers differ per site. The pattern does not.

What keeps it from being duplicate content

Every Markdown response carries a canonical link back to its HTML page, and no twin reaches the sitemap. A build check fails if a page has no twin, if a twin has the wrong canonical, if a twin is empty, or if one ended up in the sitemap after all.

You are reading a site that does this

Add .md to any article URL here and you get the source instead of the page.

/articles/harness-agnostic-workspaces.md

The same article, fetched two ways. The Markdown response carries a canonical link back to the HTML page, so the two never compete in search.

80%fewer tokens than the same page as HTML

46.8%of GoogleOther fetches taken as .md

42.7%of GPTBot fetches taken as .md