CI that keeps docs code samples working

The idea

Documentation code samples rot silently: the SDK adds a required parameter, an endpoint changes shape, and the quickstart keeps confidently displaying code that throws. The first person to run it is a prospective user in their first ten minutes with the product, which is precisely the moment that decides activation. Docs platforms render snippets beautifully; not one executes them.

Big API companies (Stripe famously) build bespoke internal systems for exactly this, which proves both the problem and the absence of a product. The wedge is packaging it as a zero-infrastructure GitHub Action: extract fenced blocks from MDX, run them in sandboxes with injected sandbox-account credentials, and annotate the failing line inside the PR diff so docs writers fix errors where they wrote them.

Who pays

The buyer is a devtools company DX or docs team, a budget line that already exists and already pays for docs tooling, at 29 to 99 dollars a month; the justification is activation, broken quickstarts measurably kill signups. Distribution: DX and technical-writing communities, free-for-public-repos adoption on OSS docs as the proof layer, and the GitHub Marketplace.

MVP scope

• GitHub Action extracting fenced blocks from markdown/MDX per config globs
• JS/TS and Python sandbox execution with timeouts, env-var injection and setup commands
• Ignore and continue annotations plus hidden inline assertions
• PR check with line-level annotations on the failing fenced block
• Free public-repo tier; paid private tier via Stripe
• Skip for v1: Go/Ruby/Java runners, browser-executed frontend snippets, auto-fixing samples, hosted docs rendering, OpenAPI-generated sample validation

Build SnippetCI, a GitHub Action plus dashboard that executes documentation code samples in CI. Stack: Node.js runner distributed as a GitHub Action, Next.js dashboard and marketing site, Stripe for billing.

Core flow: team adds the Action to their docs repo with a snippetci.yml config: glob patterns for MDX/markdown files, language runners to enable (JS/TS via Node, Python via a pinned container), env vars to inject (API keys for a sandbox account, injected from GitHub secrets so quickstarts run against a real test tenant), and per-language setup commands (npm install of the team's own SDK at the PR's version, which is the point: docs and SDK tested together).

On each PR the Action extracts fenced code blocks, skipping ones annotated with a snippetci-ignore comment and stitching multi-block tutorials via snippetci-continue annotations so step 2 runs in step 1's context. Each snippet executes in an isolated sandbox with a timeout; a non-zero exit, uncaught exception or failed inline assertion (support doc-hidden assertion comments like # expect: status 200) fails the check. Failures annotate the exact line of the exact fenced block in the PR diff via the Checks API, so a docs writer sees the error where they wrote it.

Dashboard (paid tier): snippet health across the whole docs site over time, flakiest snippets, last-passed-at per page, and a public badge.

Pages: landing with a broken-quickstart horror framing, docs, dashboard, billing.

Pricing: free for public repos, 29 GBP/month private, 99 GBP/month with dashboard, history and scheduled full-site runs.