PII redaction proxy for LLM API calls

The idea

Every healthtech, legaltech, and fintech startup has the same stalled Jira epic: an LLM feature the compliance officer keeps vetoing because raw client text — names, medical record numbers, case details — would leave the building and land on a model provider’s servers. Engineering teams don’t want to build entity detection, deterministic tokenization, and streaming re-hydration themselves; they want the veto lifted this sprint.

LLM Gateway proved developers happily route model traffic through middleware. But gateways compete on routing and cost, which is a nice-to-have budget. Redaction competes on the compliance checkbox — the thing standing between a team and shipping. The product is a one-line base-URL swap: PII becomes stable tokens on the way out, gets restored on the way back, and an audit log shows exactly what left the building. That log is the artifact the compliance officer actually wanted all along.

Who pays

The buyer is an engineering lead at a seed-to-Series-B healthtech, legaltech, or fintech company, usually with a compliance officer or counsel forcing the purchase — $99 to $499 a month depending on volume and features like BAAs and custom entity packs. It’s bought to unblock a roadmap item, so sales cycles are short and churn is low: ripping it out means re-arguing with compliance. Distribution: content targeting the exact search (HIPAA-compliant LLM API), HN launches, and integrations listed in LLM framework docs.

MVP scope

• Reverse proxy compatible with OpenAI and Anthropic APIs, one base-URL swap to adopt
• NER plus regex detection for the core PII set; deterministic per-request tokenization and response re-hydration, streaming included
• Policy editor: entity toggles, custom regex, redact-or-block mode
• Audit log (metadata only) with CSV export
• Stripe subscriptions with request metering
• Skip for v1: self-hosted deployment, fine-tuned custom NER, image/document redaction, SOC 2 report (start the process, don’t gate launch on it)

Build Redactor, a PII redaction proxy for LLM API calls. Stack: a Go reverse proxy (or Node if faster to ship), local NER models (spaCy or GLiNER served via a sidecar) for entity detection, Postgres for config and audit logs, Stripe for billing.

Core flow: 1) Customer swaps their OpenAI/Anthropic base URL for their Redactor endpoint and adds a Redactor API key. 2) The proxy parses the outbound request body, runs NER plus regex packs (SSN, MRN, email, phone, DOB, addresses) over every message, and replaces each entity with a deterministic token like PERSON_1 or MRN_2 — the same input string always maps to the same token within a request, so the LLM keeps referential coherence. 3) Request forwards to the real provider with the customer's own upstream key. 4) The response streams back through the proxy, which re-hydrates tokens to original values before the client sees them. 5) An audit log records timestamp, model, entity types redacted, and counts — never the raw values.

Pages: dashboard (request volume, entities caught), policy editor (toggle entity types, add custom regex, choose redact vs block), audit log with export, API keys, billing. Ship SDKs as one-line base-URL docs, not client libraries.

Non-obvious details: handle streaming SSE responses by buffering only until a token boundary could span chunks; store the token map in request-scoped memory, never in Postgres.

Pricing: $99/mo (100k requests), $249/mo (1M, custom regex), $499/mo (SSO, BAA, self-host license).