Everything Screenshots Can't Show Is Still My Job: Security Audit of an AI-Built Site

Most of what gets built with AI right now is judged on a screenshot: a dashboard that looks like a product, a SaaS tool with a landing page and a pricing table, an internal app a team starts using the same afternoon. I build this way too, and the speed is real.

The trouble is that a screenshot can’t show the half of the work that decides whether what you built is safe to put in front of people. That half covers who can write to the database, what happens when a visitor sends something hostile, whether the cache ever hands one person another person’s page, and whether someone using a screen reader can get past the hero. None of that has a visual, so it rarely makes it into the prompt, and more often than not it just gets skipped.

I designed and built this site with AI assistance, and I’ve been open about that. Anything I ship for someone else gets audited before it launches, because it only takes one bad bug to turn into a data leak. This site doesn’t hold sensitive data, so I let its audit wait longer than I would on anyone else’s project. This week I ran it with the same checks I run before any launch, and I’m publishing what it found, including the one serious miss, because the way findings get handled is the part of this work I most want to show.

What the Audit Covered

The audit had three passes. First, I read every file the server runs and every script the site ships, in full. Second, I sent read-only requests to the live site to check headers, redirects, odd methods, path tricks and the editor’s preview endpoint. Third, I checked the platform settings around the code, which are the parts a repository can’t show you: the CMS dataset’s access mode and allowed origins, the deploy workflow’s permissions, and the DNS zone’s TLS settings.

It produced thirteen findings: one High, four Medium, and the rest hardening. The code fixes went out in two pull requests, the second of which enforced the script policy, along with a test suite that now runs on every deploy. One more was a switch in the Cloudflare dashboard.

The number needs some context. The site already had a Content Security Policy, signed editor sessions, parameterized queries and no secrets in the repository, so it was in better shape than most of what ships. It still had a hole that would have let a Markdown file run code in a visitor’s browser. Exploiting it would have required write access to one of my own repositories, including any AI agent pushing to them. There’s no sign anyone ever tried. For clarity all findings described here were fixed before publication.

Two layers divided by a horizontal line. Above it, one box labelled the page, described as layout, type, color, motion, the demo that gets judged. Below it, six boxes: data access, render-time checks, cache rules, the policy, the pipeline, and accessibility, each with a line on what it covers. — Figure 1 The screenshot covers the top layer. The audit covers everything underneath it.

The Hole

The /code section of this site renders each repository’s README. The Markdown library I use, marked, lets you override how raw HTML is rendered, and I’d done that so any tag in a README comes out as text. I tested it with <script>, with <img onerror>, and with the rest of the usual list. Everything came out escaped, so I considered it handled.

What I hadn’t read was the lexer. When marked meets an inline <pre>, <code>,or <kbd> tag that never closes, it marks the text that follows as already escaped and sends it to a different renderer, the one for plain text, which prints escaped text as it stands. My override never saw that text because it wasn’t an HTML token anymore. x <pre> <svg onload=alert(1)> came out of the renderer with the svg element intact.

The library’s documentation tells you to sanitize the output. I had only sanitized the input path I could see and assumed it covered the rest.

The fix is small. It clears that flag on the lexer’s text tokens before rendering, so the plain-text renderer escapes them like anything else. Behind it sit two more layers, so the fix doesn’t have to be perfect on its own: a tripwire that returns the whole README as plain text if a script-bearing element ever appears in the output, and seven test cases built from hostile READMEs that the deploy runs before it builds. The tests also check the other direction, because marked’s own parser sets the same flag on HTML it generated itself (task-list checkboxes, tight lists), and a blanket fix would have printed that markup as text.

Left, a chain of five boxes: a README containing an unclosed pre tag, the lexer flagging what follows as escaped, the HTML renderer override that never sees it, the text renderer printing it as it stands, and the page with a live svg element. Right, three numbered plates: clear the flag before rendering, a tripwire that returns the README as plain text, and seven test cases built from hostile READMEs that run on deploy. — Figure 2 Where the text went around the override, and where the fix sits.

If I had to pick one lesson from the whole audit, it’s this one: a sanitizer you didn’t write has code paths you haven’t read. So test what comes out of it, and test it with hostile input. I had tested my configuration with friendly assumptions, and that’s how this one got through.

The Other Findings

Caching and authorization have to be checked together. The social cards for shared links are rendered on demand and cached for an hour. Two cache rules were joined with else if, and a card drawn in the editor’s draft mode could fail the first test and fall into the second, which stored it publicly. A draft title could have been the public card for an hour. The preview check now comes first. If I’d written the two conditions out as a table before writing the if, I’d have seen the empty cell.

Read what your cache does with Vary. Every response says Vary: Cookie so the editor’s preview misses the cached public page. Cloudflare’s cache keeps one stored copy per distinct cookie header, so a request carrying any cookie the site never set, whether a stray analytics cookie or one chosen on purpose, was rendered fresh and stored as its own copy. Those requests are now rendered but never stored. The behavior is documented, and I’d missed that page.

Check a link where it gets rendered. The CMS validates link fields when I type in them, but anything with a write token can store any value, and my renderers put that value straight into href. A shared rule now runs where the link lands and allows http, https, mailto, a path on this site, or a fragment. Everything else renders as text. The same rule now checks a repository name from the CMS before it’s placed in an API path that carries a token.

Redirect parameters need a same-origin check, even behind a secret. The editor’s preview handshake redirects to the page the editor asked for. One URL shape survived the library’s normalization as a protocol-relative address. It now accepts a path on this site and falls back to the home page for anything else.

HSTS only helps once plain HTTP redirects. The site sent a strict-transport header, which tells a browser to use HTTPS from then on. But the zone setting that redirects plain HTTP was off, so the first visit over http was served in the clear. Now the zone redirects, and the Worker redirects behind it in case the setting ever changes.

The deploy pipeline got the same treatment as the site. The repository already limited workflows to read-only access, and the workflow file now declares that itself, so it no longer depends on a setting someone could change. It also pins both actions to exact commits, deploys only from the main branch, and runs the tests before every build. And it stops the deploy if the staging site is ever reachable without its login, where before it only logged a warning.

Allowing inline scripts made the README bug much worse. My policy allowed inline scripts because the site’s own motion bootstrap and its build-inlined component scripts depended on them, and that’s what turned the README hole from a markup bug into script execution. The site only needs one inline script, the one that sets the motion preference before first paint. It’s now named by its hash, the component scripts ship as files, and the policy carries base-uri and form-action. The hashed policy ran in report-only mode for one deploy beside the enforced one, so a violation would have shown up in the console without breaking a page. None showed up on any page or in the editor’s preview frame, and the next deploy made it the enforced policy. Watching that console also turned up an unrelated bug in the editor’s draft preview of the /code pages, which I fixed the same morning. It never affected visitors, so it isn’t one of the thirteen.

A two-by-two table. Columns are public visitor and editor in draft mode; rows are page or note card and code card. The public cells read cache one hour, the draft cells read never store, and the draft page-card cell is marked as the one the else-if stored. — Figure 3 The cache rule as a table, with the one cell the else if got wrong.

Accessibility Is the Same Discipline

I put accessibility in the same audit on purpose, because it fails for the same reason security does: neither one shows up in a screenshot. Nobody prompts for a skip link, because nobody sees it until they need it. The site has one, with a visible focus ring, a reduced-motion path that works whether or not JavaScript runs, native dialogs that return focus, 44 pixel targets, code blocks a keyboard can reach, and 33 documented contrast pairs. Lighthouse puts accessibility at 100 on every route I measured. I got that score by planning for it, the same way the security findings came from the few places where I hadn’t planned.

Adoption belongs here too. That sounds like a stretch until you’ve shipped an internal tool nobody opened twice. The questions that decide whether people adopt a new tool (who it’s for, what they do first, what happens when it’s wrong) are the same ones a threat model asks from the other side.

The Checklist I’d Hand Anyone Building This Way

None of this is overkill, and most of it fits in an afternoon.

Write the data story in a paragraph before the first screen. Where does each kind of data live, who can read it, who can write it, and what’s the blast radius if the write token leaks? If the answer for any table is “anyone with the URL”, stop there.
Treat every CMS field, API response and file as input. Validate at the render boundary, since that’s the one you control. A form validator only helps the person typing.
Put caching and authorization in one table. Rows are who’s asking, columns are what they’re asking for, and every cell gets filled. Empty cells tend to hide in else if chains.
Ship a Content Security Policy without 'unsafe-inline' for scripts. Hash the one or two inline scripts you actually need. That way the next markup bug can’t run script.
Test the output with hostile input. A dozen fixtures in a test file that runs on every deploy is enough, as long as they check what the renderer actually produces.
Lock the pipeline. Use minimal token permissions and pinned actions, deploy only from the main branch, and make the staging check fail instead of warn.
Read the cache documentation for the headers you send. Vary in particular.
Run an accessibility pass with the same seriousness. Go keyboard only, use a screen reader once, turn reduced motion on, measure contrast, and write the numbers down.
Publish a security.txt and keep an audit log you’d be willing to summarize in public. This note is the public summary of mine.

I wasn’t building this site for a client or an employer, and I let my guard down in a way I wouldn’t on anyone else’s project. Nothing grave would have come of these findings. They still make a good lesson in what slips through when you stop approaching the work with caution, especially when AI agents are writing and shipping a lot of the code. The circumstances kept the stakes low here, and that won’t always be the case.

If you take nothing else away from this note, let it at least be this. Put yourself in a business owner’s position and ask how you’d feel if someone shipped vulnerable code that cost your business:

its reputation
the safety of its clients and their data
the security of its internal infrastructure
legal exposure or litigation down the line

Every one of those costs can start with a gap that looks fine in a screenshot, but gaps like that always have consequences, and the worst ones usually land on someone else. That’s why everything a screenshot can’t show is not only my job but my responsibility, and the responsibility of anyone in this line of work, to audit and fix.