agentmarkup.dev served a soft-404. Our own tooling missed it.
agentmarkup is a tool for making websites machine-readable. Its own website was telling machines that pages existed when they did not. Every unknown path returned HTTP 200 with the app shell, so an agent probing /openapi.json or /about could reasonably conclude that both resources existed. At the time, neither did.
The bug was real, and the blind spot was ours
The cause was ordinary hosting behavior. Cloudflare Pages falls back to index.html when the deployed output has no 404.html. That fallback returned the application shell with a successful status for any invented path. To a person, it looked like a not-found page after the client app loaded. To a fetch-based agent, it looked like a page that existed.
I fixed it the same week. The build now emits a real prerendered 404.html, and the worker negotiates the response body. Agents that explicitly prefer markdown or plain text get a short machine-readable 404. Everyone else gets the HTML page. Both variants return HTTP 404 and include Accept in the Vary header.
The uncomfortable part is why the bug survived. Neither the hosted checker nor @agentmarkup/audit requests a path that should not exist. Both defend against soft-404 responses on the assets they already fetch, but neither performs a deliberate probe. That gap is worth naming rather than glossing over.
Update: this check now ships. Both @agentmarkup/audit 0.2.5 and the hosted checker request a path that should not exist and compare the response against the homepage. A 404 or 410 passes. A 200 matching the homepage is an error, because every path then appears to exist. A probe that never completes reports that the check could not be made, because a timeout and a soft-404 are not the same thing.
The honest output is a specific limitation, a reproducible case, and a check that still needs to be added.
In Claude's plugin catalogs, most of these tools stop at the report
On 2026-08-23, I fetched the two Claude plugin catalog manifests directly and matched every entry name and description against llms.txt, GEO, AEO, AI search, and AI crawler keywords, then read the matches.
- Anthropic's
claude-plugins-officialcatalog contained 286 plugins. Not one was anllms.txtor AI-discoverability plugin of any kind. - The
claude-communitycatalog contained 2,281 plugins. I found 23 close to this space. - Those 23 were near-uniformly auditors and scorers. The pattern across their descriptions is consistent: 0-100 health scores, letter grades, fixed-dimension check counts, and brand-perception scoring across assistants.
- Not one of the 23 installs a build-time dependency that emits the missing artifacts. None gates CI.
This is a dated catalog snapshot, not a claim about every tool on the internet. It does show a clear pattern in these catalogs: run an audit, summarize the findings, produce a score, and stop. Audits are useful, including ours. But after a report says your llms.txt is missing or your structured data is invalid, the site is still exactly as it was before the report ran.
What a build integration does instead
The agentmarkup plugin starts by inspecting which layer owns the final HTML. It then proposes one plan: the package to install, the files to change, the commands to run, and any live origin it would audit. Nothing is installed, written, built, or fetched until the user approves that plan.
Once approved, it installs the appropriate build-time integration, writes the configuration, runs the real site build, and validates the emitted output. Depending on the project, that can mean @agentmarkup/vite, @agentmarkup/astro, @agentmarkup/next, @agentmarkup/nuxt, the framework-agnostic @agentmarkup/cli, or direct helpers from @agentmarkup/core.
The resulting build can contain:
- a generated
llms.txtand optionalllms-full.txt - validated JSON-LD and an injected discovery link
- AI crawler rules and optional Content-Signal headers that preserve existing hand-written policy
- optional markdown mirrors for thin, noisy, or client-rendered HTML, not as a universal requirement
- an optional A2A Agent Card for discovery only, not an agent runtime or task endpoint
The CLI's check command can then run after the normal build and exit non-zero when required output is broken. That is the practical difference. The result is not another report to remember. It is a dependency, configuration, generated artifacts, and a CI condition that remain with the repository.
No score is part of the honesty
agentmarkup deliberately does not produce a readiness score, grade, or percentage. A missing required field is an error. A missing recommended field is a warning. That is the entire scale.
Collapsing unrelated checks into one number requires hidden choices about how much each item is worth. It also invites a much larger inference: that a higher number predicts inclusion, ranking, traffic, or citation by an AI system. We do not have evidence for that, so the tool does not make the claim.
What it can establish is narrower and useful: whether an artifact exists, whether it parses, whether required fields are present, whether crawler rules conflict, whether final HTML contains real content, and whether the build stays valid in CI. When a check does not exist, as with our soft-404 probe, the right response is to say so plainly.
Why I built it this way
Anima Felix needed this first. ChatGPT and Claude could find the site, but they could not really understand it. I fixed it there, then open-sourced the tooling we used.
That origin still sets the standard. The work is only useful when it changes what the site serves and keeps that output from drifting. A good audit points to the problem. A build integration makes the fix part of the site.
Use the plugin where you already work
agentmarkup was published in OpenAI's Plugin Directory on 2026-08-23. That directory is shared by ChatGPT and Codex, and the plugin can be installed from there.
The plugin has also been submitted to Anthropic's community directory and is pending review. Claude Code users can install it today from the repository's own marketplace:
/plugin marketplace add agentmarkup/agentmarkup
/plugin install agentmarkup@agentmarkupThe build-time packages are @agentmarkup/core, vite, astro, next, nuxt, and cli. The live crawler package is @agentmarkup/audit. Current versions are on npm.
Before a live audit, the plugin names the target origin and asks the user to confirm that they own or operate it. Approval covers the stated plan, not the rest of the session.
The plugin will not promise that an AI system will include, rank, cite, or send traffic to a site. It will make the site's machine-readable layer concrete, testable, and part of the build.