API Design · Testing

Five Bugs
Nobody Was Looking For

Fixing NestJS's incomplete Swagger docs took an afternoon. Verifying the fix by actually booting the app and curling every documented error path — instead of trusting that the annotations compiled — surfaced a bug that had nothing to do with documentation. Rolling the same discipline out to the other four languages surfaced four more, one of them serious enough that a real deployment's schema migrations would have silently stopped running. Two further bugs turned up along the way, incidental to that count but no less real.

Every @ApiOperation in the NestJS implementation had an operationId and nothing else — no summary, no description. Every endpoint documented its success response and nothing else, never the 400, 401, or 404 it actually threw. Every one of 32 DTOs had a bare @ApiProperty() with no options at all. The fix was mechanical: a shared ErrorResponseBody DTO, an explicit @Api<Status>Response per endpoint cross-checked against that handler's own error-mapping code, descriptions on every field.

Verifying by Booting the App, Not Reading the Diff

The fix was checked by actually starting the app against real Postgres and LocalStack and curling /docs-json and the live error paths — not by trusting that annotations which compile are annotations that are true. That check immediately found something documentation review alone never would: generateErrorResponse's error field used NestJS's HttpStatus enum reverse-lookup, which produces "NOT_FOUND", SCREAMING_SNAKE_CASE — inconsistent with the documented contract's "Not Found" and with the global validation pipe's own hardcoded "Bad Request". A pure code review would have read both pieces separately and never noticed they disagreed; only an actual response body sitting next to the actual doc made the mismatch visible.

"Every Language With a REST API Needs This"

Checking the other four found four different flavors of the same absence. Java-springboot and Kotlin-springboot didn't have springdoc-openapi as a dependency at all — zero OpenAPI capability, already self-documented in each language's own docs as "not yet introduced," a note that had apparently sat there long enough to stop meaning anything. Go had no mention of Swagger anywhere, not even as a plan. FastAPI was the interesting one: the framework auto-generates a bare OpenAPI skeleton, so /docs renders something and looks finished — but not one route had summary=, description=, or responses=, the identical gap NestJS had, just disguised by a framework default that happens to produce output.

No harness rule in any of the five languages checked completeness at all — not even NestJS's now-fixed implementation had a regression guard. The checklist had no line item for API documentation, so even a careful manual pass would never have surfaced it on its own. Four separate reasons, one shape: a gap that was easy to leave undocumented, easy to leave unenforced, and in FastAPI's case, easy to mistake for already done.

Rolling It Out, With One Instruction That Mattered

Fixing the other four languages meant four parallel agents, each given NestJS's finished implementation as the reference and one explicit instruction: cross-check each endpoint's real error-mapping code, and verify against a running app, not against what the code appears to do. That instruction is the reason a documentation task turned into five unrelated, pre-existing production defects — none of them anyone was looking for, all of them only visible to something that actually sent a request and read the response.

The most serious one was in Java-springboot. Spring Boot 4 had split Flyway's autoconfiguration into its own separate starter module — and the dependency for it was missing. Database migrations were silently never running against a real database. Nothing in the test suite had ever caught it, because the tests used ddl-auto: create-drop, which builds the schema from the entity mappings directly and has no use for Flyway at all. A production deployment would have booted clean, served traffic, and simply never applied a single migration — invisible until the schema drifted far enough from what the entities expected to fail loudly, at the worst possible time to discover why.

The rest, smaller but all real, complete the count: Kotlin's Spring Security returned its own generic 403 for an unauthenticated request before the app's exception handler ever got a chance to produce the documented 401 shape. Go's auth middleware sent a 401 as plain text, not the JSON schema its own docs promised, and several validation 400s had the same problem. FastAPI had no exception handler at all for an invalid JWT — a bad token produced an unhandled 500 instead of a clean 401. That's the five the title counts. Two more turned up along the way, outside that count but no less real: Java's own /v3/api-docs and Swagger UI required a bearer token to view — the API documentation was, itself, not publicly reachable. And NestJS's own scaffolding generator had a bug in the very code the Swagger fix was touching: a generated cancel handler's "already cancelled" domain error was never mapped in the controller's catch block, quietly producing a 500 where a 400 was intended.

The pattern underneath all seven

None of these bugs were about documentation. Every one of them was already sitting in production-shaped code, waiting for a request shaped exactly the way its author never happened to send one. What found all seven wasn't a smarter reviewer — it was a rule applied uniformly: don't just make the annotation compile, prove the thing it describes is actually true by asking the running app.

The Follow-Up Nobody Planned For

Three of the four rollout agents shipped a fix that passed everywhere except the scaffolding-regression check — each language's create-domain generator still emitted endpoints without the new required annotations, because the generator template was never told the bar had moved. Kotlin's agent, watching this happen to the other three first, fixed its own generator proactively and passed on the first try. The lesson generalized cleanly: any harness rule that checks per-endpoint or per-file content will very likely need a matching scaffolding-generator update — that's not an edge case to discover later, it's a second step to budget for up front.

Further reading in the repo

docs/architecture/api-response.md — the completeness bar, now a root-level requirement · api-documentation.evaluator.ts — the harness rule that now enforces it