API Design · Testing
Five Bugs
Nobody Was Looking For
An API annotation that compiles is not yet an API annotation that's true, and the cheapest way to tell the difference is to ask the running app. Fixing the incomplete Swagger docs in the NestJS version of my example project (the same backend design, implemented in five languages) took an afternoon. Verifying the fix by booting the app and curling every documented error path, instead of trusting that the annotations compiled, surfaced a bug that had nothing to do with documentation. Rolling the same discipline out to the other four languages surfaced four more, one of them serious enough that a real deployment's schema migrations would have stopped running without a sound. Two further bugs turned up along the way, incidental to that count but no less real.
Every @ApiOperation in the NestJS implementation had an operationId and nothing else: no summary, no description. Every endpoint documented its success response and nothing else, never the 400, 401, or 404 it threw. Every one of 32 DTOs had a bare @ApiProperty() with no options at all. The fix was mechanical: a shared ErrorResponseBody DTO, an explicit @Api<Status>Response per endpoint cross-checked against that handler's own error-mapping code, descriptions on every field.
Verifying by Booting the App, Not Reading the Diff
The fix was checked by starting the app against real Postgres and LocalStack and curling /docs-json and the live error paths, not by trusting that annotations which compile are annotations that are true. That check immediately found something documentation review alone never would. generateErrorResponse's error field used NestJS's HttpStatus enum reverse-lookup, which produces "NOT_FOUND", SCREAMING_SNAKE_CASE, inconsistent with the documented contract's "Not Found" and with the global validation pipe's own hardcoded "Bad Request". A pure code review would have read both pieces separately and never noticed they disagreed; only a response body sitting next to the doc made the mismatch visible.
"Every Language With a REST API Needs This"
Checking the other four found four different flavors of the same absence. Java-springboot and Kotlin-springboot didn't have springdoc-openapi as a dependency at all. They had zero OpenAPI capability, already self-documented in each language's own docs as "not yet introduced," a note that had apparently sat there long enough to stop meaning anything. Go had no mention of Swagger anywhere, not even as a plan. FastAPI was the interesting one: the framework auto-generates a bare OpenAPI skeleton, so /docs renders something and looks finished — but not one route had summary=, description=, or responses=, the identical gap NestJS had, just disguised by a framework default that happens to produce output.
Each language had an architecture checker (a script that statically checks the code against the documented rules), and none of the five checked documentation completeness at all; not even NestJS's now-fixed implementation had a regression guard. The checklist had no line item for API documentation, so even a careful manual pass would never have surfaced it on its own. Four separate reasons, one shape: a gap that was easy to leave undocumented, easy to leave unenforced, and in FastAPI's case, easy to mistake for already done.
Rolling It Out, With One Instruction That Mattered
Fixing the other four languages meant four parallel agents, each given NestJS's finished implementation as the reference and one explicit instruction: cross-check each endpoint's error-mapping code, and verify against a running app, not against what the code appears to do. That instruction is the reason a documentation task turned into five unrelated, pre-existing production defects. None of them were anything anyone was looking for, and all of them were only visible to something that sent a request and read the response.
The most serious one was in Java-springboot. Spring Boot 4 had split Flyway's autoconfiguration into its own separate starter module, and the dependency for it was missing. Database migrations were never running against a real database, and nothing said so. Nothing in the test suite had ever caught it, because the tests used ddl-auto: create-drop, which builds the schema from the entity mappings directly and has no use for Flyway at all. A production deployment would have booted clean, served traffic, and simply never applied a single migration — invisible until the schema drifted far enough from what the entities expected to fail loudly, at the worst possible time to discover why.
The rest, smaller but all real, complete the count. Kotlin's Spring Security returned its own generic 403 for an unauthenticated request before the app's exception handler ever got a chance to produce the documented 401 shape. Go's auth middleware sent a 401 as plain text, not the JSON schema its own docs promised, and several validation 400s had the same problem. FastAPI had no exception handler at all for an invalid JWT, so a bad token produced an unhandled 500 instead of a clean 401. That's the five the title counts. Two more turned up along the way, outside that count but no less real. Java's own /v3/api-docs and Swagger UI required a bearer token to view, so the API documentation was, itself, not publicly reachable. And NestJS's own scaffolding generator had a bug in the very code the Swagger fix was touching: a generated cancel handler's "already cancelled" domain error was never mapped in the controller's catch block, producing a 500 where a 400 was intended.
None of these bugs were about documentation. Every one of them was already sitting in production-shaped code, waiting for a request shaped the way its author never happened to send one. What found all seven wasn't a smarter reviewer. It was a rule applied uniformly: don't just make the annotation compile, prove the thing it describes is true by asking the running app.
The Follow-Up Nobody Planned For
Three of the four rollout agents shipped a fix that passed everywhere except the check that covers the code generator. Each language's create-domain generator (a script that generates a skeleton domain from a name) still emitted endpoints without the new required annotations, because the generator template was never told the bar had moved. Kotlin's agent, watching this happen to the other three first, fixed its own generator proactively and passed on the first try. The lesson generalized cleanly: any checker rule that checks per-endpoint or per-file content will very likely need a matching generator update. That's not an edge case to discover later; it's a second step to budget for up front.
docs/architecture/api-response.md (the documentation completeness bar, now a root-level requirement in backend-service-playbook, my example project that implements the same backend design in five languages side by side) · api-documentation.evaluator.ts (the NestJS checker rule that now enforces it)