SYN-342: Independent documentation publication review
The renderer preparation and verification source was frozen at
140b2b52640d29bbc774709adeba728cab5864e2. The actual 14-method suite passed using
the pinned official mdBook and mdbook-mermaid binaries. Their executed bytes
matched the verified Darwin archive contents. No browser or JavaScript execution
is claimed.
Original mutation campaign
Eight plausible production variants ran against the unchanged original tests. Five failed by relevant assertions; three survived. No attempt had runner errors, skips, missing results or other invalid infrastructure outcomes.
| Production variant | Original 14 methods |
|---|---|
| Omit the newly added diagram ADR from the generated chapter set | Survived |
| Omit canonical Quickstart from the generated chapter set | Survived |
| Run CLI diagram verification only for the release-pipeline chapter | Survived |
| Accept a missing diagram container | Assertion kill |
| Accept raw Mermaid code fences | Assertion kill |
| Accept initializer references before the runtime | Assertion kill |
| Identify the Mermaid runtime by filename instead of pinned bytes | Assertion kill |
| Normalize canonical source bytes during preparation | Assertion kill |
The chapter omissions left the producer’s own manifest self-consistent. Existing tests derived coverage from that manifest, so they did not establish the complete canonical source inventory. Direct calls to the diagram helper checked individual pages but did not prove that the production CLI invoked it for non-release pages. These are demonstrated gaps in the original tests, not additional defects in the unchanged pristine renderer source.
Independent strengthening
Three additional tests in scripts/check-docsite-independent-tests.py
compare the manifest with the complete Git-tracked product chapter policy, check
each independently expected generated page, and invoke the actual publication
CLI after removing a diagram or initializer from a non-release chapter. A generic
runner error cannot satisfy those CLI controls: the tests also require the
intended verifier assertion reason.
The pristine combined suite passed all 17 methods. The three original survivor patches then ran byte for byte against the frozen strengthened tests. All three were killed by assertions, with no survivors or invalid attempts. The release-only CLI variant failed both non-release controls. Restored production passed all 17 methods in 4.444 seconds, with no failures, errors, skips or expected failures. The original eight-variant verdicts remain unchanged and separate.
The allowlisted receipt records exact production/test identities, patch and private log digests, original verdicts, strengthened replays and restored results. Raw logs, downloaded tools and generated books remain private. No production fix was needed for this test-coverage correction.
Required invocation and scope
Run both actual suites with the reviewed official tools on PATH:
python3 scripts/check-docsite.py -v
python3 scripts/check-docsite-independent-tests.py -v
Both invocations must be required by Pages validation. The original workflow only invoked the original 14 methods. Wiring the independent command into CI is a subsequent implementation change and is not included in the original frozen mutation review. Validate the final integrated source separately.
These tests exercise real mdBook preparation, the actual Mermaid preprocessor, generated HTML and the production publication CLI. They establish chapter inventory, structural diagram coverage and local script-byte identity. They do not establish SVG rendering, JavaScript execution order in a browser, console behavior, themes, layout, search interactions or the deployed Pages artifact. Ubuntu tool execution, final hosted CI, browser acceptance and publication remain separate gates. No Cargo, native GUI, Swift, Xcode, push, tag or release action was performed in this review.