release: Closes GT-666, Closes GT-667, Closes GT-668 — the corpus mapping stated two false things about its own rules, and the guard proving one of them had expired - #483
Merged
Conversation
…ules Evolith governance invariants (#482) Closes GT-666. The mapping an auditor reads was making a false statement about the corpus, in the one document whose purpose is to be checkable by somebody who does not trust us. THE NOTE, VERBATIM, read out of the committed artifact before touching anything. All 16 rules of the three international-standard packs carried `ruleClass: governance` and this: "A governance invariant over Evolith artifacts (inheritance, open-core boundary, satellites, evidence). No international structural equivalent." Every clause is false of a NIST SP 800-218 practice. And `--check` passed while guard 46 reported the chain current AND at a fixed point: nothing was stale, the artifact simply asserted the opposite of the truth. That is this session's recurring shape — an instrument answering confidently to a question nobody asked it. CLASSIFIED BY THE PACK'S OWN DECLARATION, not by directory. A walk of all 180 `*.rules.json` found a top-level `standard` block on exactly 3. It is a statement the document makes about itself, so it travels with the file and a fourth pack classifies correctly wherever it lands — which a `standards/` prefix row could not have achieved. The directory then guards the OTHER direction: a `*.rules.json` under `standards/` whose rules carry no `standard` block now FAILS the generator. A declared pack anywhere is classified; an undeclared pack in that directory is a build failure. Both shipped declaration shapes are read ({id} and {name, edition}) — a reader of one would have dropped two of three packs and still passed on the third. GUARD, seen failing against reality: `65-validate-standards-rule-class.mjs` was run against the REAL pre-fix artifact read out of git and reported 64 findings (16 rules x 4 broken assertions), not against an invented fixture. Red on guard 43's empty tree, INSTRUMENTED per guard 42, wired into Governance guards with a 19-case self-test — which found a defect in the guard itself: its denominator counted only DECLARING packs, so a tree whose packs had lost their declarations died with "the location moved", the one diagnosis certain to be wrong. MEASURED AFTER: 16 rows `international-standard`; `governance` 71 -> 55; corpus unmoved at 412 rules / 180 files; mappedToIso5055 and adoptedFraction unchanged; no pinned count in rule-corpus-triage.spec.ts moved. NOT CLAIMED: `analyser.adoptable: "no"` on the four ISO5055-* rows is ALSO false — the pack is decided by an off-the-shelf analyser's SARIF, not by a handler somebody must author. It is derivable and narrow but moves the handler-backlog arithmetic GT-598 produces, so it is left to its own gap rather than smuggled in here. The standards README figures were stale BEFORE this work (391/175 against a real 412/180, since GT-662..GT-665); refreshed and stated as pre-existing. Board: 652 / 664.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
…were published as rules somebody must write, and counted into the backlog (#487) * feat(standards): Closes GT-667 — four rules an analyser already decides were published as rules somebody must write, and counted into the backlog Closes GT-667. `analyser.adoptable` has one stated meaning in the generator's own header: `no` means "the predicate is repository- or product-specific and must be authored". The four ISO5055-* rows carried `no` while GT-662..GT-664 had already shipped that pack as an ADAPTER over a free analyser's SARIF. Nothing about it was ever going to be authored. `examples` was `[]` on all four, so the row asserted a verdict and named nothing a reader could check. READ OUT OF THE COMMITTED ARTIFACT before anything was touched. Not new information either: GT-666 found it while measuring, wrote it into its own closure record as NOT claimed, and left it because it moves the arithmetic GT-598 exists to produce. This is that arithmetic. DERIVED, following GT-666's pattern for ruleClass: the rules declare the analyser themselves in `enforce.config.analyser`. THE SIGNAL WAS CHOSEN AGAINST THE CORPUS, not by inspecting the four rows it had to move. A walk of all 180 *.rules.json found `enforce` on 10 of 412 rules and `enforce.config.analyser` on exactly 4 — these four. The other 6 are what rules OUT `enforce.tool`: they name dependency-cruiser and then supply the predicate themselves. HXA-07 declares the tool and writes the rule, from `^src/(domain|core)/.+\.(spec|test)\.ts$` to `node_modules/(@nestjs/testing|testcontainers)/` — the tool is the ENGINE and the rule is Evolith's, written against this repository's own layout, which is "repository-specific" verbatim and correctly `no`. Deriving from `enforce.tool` would have flipped HXA-06 and HXA-07 on the strength of a third-party binary appearing in the clause. `yes` AND NOT `partial`, which was the one arguable call. The ESLint path reaches 11 of 138 weaknesses, but that is COVERAGE, and `partial` means the rest is OURS to author. Here the rest is closed by a tenant pointing the same adapter at a better analyser. Handler work is zero; coverage stays reported in the pack's notEvaluableHere block and the GT-569 advisory. `examples` NAMES THE DECLARED ANALYSER. An adoptable:yes row naming nothing is the unfalsifiable claim this artifact exists to remove. Read from the rule, so a corpus configured for semgrep regenerates saying semgrep; an enumerated list would be stale the moment the config changed. CONFLICT FAILS THE BUILD, AND THE FAILURE WAS OBSERVED, not asserted: injecting a contradicting MAP row for ISO5055-SEC into a copy of the generator exits non-zero naming both values and both fixes. Nothing triggers it today, which is why it was exercised against an injected conflict rather than left an untested branch. MEASURED AFTER: analyserAdoptable 42 -> 46, fraction 0.1019 -> 0.1117 (0.1578 -> 0.1675 incl. partial), byClass.international-standard.adoptable 0 -> 4; adoptableFromAnalyser 5 -> 9, incl. partial 10 -> 14, adoptedFractionOfBacklog 0.0962 -> 0.1731 (0.1923 -> 0.2692), remainderToAuthor 42 -> 38. Unmoved: realBacklogSize 52, corpus 412/180, mappedToIso5055 37, adoptedFraction 0.0898. A row-by-row diff against the pre-change artifact: exactly 4 rows changed, 0 rows with any non-`analyser` field changed. No pinned count in rule-corpus-triage.spec.ts moved — 156 suites / 1851 tests green from a clean dist (1 suite skipped, pre-existing; this branch touches no TypeScript). THE SHARE MOVED BECAUSE THE DESCRIPTION WAS CORRECTED, NOT BECAUSE ANYTHING WAS ADOPTED. Nothing shipped between 9.6% and 17.3%. Both READMEs say so where the jump appears. The honest reading is worse: the backlog had been overstated by four rules since GT-662 landed earlier the same day. SSDF AND SLSA CORRECTLY STAY AT `no`, argued rather than left silent — they are the same rule class as the four that moved. They carry no `enforce` block: SsdfRuleHandler/SlsaRuleHandler decide them natively, and no off-the-shelf analyser decides "does this repository satisfy SSDF PW.4.1". Those predicates read this repository's own workflows, manifests and evidence, so the work genuinely had to be authored, and it was (GT-659, GT-665). One pack declares an analyser, the other two ship a handler. NO NEW GUARD, deliberately: the generator itself throws on conflict and --check sits in the derived-artifact chain, so a second guard would add a denominator to maintain and no coverage. Board: 653 / 665. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(governance): GT-667 — closure record for the analyser-adoptable correction The closure evidence for GT-667, pointing at c6dcb27 as its closureCommit: the commit that carries the derivation, the regenerated artifacts and the restated handler-backlog arithmetic. Records what was measured rather than what was intended — the before/after of every figure the correction moves, the row-by-row diff confirming a blast radius of exactly 4 rows with no non-`analyser` field touched, the observed failure of the MAP-versus-declaration conflict check, and the two decisions taken with their reasons: `analyser.examples` names the declared analyser, and the SSDF and SLSA rules correctly stay at `no`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
beyondnetPeru
changed the title
…ed to a branch, so it expired the day the fix landed (#486) * Closes GT-668 — the proof that a guard had been seen failing was pinned to a branch, so it expired the day the fix landed GT-666's self-test kept one case that mattered more than the other eighteen: the guard checked against the REAL pre-fix artifact, not a sketch of one. Its comment says why, and it is right — «Reconstructed fixtures agree with whatever the author believed was wrong; this one cannot.» It got that artifact with `git show ${PRE_FIX_REF}:iso-5055-mapping.json`, and PRE_FIX_REF defaulted to origin/develop. True while the fix lived on a branch. 59d62ba merged, origin/develop began serving the CORRECTED artifact, and a case written to count 64 findings started counting 0 — green on its own branch, red the moment it mattered, blocking promotion PR #483. Reproduced on develop before anything was touched: 19 tests, 18 pass, `AssertionError: 0 !== 64`. The worse half is the one nobody would have seen. The case carried a shallow-clone escape — `if (before.status !== 0) return void assert.ok(true, 'SKIPPED: …')`. Under fetch-depth:1, a detached history, or a fork without origin/develop, the identical rot PASSES IN SILENCE. That is an escape hatch out of the one case establishing this guard has ever been OBSERVED failing rather than declared able to. So: keep the intent, freeze the evidence. The pre-fix mapping is committed at .harness/fixtures/standards-rule-class/iso-5055-mapping.pre-gt-666.json, taken from 0130834 (59d62ba^, blob 6684e8a) — the fixtures home this repo already uses, not a new one. Not trimmed: all 412 rows, because trimming is an editorial act on a fixture whose whole value is that no author's belief touched it. One `_fixture` provenance key added (JSON has no comments) and the claim is checkable, not asserted — drop it, re-serialise, and the blob reproduces byte for byte. The git show, the env var and the skip are deleted. No replacement skip: a shallow clone constrains a test that shells out to git, not one that reads a file in its own tree, so an unreadable fixture is a real failure and red is correct. Assertions unchanged and unweakened — 64 findings, 16 of them `is classified governance`. One case ADDED, not relaxed: a fixture "refreshed" from the live mapping is RED, and that was observed rather than argued. Simulating the refresh turns 2 of 20 cases red, so the negative fixture cannot quietly become a copy of the artifact it exists to catch. Measured: node --test 20/20 pass, 0 skipped. 42 -> 78 guards classified, 57/57 refuse a zero-element scan. 43 -> 54/54 exercised guards turned red. core-domain clean rebuild, 157 suites / 1853 tests green. 01, 04, 09, 46 pass; ci-runner governance fails only on this commit's own PENDING closureCommit. No corpus file, no generator and no derived artifact touched — deliberately off build-iso-5055-mapping.mjs and iso-5055-mapping.json, where a parallel session is changing analyser.adoptable. That work moves neither ruleClass nor the pack rule count, so the frozen 64/16 figures are unaffected by it. Registered GT-668, not GT-667: GT-667 is claimed by branch name on two concurrent branches (gt-667-analyser-adoptable, claude/gt-667-iso5055-adoptable) carrying that follow-on. Guard 49 compares against origin/main and could not have seen either, so the id was taken by union of branches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(evidence): point GT-668's closure at the commit that carries it a620ed3 is the commit with the frozen fixture and the reworked self-test. 08-validate-tracking now passes: 665 gaps, 641/641 EN/ES catalog sections, 635 closure records. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(gaps): GT-666's catalog entry described a mechanism GT-668 replaced That paragraph said, in the present tense, that the self-test "reads the pre-fix iso-5055-mapping.json out of git" — which stopped being true one commit later. Left alone it is the same class of thing GT-668 is about: a document whose claim quietly diverged from the artifact without anyone editing it. The counts it reports (64 findings, 16 rules x 4 broken rules) are unchanged and still measured, so only the mechanism is marked superseded. EN and ES both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
beyondnetPeru
changed the title
beyondnetPeru
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release
develop→main. Two gaps, both against the same artifact: the corpus mapping an auditor reads.Closes GT-666 — 16 conformance controls of three published standards were labelled Evolith governance invariants
build-iso-5055-mapping.mjsclassified by path prefix through aCLASS_BY_FILEtable withgovernanceas the fallback, and the table had nostandards/row. All 16 rules of the three packs — NIST SP 800-218 (8), ISO/IEC 5055:2021 (4), SLSA v1.0 Build track (4) — were published asruleClass: governancecarrying, verbatim: "A governance invariant over Evolith artifacts… No international structural equivalent." Every clause of that is false of an SSDF practice.Classification now derives from the pack's own top-level
standardblock, so a fourth pack classifies correctly wherever it lands; the directory is enforced in the other direction — an undeclared pack understandards/fails the generator instead of defaulting.65-validate-standards-rule-class.mjs(+19-case self-test) holds both directions and was observed failing against the real pre-fix artifact read out of git: 64 findings.governance71 → 55, 16 rowsinternational-standard, corpus unmoved at 412 rules / 180 files.Closes GT-667 — four rules an off-the-shelf analyser already decides were published as rules somebody must write
analyser.adoptable: nomeans, in the generator's own header, "the predicate is repository- or product-specific and must be authored". The fourISO5055-*rows carried it while GT-662…GT-664 had already shipped that pack as an adapter over a free analyser's SARIF. BecausehandlerBacklogcounts adoptability against theunimplemented-nativeclass — exactly where those four sit — the artifact that exists to size the backlog was overstating it by four rules.Derived, following GT-666's pattern: the rules declare the analyser themselves in
enforce.config.analyser, carried by exactly 4 of the 412 corpus rules.enforce.toolwas rejected as the signal because 6 other rules namedependency-cruiserand then supply the predicate themselves — there the tool is the engine and the rule is Evolith's.analyserAdoptable42 → 46;adoptableFromAnalyser5 → 9;remainderToAuthor42 → 38. A row-by-row diff against the pre-change artifact: exactly 4 rows changed, 0 rows with any non-analyserfield changed. The share moved because the description was corrected, not because anything was adopted — both READMEs say so where the jump appears.GT-667 was worked twice in parallel; #485 reached identical arithmetic by enumerating four
MAProws and was closed as superseded, with the comparison recorded on it rather than lost with the branch.Closes GT-668 — the proof that a guard had been seen failing was pinned to a moving branch
GT-666's strongest test read the pre-fix artifact with
git show origin/develop:<mapping>. That is evidence anchored to a moving ref: it read the defect for exactly as long as the fix lived on a branch, and the moment GT-666 merged,origin/developbegan serving the corrected artifact — the case asserting 64 findings found 0. It also carried a shallow-clone skip, so in a slightly different checkout the same rot would have passed in silence.The pre-fix artifact is now a committed fixture (
.harness/fixtures/standards-rule-class/iso-5055-mapping.pre-gt-666.json), byte-identical to the blob GT-666's parent commit carried, with the skip and the env override both removed — an escape hatch out of the one case that proves the guard was ever observed failing can only ever hide a failure. Guard 65's self-test runs 20/20 with zero skips.This is what turned the release red, and it was pre-existing: it broke when GT-666 merged, independently of GT-667.
Verification
The two mapping closures regenerate the derived artifacts in order (
capture-native-evaluability-snapshot.mjs→build-iso-5055-mapping.mjs) and leave46-validate-derived-artifact-orderreporting the chain current and at a fixed point. GT-668 touches no derived artifact — it replaces a fetched fixture with a committed one. On the merged tree: guards 08/09/04/01/49/50/65,generate-executive-summaryandci-runner.mjs governancepass, guard 65's self-test runs 20/20 with zero skips, and core-domain runs green from a clean dist with no pinned count inrule-corpus-triage.spec.tsmoved — none of the three touches the corpus, only its description and the evidence behind it.Board: 654 / 666.
🤖 Generated with Claude Code