feat(core): add fail-closed benchmark release decision contract - #218
feat(core): add fail-closed benchmark release decision contract#218seonghobae wants to merge 63 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthrough새로운 공개 Changes릴리스 수용
Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk: 🟡 Moderate · up to The release-decision contract still accepts limitation text with surrounding spaces, which can let equivalent limitations evade duplicate detection and create ambiguous buyer-visible release reports. This is localized but affects fail-closed correctness; merge should wait for the input-validation fix and documentation follow-up. Sequence Diagram(s)sequenceDiagram
participant BenchmarkEvidence
participant decide_release
participant DeclaredLimitation
participant ReleaseDecisionReport
BenchmarkEvidence->>decide_release: 결과 전달
DeclaredLimitation->>decide_release: 제한사항 전달
decide_release->>decide_release: 입력과 필수 스위트 검증
decide_release->>ReleaseDecisionReport: 릴리스 결정 및 증거 반환
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
- Record 158 open PRs (44 ready, 114 draft) with refreshed exact base/head evidence for the #208-#222 release, enterprise-approval, BAP, and WARC/PROV chains while retaining the 2026-08-21 rows as regression anchors. - Add governance issues #212 and #215 to the operational signal table and note issue #206 closure between snapshots. - Record the required-check provider-failure RCA for the fail-closed Strix re-dispatches on #208, #218, and #220 without weakening the gate. - Tighten the completion-gap contract so superseded inventory counts cannot pass as current evidence.
| | 0x17b4..=0x17b5 | ||
| | 0x180b..=0x180f | ||
| | 0x200b..=0x200f | ||
| | 0x2028..=0x202e |
There was a problem hiding this comment.
📝 Info: Filter is stricter than the documented ignorable set
disallowed_release_limitation_character at release_acceptance.rs rejects 0x2028..=0x202e, two code points wider than the reviewed set documented in the unicode17 test and doctoring (0x202a..=0x202e, 4,174 points). This only rejects U+2028/U+2029 (line/paragraph separators) as well, so behavior is stricter, not wrong, and no test breaks. Align code and docs on the intended set.
Was this helpful? React with 👍 or 👎 to provide feedback.
| let mut limitation_claims = std::collections::BTreeSet::new(); | ||
| for limitation in declared_limitations { | ||
| if !limitation_claims.insert(limitation.unsupported_claim()) { | ||
| return Err(ReleaseDecisionError::DuplicateLimitationClaim); | ||
| } | ||
| } |
There was a problem hiding this comment.
📝 Info: Duplicate-claim dedup depends on constructor canonicalization
decide_release dedups claims by raw string identity (release_acceptance.rs:315-320). This holds only because DeclaredLimitation::new already enforces NFC and rejects surrounding whitespace. Relaxing that constructor would silently weaken the duplicate-claim guarantee.
Was this helpful? React with 👍 or 👎 to provide feedback.
Buyer-visible gap
Issue #203 requires a deterministic release decision that cannot promote missing, inconclusive, duplicated, failed, oversized, ambiguously represented, or non-canonical mandatory-suite evidence into an accepted commercial release. This bounded slice owns only the pure decision contract; benchmark execution, thresholds, corpus licensing/statistics, signed artifact binding, and suite runners remain outside this PR.
Implemented boundary
originweave_core::release_acceptanceprovides the five mandatory suite identities, deterministicaccepted/accepted_with_declared_limitations/rejected/inconclusiveaggregation, exact failed/inconclusive/missing evidence, fail-closed duplicate-suite rejection, bounded buyer-visible limitations, Unicode 17.0.0 presentation-safety validation, NFC and canonical surrounding-whitespace rejection, duplicate limitation-claim identity rejection, and no model/status/author override path.A known threshold failure remains
rejectedeven if other evidence is incomplete. Missing or inconclusive mandatory evidence is never promoted to acceptance.Test-first repair lineage
Historical failing/predecessor heads below are regression lineage only, never current passing evidence.
Duplicate limitation identity
955d24b8a3eca9819e219932dfad3aaece61707f: CI32675693535, Rust job97283280184; repository contracts/format/check passed andconflicting_consequences_for_one_limitation_claim_fail_closedfailed against the old production behavior.b71d6ba3948b799155d177874ab00fa8e2a9d526:decide_releaserejects repeatedunsupported_claimidentity with typedReleaseDecisionError::DuplicateLimitationClaimbefore report emission.5c0478985edccdda33851903282145056ddc2ba6pins both conflicting- and identical-consequence duplicate-claim rejection and preserves the 64-item resource ceiling with unique claims.Canonical limitation text and exact coverage
The release contract rejects non-NFC text and leading/trailing whitespace in both
unsupported_claimandbuyer_consequence, preventing presentation variants from bypassing one-authoritative-claim identity. Exact head51c8ee82e6fd2fa0baf9e8ad76870f3c4e375580then exposed a coverage-evidence defect: CI32764379012, Production coverage job97550390176reported542/544branches,3831/3833lines, and4763/4765regions. The repair at0a9bef98dc5a1e866207d8cf3999c0defeb9a83fadded leading-claim and trailing-consequence resource-bounds regressions without changing production behavior and restored exact four-axis coverage.Current-head canonical-format repair
An intervening contributor move produced exact head
ad149e357c94a4ec050e5452be2f81b82f9b4b2e. CI32776557846, Rust job97588928011failed atcargo fmt --all -- --checkbecausecrates/originweave-core/tests/release_acceptance_resource_bounds.rslacked its canonical final newline. No production behavior failed. The canonical owner branch was repaired with the smallest formatting-only change; no product code was mutated merely to retrigger CI.Exact-current state and evidence
Protected
mainis exactb05d5acca82b9d916ada2c8e82f59f92a89817e1. This PR is open, non-draft, mergeable, and currently exact headcd33b3befabf2cf2efdd8c375d4293dc4408119bagainst that base.On unchanged exact head
cd33b3befabf2cf2efdd8c375d4293dc4408119b:32782969262: success;97608806449: exact-head checkout; repository contracts, canonical formatting, locked workspace/all-target check, full tests, strict Clippy, and rustdoc success;97608806271: exact-head checkout; owned production functions, lines, regions, and branches all exactly 100%;9540469189,exact-coverage-cd33b3befabf2cf2efdd8c375d4293dc4408119b, ZIP SHA-2560da6085f95a395c5142c0bdf183782bd6ec1add9737726fce485f7d923582f3a;32782969238: success;32782969224: success; and32782969166: success after bounded non-busy wait and exact-head refetch; no source mutation was used to retrigger the lane.Formal review inventory contains automated
COMMENTEDevidence only; GitHub records no qualifying counted independentAPPROVEDreview. Two current inline threads remain unresolved and intentionally stay unresolved because they are informational contract notes rather than repaired findings: one notes that the presentation filter also rejects U+2028/U+2029 beyond the documented Default_Ignorable set, and one records that duplicate-claim identity relies on the constructor's NFC/whitespace canonicalization. No predecessor-head check, review, scanner result, status, synthetic merge, queued/skipped/cancelled evidence, or model output is promoted as current-head proof or independent approval.Scope and authority boundary
This PR does not claim release readiness and does not implement #201 signed-artifact authority, #202 enterprise operability, browser/security/recovery suite runners, benchmark persistence, model routing, or benchmark statistics. It introduces no database and no workflow mutation.
Protected-main
AGENTS.mdremains authoritative. Scheduled repository work does not merge, self-approve, tag, publish, mutate workflows, add secrets, or weaken gates.