diff --git a/AGENTS.md b/AGENTS.md index cf622c4..d1a7cc2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -81,15 +81,27 @@ Against an already-running enclave (no observation window): `just verify-now configs/generated/ (= just generate-configs) +sim generate # the named scenarios -> configs/generated/ (= just generate-configs) +sim generate --curated # + the curated composable coverage configs sim generate cb-mux --out-dir /tmp/x +sim scenario --base cb-basic --set clients=geth-teku,get_header=stream # compose (stdout) +sim scenario --spec spec.json # full ScenarioSpec as JSON (the AI-drivable surface) sim generate --check # drift gate: nonzero if on-disk configs != what the generator emits sim preflight configs/generated/cb-mux.yml # ~1s: parse the rendered config with the REAL helix image sim checks --list [--json] # the check contract, machine-readable @@ -257,7 +269,8 @@ you MUST update this file AND its companion doc IN THE SAME COMMIT:** |---|---| | a check's id, tier, verdict conditions, or data source | `docs/CHECKS.md` **and** `src/bin/sim/checks_catalog.rs` | | a CLI flag, a `just` target, the launcher | this file + `README.md` | -| a scenario, the generator, the helix/CB blocks | `tests/fixtures/golden-configs/` (regenerate) + this file's scenario list | +| a scenario, the generator, the helix/CB blocks | `tests/fixtures/golden-configs/` (regenerate) + this file's scenario list + `README.md` + the `genmodel` rows in `docs/ARCH.md` | +| the `ScenarioSpec` surface, its knobs/clients, or `curated()` | `docs/composable-scenarios.md` + this file's "Composable scenarios" + `README.md` + `tests/fixtures/curated-configs/` (`BLESS_CURATED=1`, only after a live run) | | a design law | `docs/DESIGN.md` | | a ratified direction, a staged-plan status | `.agent/NORTH-STAR.md` | | a live-devnet result or a new defect | `.agent/SWEEP-BACKLOG.md` (findings log) | @@ -270,6 +283,11 @@ in `src/checks/cb_metrics.rs` while missing from BOTH `checks_catalog.rs` and `d commit that added it (caught on the next read, now synced). Adding a check is a THREE-file change. If a real registry ever lands in `src/checks`, derive the catalog from it and delete this note. +**Never hard-code a COUNT in prose** ("six scenarios", "all nine", "the two client pairs"). Counts drift the +moment a scenario or client is added and go silently wrong across every doc that repeated them (this happened: +"six"/"nine"/"6 scenarios" were stale in README, ARCH.md, and the runbook simultaneously). Reference the +source of truth instead — `Scenario::ALL`, the `ClientPair` variants, `curated()` — and describe, don't tally. + **New hard-won gotchas belong in the "Known traps" section above**, with the evidence that bought them. A trap that lives only in a commit message will be paid for twice. diff --git a/README.md b/README.md index bf0a0e7..e8bf36f 100644 --- a/README.md +++ b/README.md @@ -64,16 +64,46 @@ Kurtosis uses a default Commit-Boost config that can be overridden by inlining i just generate-configs ``` -Six scenarios are generated: +`sim generate` emits the named scenarios (the frozen, byte-goldened regression set — the full list is +`Scenario::ALL` in `src/bin/sim/genmodel/scenario.rs`). The headline ones: | Config | What it tests | |---|---| | `cb-basic.yml` | Single relay (helix), default CB config | +| `cb-basic-nethermind-prysm.yml` | cb-basic on an alternate EL/CL pair (Law 7) | | `cb-multiple-relays.yml` | Two helix relay instances, aggregated bidding | | `cb-mux.yml` | Mux routing — 128 validators to helix-1, 128 to helix-2 | | `cb-skip-sigverify.yml` | Fast path with BLS signature verification disabled | | `cb-timing-games.yml` | Aggressive per-relay timing overrides for late bidding | | `cb-extra-validation.yml` | Extra get_header validation via local EL RPC | +| `cb-ws-stream.yml` | getHeader over the websocket bid stream | + +For any combination outside the frozen named set (a feature on a specific client, another CL, ...), compose +one with `sim scenario` — see [Composable scenarios](#composable-scenarios-sim-scenario) below. + +### Composable scenarios (`sim scenario`) + +The named scenarios above are frozen points. To compose features freely — e.g. the +websocket stream on the prysm client pair with timing games, a combination no named +scenario covers — use `sim scenario`, which renders a `ScenarioSpec` through the same +assembly seams the goldens pin (so a rendered config is valid by construction): + +```bash +# Start from a named base and apply typed field overrides: +cargo run --bin sim -- scenario \ + --base cb-basic --set get_header=stream,clients=nethermind-prysm,timing_games=true \ + --show-spec --out configs/generated/cb-ws-prysm-tg.yml + +# Or supply a full ScenarioSpec as JSON (the AI-drivable surface; unknown keys rejected): +echo '{"topology":"mux"}' | cargo run --bin sim -- scenario --spec /dev/stdin +``` + +Overridable knobs: `clients` (geth-lighthouse | nethermind-prysm | geth-teku | geth-nimbus | geth-lodestar — +all 5 mainstream CLs), `topology` +(single | two-relays | divergent-relays | mux), `get_header` (http | stream | stream-nokey), +`sigverify` (on | skip | skip-poisoned | poisoned-control), `min_bid` (none | ``), and the +booleans `timing_games` / `extra_validation` / `signer`. `--show-spec` previews the resolved spec +and the features it arms. Design + rationale: [`docs/composable-scenarios.md`](docs/composable-scenarios.md). ## Quick start diff --git a/docs/ARCH.md b/docs/ARCH.md index 066bac2..b0781eb 100644 --- a/docs/ARCH.md +++ b/docs/ARCH.md @@ -115,8 +115,9 @@ different report types because `sim` runs before/around a devnet, not against a | `main.rs` | Dispatch to the three verbs; `tracing` init; nonzero exits on error. | | `generate.rs` | IO boundary for `sim generate`: `.env` image overrides, atomic assemble-then-write, and `--check` drift gate (`generate::check`). Pure assembly lives in `genmodel`. | | `genmodel/mod.rs` | Golden-fixture harness (`golden`, `assert_matches_golden`, `extract_block_scalar`) — the byte-identity oracle for the verbatim config port. | -| `genmodel/scenario.rs` | `Scenario` enum (six scenarios) + `Images` map; `args_file_in()` joins the static fragments + helix const + CB block into a full args-file; `build_mev_params`. | -| `genmodel/helix.rs` | `HELIX_RELAY_CONFIG` — the helix YAML block, byte-identical across all six scenarios (const). | +| `genmodel/scenario.rs` | `Scenario` enum (the named scenarios, `Scenario::ALL`) + `Images` map; `args_file_in()` joins the static fragments + helix const + CB block into a full args-file; `build_mev_params`. `to_spec()` maps each named scenario to a `ScenarioSpec`. | +| `genmodel/spec.rs` | `ScenarioSpec` — the flat, composable, structured (AI-targetable) scenario surface (closed-enum knobs: clients / topology / get_header / sigverify / min_bid / timing_games / extra_validation / signer). `render()` reuses the same assembly seams as `args_file_in` (proven byte-identical for every named scenario), so any composition renders a valid config. `curated()` = high-value composed specs frozen as goldens. Drives `sim scenario` + `sim generate --curated`. See [`composable-scenarios.md`](composable-scenarios.md). | +| `genmodel/helix.rs` | `HELIX_RELAY_CONFIG` — the helix YAML block, byte-identical across all named scenarios (const). | | `genmodel/cb.rs` | The CB TOML block: `cb_toml(CbParams)` + `cb_toml_mux(node0, node1)` — verbatim port of the Python builders, generate-time knobs injected by string building. | | `render.rs` | The **compatibility contract** with the fork: `extract_config_blocks` (pull the two `|` scalars), `substitute_runtime_vars` (strip the `{{ range }}` loop, fill `{{ .VAR }}` dummies), `default_dummies`. Pure. | | `preflight.rs` | `sim preflight`: render + run the real helix image + `classify_helix_probe` (pure, 3-valued). | @@ -130,7 +131,7 @@ different report types because `sim` runs before/around a devnet, not against a This is the load-bearing seam. `sim generate` emits a Kurtosis args-file whose `mev_params` carries **two `|` block scalars** that the forked ethereum-package parses and fills at launch: -- `helix_relay_config: |` — the helix relay's YAML config (byte-identical across all six scenarios). +- `helix_relay_config: |` — the helix relay's YAML config (byte-identical across all named scenarios). - `commit_boost_config: |` — the commit-boost sidecar's TOML config (varies ≤ ~7 lines per scenario). Both blocks are **opaque scalars to YAML** and contain unrendered **Go-template holes** that only the diff --git a/docs/DEVELOPING.md b/docs/DEVELOPING.md index ccb3139..721b402 100644 --- a/docs/DEVELOPING.md +++ b/docs/DEVELOPING.md @@ -143,6 +143,13 @@ A scenario is a typed devnet configuration that assembles into a Kurtosis args-f fixtures. The config↔fork coupling (the two `|` block scalars, the runtime template holes) is explained in [`docs/ARCH.md`](ARCH.md) §4 — read it before touching the block bodies. +> **Composing instead of authoring.** For a one-off combination of *existing* features (e.g. the ws stream on +> the prysm pair with timing games), you do not need a new named scenario — use `sim scenario --base/--set` or +> `--spec ` (`genmodel/spec.rs`, [`docs/composable-scenarios.md`](composable-scenarios.md)). Add a NEW +> named scenario + golden below only when a combination is worth freezing as a regression anchor, or when a +> feature needs a config knob no `ScenarioSpec` field exposes yet (add the field, keeping +> `lower_reproduces_every_scenario` green). + ### Steps 1. **Add the variant** to the `Scenario` enum in diff --git a/docs/composable-scenarios.md b/docs/composable-scenarios.md new file mode 100644 index 0000000..1ac4aa1 --- /dev/null +++ b/docs/composable-scenarios.md @@ -0,0 +1,130 @@ +# Composable scenarios — design + +Status: implemented (`feat/composable-scenarios`, `genmodel/spec.rs` + `sim scenario`). Ship-order steps 1-3 +landed; step 4 (promote curated combos to named+goldened) and step 5 (NL front-end) are follow-ups. + +## Validation (live) + +- Offline: `lower_reproduces_every_scenario` proves `render(spec) == args_file_in` byte-for-byte for all 13 + named scenarios; round-trip + composite-order + total-render property tests green; full suite + clippy + `-D warnings` clean. +- Real-schema: a novel compose (`get_header=stream` + `clients=nethermind-prysm` + `timing_games`) passes + `sim preflight` (helix config-parse against the real image). +- End-to-end: a novel compose (`cb-timing-games` base + `extra_validation`, two-relays — no named scenario, + no golden) rendered by `sim scenario`, stood up a full Kurtosis devnet, and BOTH composed features were + positively proven from CB debug logs: `feature.timing_games` PASS (385 marker lines), + `feature.extra_validation` PASS (126 marker lines); overall exit 0. The two WARNs (get_header deadline + timeouts, p95 latency) are the expected artifacts of aggressive timing games, not failures. + +## Problem + +A test scenario is one hardcoded `Scenario` enum variant mapped to one frozen golden YAML fixture (byte-diff +acceptance). Features (WS header-stream, timing-games, extra-validation, skip-sigverify, min-bid, mux / +multi-relay topology, EL/CL client pair, signer) are not composable: producing "WS + prysm + timing-games" +needs a whole new fixture and a new enum variant. The ask: make scenarios composable, drivable by a structured +(AI-targetable) surface, and enumerable/testable — without a combinatorial pile of golden fixtures. + +## The design (post-grill: 3 design agents + 3 adversarial lenses) + +**`ScenarioSpec`** — a flat struct of closed enums / `Option`s is the single source of truth. It is the surface +a caller (human, or an agent emitting JSON) targets; it is `lower()`'s input; its `armed_features()` is the +verifier oracle. Closed enums make illegal values inexpressible; **smart constructors** make the three known +illegal combinations unrepresentable, so there is no `validate()` returning conflicts — illegal states don't +compile. + +Axes (each maps onto an EXISTING seam, nothing new is emitted): +- `clients: ClientPair {GethLighthouse, NethermindPrysm}` → `ElCl::{DEFAULT, ALT}` +- `topology: Topology {Single, TwoRelays, DivergentRelays, Mux}` — relay count + subsidy intent in one knob +- `get_header: HeaderTransport {Http, Stream{api_key: Present|Absent}}` — Absent = the ws-nokey negative control +- `timing_games: bool` (timeouts 400/2000 ride it), `extra_validation: bool`, `signer: bool` +- `sigverify: Sigverify {On, Skip, SkipPoisoned, PoisonedControl}` — collapses the mutually-exclusive combos +- `min_bid: MinBid {None, Floor(f64)}` — subsidy is DERIVED (Floor ⇒ subsidy 0), never a spec field + +**`lower(spec, images, keys_dir) -> args_file`** — deterministic, total on any constructible spec. Reuses +verbatim: `CbParams` + its `extra_pbs_lines`/`per_relay_lines` `Vec` seams, `cb_toml`, `cb_toml_mux` +(mux is a dedicated exclusive branch — structurally a different template), `build_mev_params`, `ElCl`, +`poisoned_relay_url`, `WRONG_RELAY_PUBKEY`, `load_pubkeys`. Subsidy, timeouts, and network_params are DERIVED +inside `lower` (they are non-orthogonal couplings each scenario bundles — see Honesty note). The seam-line +fragments are composed in a FIXED canonical order, pinned by a dedicated composite-spec test (below). + +**Acceptance model** (byte-golden evolves, does not die): +- The 13 named scenarios: `NAMED: &[(&str, ScenarioSpec)]` const table. `every_scenario_matches_its_golden` + byte-diffs `lower(named)` against the frozen golden (UNCHANGED `assert_matches_golden`). This is the + migration safety net AND the regression anchor. `Scenario::from_name`/`ALL` stay working, backed by `NAMED`. +- The combinatorial space is NEVER byte-goldened. Two OFFLINE guards over an enumeration of the pruned product: + - `every_pruned_spec_renders_without_panic` — `lower` is total across the legal space. + - **round-trip**: `detect_enabled_features(lower(spec)) == spec.armed_features()`. Documented as a + RENDERER-DRIFT guard ONLY — it proves emit↔detect agree, NOT that the config is valid CB (both sides share + the same key strings, so a shared typo passes; `[pbs]` has no `deny_unknown_fields`). Real config + validation is `sim preflight` (Law 1), which callers run before a live run. + - **composite-spec fragment-order pin**: a unit test asserting `to_cb_params()` output for a COMPOSITE spec + (e.g. timing+extra-validation) has the exact expected line order. The 13 goldens each pin ONE order; only a + composite test catches a canonical-order regression on combinations they don't cover. + +**Driving surface** (the "run me a scenario like X with Y and Z"): +- `sim generate --base --set k=v,...` — a deterministic keyword overlay: start from a named base, apply + typed field updates, render. Zero model. This is the composability UX. +- `sim generate --spec ` — render a full `ScenarioSpec` supplied as JSON (`deny_unknown_fields`). + This is the AI-driven entry: an agent (Lisa, chat) composes the spec and passes it; validity is by + construction because output comes from `lower()`. + +The system is **AI-driven by construction** — the structured surface IS what an agent targets — without a +brittle in-binary LLM/NL parser. A natural-language front-end (`sim scenario ""`) is a thin, optional +add on top of this surface; it is deliberately deferred (see Cut list) until the deterministic core is proven. + +## Cut list (grill-driven — what we deliberately did NOT build) + +- **No `sim matrix` live-sweep verb / no "N/M pass" coverage integer.** At ~10 min/cell live the sweep has no + consumer, and a single pass-tally conflates config-rendered / never-run / expectation-downgraded cells — the + coverage-theater trap. Enumeration ships as offline tests only. If genuine cross-situation coverage is wanted, + the right form is promoting a few high-value combos to NAMED, runnable, byte-goldened scenarios (below). +- **No 4-valued `Expectation` / `expected_checks(spec)`.** "Proven" is a RUNTIME outcome (a marker fires only + if a bid/getHeader lands in the window; min-bid rejection needs `rejections>0`), so a spec cannot soundly + declare it. Gating `--require-feature-proof` off a per-spec expectation table re-hardens what the classifiers + softened to WARN and manufactures flaky reds. `--require-feature-proof` keeps deriving from the RENDERED + config as it does today. The spec declares only `armed_features()` (2-valued: armed / not). +- **No `Conflict`/`validate()`.** Smart constructors make illegal combos unrepresentable; `min_bid ⇒ subsidy 0` + is a derivation, not a reportable clamp. +- **No in-binary NL/AI layer, no schemars overlay type** yet. Deferred behind the deterministic surface. + +## Client coverage (the `clients` axis) + +The `clients` axis is the Law-7 matrix. CLs are the axis that matters for CB behavior (the blinded-block / +get_header flow), so the additional pairs vary the CL against geth; `nethermind-prysm` keeps its historical EL. +`ClientPair` variants: `geth-lighthouse` (default), `nethermind-prysm`, `geth-teku`, `geth-nimbus`, +`geth-lodestar` — i.e. **all 5 mainstream CLs** (lighthouse, prysm, teku, nimbus, lodestar). Adding a client is +an `ElCl` + `ClientPair` variant + serde name; the rpc_url naming (`el-1-{el}-{cl}`) is already parametric. + +## Curated coverage points (the right "enumerate situations") + +Rather than a Cartesian sweep, `spec::curated()` freezes a handful of genuinely-interesting composed specs as +named+goldened regression anchors (`tests/fixtures/curated-configs/`), each **live-validated on a devnet** +before its golden is trusted (a golden of a config that has never run is worthless). Emit them with +`sim generate --curated`. + +| Curated point | Why | Live result | +|---|---|---| +| `cb-basic-teku` | teku CL (Law 7) | 14 PASS / 0 WARN / 0 FAIL; 33 payloads, 100% MEV | +| `cb-basic-nimbus` | nimbus CL (Law 7) | 14 / 0 / 0; 31 payloads, 93.9% MEV | +| `cb-basic-lodestar` | lodestar CL (Law 7) | 14 / 0 / 0; 31 payloads, 93.9% MEV | +| `cb-ws-prysm` | ws stream on prysm — the highest-suspicion route coupling | 15 / 1 / 0; **ws stream FIRED** (30 headers, 1 startup-race fallback) — the coupling concern is refuted by measurement | +| `cb-timing-extra-validation` | the composition claim (both markers must fire) | both `feature.timing_games` + `feature.extra_validation` proven from CB logs | + +Deferred (add later, each with a live run): `poison × prysm` (skip_sigverify differential on the ALT pair), +`min_bid (Floor) × prysm` (the `[pbs]` silent-ignore canary against a real CB parse). + +## Honesty note (orthogonality is partly fiction) + +The struct advertises a product space, but `lower` only honors a subregion: mux composes with nothing (dedicated +template), min-bid/poison require Single relay, subsidy/timeouts/network_params are derived not chosen. This is +real and stated: the win is that the illegal region is made unrepresentable by construction (smart constructors) +rather than absent-from-a-match, and that the 13 named scenarios are reproduced byte-for-byte through the new +path. We do NOT market "features combine freely." + +## Ship order + +1. `ScenarioSpec` + smart constructors + `lower` + `armed_features` + the 13-named migration (byte-golden net). +2. Offline guards: round-trip + `renders_without_panic` + composite fragment-order pin. +3. `sim generate --base/--set` and `--spec` driving surface. +4. (Follow-up) promote the 4 curated combos to named+goldened scenarios after a live run each. +5. (Deferred) NL front-end, if demand. diff --git a/docs/local-kurtosis-e2e.md b/docs/local-kurtosis-e2e.md index ede8475..6d5f824 100644 --- a/docs/local-kurtosis-e2e.md +++ b/docs/local-kurtosis-e2e.md @@ -14,7 +14,8 @@ ## The two flows (pick one) - **cb-testing (this repo) — the verification harness.** `cb-verify` Rust binary with tiered - pass/fail checks, 6 scenarios, forked `Commit-Boost/ethereum-package` submodule, helix relay. + pass/fail checks, the named scenarios (`Scenario::ALL`) + composable `sim scenario`, forked + `Commit-Boost/ethereum-package` submodule, helix relay. This is the authoritative flow (success criteria baked in). **We use this.** - **commit-boost-client `just kurtosis-*`** — lighter alt: upstream `ethereum-package`, `mev_type: commit-boost`, mev-boost-relay, no pass/fail harness. Good for a quick smoke. diff --git a/src/bin/sim/cli.rs b/src/bin/sim/cli.rs index f21dcea..0acfe3c 100644 --- a/src/bin/sim/cli.rs +++ b/src/bin/sim/cli.rs @@ -77,5 +77,33 @@ pub enum Command { /// generator would produce, and exit nonzero on any drift (CI / agent gate). #[arg(long)] check: bool, + /// Also emit the curated composable coverage configs (the additional CL + /// clients + high-value feature combos; rendered from `ScenarioSpec`). + #[arg(long)] + curated: bool, + }, + /// Render a COMPOSABLE scenario config from a structured `ScenarioSpec` — a + /// full JSON spec, or a named base with typed field overrides. Unlike + /// `generate` (the 13 frozen named scenarios), this composes features freely + /// (e.g. ws + prysm + timing-games). Output is a Kurtosis args-file, valid by + /// construction (it renders through the same seams the goldens pin). + Scenario { + /// Path to a `ScenarioSpec` JSON file (the full structured surface). + /// Mutually exclusive with `--base`/`--set`. + #[arg(long)] + spec: Option, + /// A named scenario to start from (e.g. `cb-mux`); default `cb-basic`. + #[arg(long)] + base: Option, + /// Comma-separated `key=value` field overrides applied onto the base, + /// e.g. `--set get_header=stream,clients=nethermind-prysm,timing_games=true`. + #[arg(long)] + set: Option, + /// Write the rendered args-file here (default: stdout). + #[arg(long)] + out: Option, + /// Print the resolved spec as JSON to stderr before rendering (preview). + #[arg(long)] + show_spec: bool, }, } diff --git a/src/bin/sim/generate.rs b/src/bin/sim/generate.rs index 157e1ca..1501ed8 100644 --- a/src/bin/sim/generate.rs +++ b/src/bin/sim/generate.rs @@ -13,20 +13,33 @@ use eyre::{Result, WrapErr, eyre}; use crate::genmodel::scenario::{Images, Scenario}; -/// Generate one scenario (by name) or all six (`None`) into `out_dir`. Reads -/// `keys/` and `.env` relative to the CWD (the repo root — how `just -/// generate-configs` and the Python generator both run). -pub fn run(scenario: Option<&str>, out_dir: &Path) -> Result<()> { - run_in(scenario, out_dir, Path::new("keys"), Path::new(".env")) +/// Generate one scenario (by name) or all named scenarios (`None`) into +/// `out_dir`, plus the curated composable configs when `curated`. Reads `keys/` +/// and `.env` relative to the CWD (the repo root — how `just generate-configs` +/// and the Python generator both run). +pub fn run(scenario: Option<&str>, out_dir: &Path, curated: bool) -> Result<()> { + run_in( + scenario, + out_dir, + Path::new("keys"), + Path::new(".env"), + curated, + ) } /// Testable core with the two IO roots injected. Assembles ALL bodies (reading + /// validating the mux key files) BEFORE writing anything, so a missing/malformed /// keys file fails cleanly with nothing written — no partial output. This mirrors /// the Python's pre-write `load_pubkeys` + `sys.exit(1)` all-or-nothing contract. -fn run_in(scenario: Option<&str>, out_dir: &Path, keys_dir: &Path, env_path: &Path) -> Result<()> { +fn run_in( + scenario: Option<&str>, + out_dir: &Path, + keys_dir: &Path, + env_path: &Path, + curated: bool, +) -> Result<()> { let images = images_from_env(env_path); - let outputs = assemble(scenario, &images, keys_dir)?; + let outputs = assemble(scenario, &images, keys_dir, curated)?; fs::create_dir_all(out_dir) .wrap_err_with(|| format!("creating output dir {}", out_dir.display()))?; @@ -43,8 +56,14 @@ fn run_in(scenario: Option<&str>, out_dir: &Path, keys_dir: &Path, env_path: &Pa /// Verify the on-disk configs already match what the generator would produce, /// WITHOUT writing (CI / agent drift gate). Errors (nonzero exit) on any drift. -pub fn check(scenario: Option<&str>, out_dir: &Path) -> Result<()> { - check_in(scenario, out_dir, Path::new("keys"), Path::new(".env")) +pub fn check(scenario: Option<&str>, out_dir: &Path, curated: bool) -> Result<()> { + check_in( + scenario, + out_dir, + Path::new("keys"), + Path::new(".env"), + curated, + ) } fn check_in( @@ -52,9 +71,10 @@ fn check_in( out_dir: &Path, keys_dir: &Path, env_path: &Path, + curated: bool, ) -> Result<()> { let images = images_from_env(env_path); - let outputs = assemble(scenario, &images, keys_dir)?; + let outputs = assemble(scenario, &images, keys_dir, curated)?; let mut drift: Vec = Vec::new(); for (name, body) in &outputs { @@ -84,6 +104,7 @@ fn assemble( scenario: Option<&str>, images: &Images, keys_dir: &Path, + curated: bool, ) -> Result> { let scenarios: Vec = match scenario { Some(name) => vec![ @@ -92,10 +113,21 @@ fn assemble( ], None => Scenario::ALL.to_vec(), }; - scenarios + let mut out: Vec<(String, String)> = scenarios .iter() .map(|s| Ok((s.name().to_string(), s.args_file_in(images, keys_dir)?))) - .collect() + .collect::>()?; + // The curated composable coverage points (rendered from ScenarioSpec, not + // the Scenario enum). `--curated` emits them alongside the named scenarios. + if curated { + for (name, spec) in crate::genmodel::spec::curated() { + out.push(( + name.to_string(), + spec.render(&spec.auto_comment(), images, keys_dir)?, + )); + } + } + Ok(out) } fn names() -> Vec<&'static str> { @@ -104,7 +136,7 @@ fn names() -> Vec<&'static str> { /// Build the image map from defaults, overridden by `.env` if present. Mirrors /// the Python `.env` key names. A missing `.env` is not an error (defaults win). -fn images_from_env(env_path: &Path) -> Images { +pub(crate) fn images_from_env(env_path: &Path) -> Images { let mut images = Images::default(); let Ok(contents) = fs::read_to_string(env_path) else { return images; @@ -173,7 +205,7 @@ mod tests { fn run_writes_all_six_matching_assembly() { let dir = std::env::temp_dir().join(format!("sim-gen-{}", std::process::id())); let _ = fs::remove_dir_all(&dir); - run(None, &dir).expect("generate all"); + run(None, &dir, false).expect("generate all"); let images = images_from_env(Path::new(".env")); for s in Scenario::ALL { let produced = fs::read_to_string(dir.join(format!("{}.yml", s.name()))).unwrap(); @@ -189,15 +221,15 @@ mod tests { fn check_passes_when_current_and_fails_on_drift() { let dir = std::env::temp_dir().join(format!("sim-check-{}", std::process::id())); let _ = fs::remove_dir_all(&dir); - run(None, &dir).expect("seed"); + run(None, &dir, false).expect("seed"); // Fresh output → check is clean. - check(None, &dir).expect("check should pass on freshly-generated configs"); + check(None, &dir, false).expect("check should pass on freshly-generated configs"); // Mutate one file → check must fail. let f = dir.join("cb-basic.yml"); let mut body = fs::read_to_string(&f).unwrap(); body.push_str("\n# hand-edit\n"); fs::write(&f, body).unwrap(); - let err = check(None, &dir).unwrap_err(); + let err = check(None, &dir, false).unwrap_err(); assert!(err.to_string().contains("out of date"), "got: {err}"); assert!( err.to_string().contains("cb-basic.yml"), @@ -217,6 +249,7 @@ mod tests { &dir, Path::new("/no/such/keys"), Path::new("/no/such/.env"), + false, ) .unwrap_err(); assert!(err.to_string().contains("pubkey file"), "got: {err}"); diff --git a/src/bin/sim/genmodel/mod.rs b/src/bin/sim/genmodel/mod.rs index c827b23..46884ac 100644 --- a/src/bin/sim/genmodel/mod.rs +++ b/src/bin/sim/genmodel/mod.rs @@ -15,6 +15,7 @@ pub mod cb; pub mod helix; pub mod scenario; +pub mod spec; /// Extract a YAML `|` block scalar (named `key`, at 2-space indent) from `yaml`, /// de-indented 4 spaces, with trailing blank lines removed. Test-only oracle diff --git a/src/bin/sim/genmodel/scenario.rs b/src/bin/sim/genmodel/scenario.rs index 564e8bf..395e0d2 100644 --- a/src/bin/sim/genmodel/scenario.rs +++ b/src/bin/sim/genmodel/scenario.rs @@ -12,6 +12,7 @@ use eyre::{Result, WrapErr}; use super::cb::{CbParams, SignerParams, cb_toml, cb_toml_mux}; use super::helix::HELIX_RELAY_CONFIG; +use super::spec; // --- Vetted static fragments (verbatim from Python) ------------------------- @@ -41,7 +42,7 @@ impl ElCl { }; /// The `participants:` fragment for this pair. - fn participants(&self) -> String { + pub(super) fn participants(&self) -> String { format!( "participants:\n - el_type: {}\n cl_type: {}", self.el, self.cl @@ -50,15 +51,15 @@ impl ElCl { /// The first participant's EL RPC endpoint, as the ethereum-package names /// it (`el-{index}-{el}-{cl}`, 1-indexed). - fn el_rpc_url(&self) -> String { + pub(super) fn el_rpc_url(&self) -> String { format!("http://el-1-{}-{}:8545", self.el, self.cl) } } -const COMMON_ADDITIONAL_SERVICES: &str = +pub(super) const COMMON_ADDITIONAL_SERVICES: &str = "additional_services:\n - dora\n - spamoor\n - prometheus"; -const COMMON_NETWORK_PARAMS: &str = r#"network_params: +pub(super) const COMMON_NETWORK_PARAMS: &str = r#"network_params: network: kurtosis network_id: "3151908" deposit_contract_address: "0x00000000219ab540356cBB839Cbe05303d7705Fa" @@ -72,7 +73,7 @@ const COMMON_NETWORK_PARAMS: &str = r#"network_params: prefunded_accounts: '{"0xb9e79d19f651a941757b35830232E7EFC77E1c79": {"balance": "100000ETH"}}' "#; -const MUX_NETWORK_PARAMS: &str = r#"network_params: +pub(super) const MUX_NETWORK_PARAMS: &str = r#"network_params: network: kurtosis network_id: "3151908" deposit_contract_address: "0x00000000219ab540356cBB839Cbe05303d7705Fa" @@ -194,8 +195,77 @@ impl Scenario { } } + /// The `ScenarioSpec` this named scenario corresponds to. The migration + /// contract: `spec.render(self.comment(), ..) == self.args_file_in(..)` for + /// every scenario (test `lower_reproduces_every_scenario`), which pins the + /// composable `ScenarioSpec::render` path against the byte-golden'd assembly. + pub fn to_spec(self) -> spec::ScenarioSpec { + use spec::{ + ClientPair, HeaderTransport, KeyPresence, MinBid, ScenarioSpec, Sigverify, Topology, + }; + let base = ScenarioSpec { + clients: match self { + Scenario::BasicAltClients => ClientPair::NethermindPrysm, + _ => ClientPair::GethLighthouse, + }, + ..ScenarioSpec::default() + }; + match self { + Scenario::Basic | Scenario::BasicAltClients => base, + Scenario::MultipleRelays => ScenarioSpec { + topology: Topology::DivergentRelays, + ..base + }, + Scenario::TimingGames => ScenarioSpec { + topology: Topology::TwoRelays, + timing_games: true, + ..base + }, + Scenario::Mux => ScenarioSpec { + topology: Topology::Mux, + ..base + }, + Scenario::MinBid => ScenarioSpec { + min_bid: MinBid::Floor(0.5), + ..base + }, + Scenario::Signer => ScenarioSpec { + signer: true, + ..base + }, + Scenario::SkipSigverify => ScenarioSpec { + sigverify: Sigverify::Skip, + ..base + }, + Scenario::SigverifyDiff => ScenarioSpec { + sigverify: Sigverify::SkipPoisoned, + ..base + }, + Scenario::SigverifyDiffControl => ScenarioSpec { + sigverify: Sigverify::PoisonedControl, + ..base + }, + Scenario::ExtraValidation => ScenarioSpec { + extra_validation: true, + ..base + }, + Scenario::WsStream => ScenarioSpec { + get_header: HeaderTransport::Stream { + api_key: KeyPresence::Present, + }, + ..base + }, + Scenario::WsStreamNoKey => ScenarioSpec { + get_header: HeaderTransport::Stream { + api_key: KeyPresence::Absent, + }, + ..base + }, + } + } + /// The leading comment block (verbatim from Python). - fn comment(&self) -> &'static str { + pub(super) fn comment(&self) -> &'static str { match self { Scenario::Basic => { "# cb-basic: Single relay (helix) with default Commit-Boost config.\n\ @@ -465,13 +535,13 @@ pub const WRONG_RELAY_PUBKEY: &str = "0xaaf6c1251e73fb600624937760fef218aace5b25 /// single helix instance as `helix-relay-2` (index = participant_count 2 + /// relay_index 0), listening on the fixed in-enclave port 4040 - confirmed by /// the live 2-helix runs (helix-relay-2/-3). -fn poisoned_relay_url() -> String { +pub(super) fn poisoned_relay_url() -> String { format!("http://{WRONG_RELAY_PUBKEY}@helix-relay-2:4040") } // --- mev_params assembly (ports build_mev_params) --------------------------- -fn build_mev_params( +pub(super) fn build_mev_params( relays: &[&str], images: &Images, cb_block: &str, @@ -541,7 +611,7 @@ fn push_block_scalar(lines: &mut Vec, body: &str) { /// so a missing/malformed keys file surfaces as a clean `run` error (and lets the /// caller validate BEFORE writing anything), matching the Python's pre-write /// `load_pubkeys` + `sys.exit(1)` rather than panicking mid-generation. -fn load_pubkeys(keys_dir: &Path, node: u8) -> Result> { +pub(super) fn load_pubkeys(keys_dir: &Path, node: u8) -> Result> { let path = keys_dir.join(format!("node-{node}-pubkeys.json")); let raw = std::fs::read_to_string(&path) .wrap_err_with(|| format!("reading pubkey file {}", path.display()))?; diff --git a/src/bin/sim/genmodel/spec.rs b/src/bin/sim/genmodel/spec.rs new file mode 100644 index 0000000..41365eb --- /dev/null +++ b/src/bin/sim/genmodel/spec.rs @@ -0,0 +1,810 @@ +//! `ScenarioSpec` — the composable, structured surface for CB test scenarios. +//! +//! A scenario is a flat struct of closed enums / `Option`s. It is the surface a +//! caller (a human, or an agent emitting JSON) targets, it is `lower()`'s input, +//! and its `armed_features()` is the verifier oracle. Closed enums make illegal +//! VALUES inexpressible; `lower()` is total on every constructible spec except the +//! one genuinely unrenderable family (mux + any CB-config injection), which it +//! rejects with a clear error rather than a silent wrong config. +//! +//! `lower()` reuses the existing assembly seams verbatim (`CbParams`, `cb_toml`, +//! `cb_toml_mux`, `build_mev_params`, `ElCl`, `poisoned_relay_url`, +//! `load_pubkeys`); it introduces no new YAML/TOML emission. The 13 named +//! scenarios are reproduced byte-for-byte (see `Scenario::to_spec` + the +//! `lower_reproduces_every_scenario` test), so byte-golden acceptance is +//! preserved; the combinatorial space is guarded by offline property tests. + +use std::path::Path; + +use eyre::Result; + +use super::cb::{CbParams, SignerParams, cb_toml, cb_toml_mux}; +use super::scenario::{ + COMMON_ADDITIONAL_SERVICES, COMMON_NETWORK_PARAMS, ElCl, Images, MUX_NETWORK_PARAMS, + build_mev_params, load_pubkeys, poisoned_relay_url, +}; +use cb_testnet_verifier::checks::feature_fired::Feature; + +/// The api key the ws stream authenticates with — a fixed devnet UUID that rides +/// validator registration so helix TOFU-binds it (see the `cb-ws-stream` comment). +const WS_API_KEY: &str = "9d5c2f4e-1b7a-4c3d-8e6f-0a1b2c3d4e5f"; + +/// The EL/CL client pair (Law 7: coverage is a matrix, not a point). The CL is +/// the axis that matters for CB behavior (the blinded-block / get_header flow), +/// so the additional pairs vary the CL against geth; `nethermind-prysm` keeps +/// its historical EL pairing. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum ClientPair { + #[default] + GethLighthouse, + NethermindPrysm, + GethTeku, + GethNimbus, + GethLodestar, +} + +/// Relay topology. Encodes relay count AND the subsidy intent in one knob: +/// `DivergentRelays` is the `[1, 2]` per-relay subsidy split that makes best-bid +/// selection a real discrimination; `TwoRelays` is two relays on the shared `1` +/// subsidy (the timing-games shape); `Mux` is per-node `[[mux]]` routing. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum Topology { + #[default] + Single, + TwoRelays, + DivergentRelays, + Mux, +} + +/// Whether the ws stream carries its api key. `Absent` is the negative control: +/// helix refuses the handshake, every slot falls back to HTTP, and the ws proof +/// is expected inconclusive (this is `cb-ws-stream-nokey`). +#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum KeyPresence { + Present, + Absent, +} + +/// getHeader transport. `Stream` sets `get_header = "stream"` per relay. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum HeaderTransport { + #[default] + Http, + Stream { + api_key: KeyPresence, + }, +} + +/// Signature-verification mode. Collapses the mutually-exclusive skip/poison +/// combinations into one closed choice so illegal shapes are not constructible. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum Sigverify { + #[default] + On, + /// `skip_sigverify = true`, clean relay — the codepath exists but a plain run + /// cannot positively observe it (honest WARN, not a failure). + Skip, + /// `skip_sigverify = true` + a wrong-pubkey literal relay: an auction winner + /// is positive proof the skip codepath fired (the differential treatment arm). + SkipPoisoned, + /// Wrong-pubkey literal relay, skip OFF: CB rejects every bid (the control arm, + /// expected to fail payload delivery). + PoisonedControl, +} + +/// The `min_bid_eth` floor. `Floor` forces the builder subsidy to 0 (a floor is +/// only meaningful with the subsidy off), so subsidy is derived, never a field. +#[derive(Debug, Clone, Copy, PartialEq, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum MinBid { + None, + Floor(f64), +} + +/// The composable scenario surface. Every field is a closed enum / `Option`, so +/// no value outside the modelled space is expressible. `Default` == `cb-basic`. +#[derive(Debug, Clone, PartialEq, serde::Serialize, serde::Deserialize)] +#[serde(deny_unknown_fields, default)] +pub struct ScenarioSpec { + pub clients: ClientPair, + pub topology: Topology, + pub get_header: HeaderTransport, + pub timing_games: bool, + pub extra_validation: bool, + pub signer: bool, + pub sigverify: Sigverify, + pub min_bid: MinBid, +} + +impl Default for ScenarioSpec { + /// The baseline = cb-basic: every knob off. Overlays move deltas off this. + fn default() -> Self { + Self { + clients: ClientPair::default(), + topology: Topology::default(), + get_header: HeaderTransport::default(), + timing_games: false, + extra_validation: false, + signer: false, + sigverify: Sigverify::default(), + min_bid: MinBid::None, + } + } +} + +impl ScenarioSpec { + /// Parse a full `ScenarioSpec` from JSON (the AI-drivable surface). Unknown + /// keys are rejected (`deny_unknown_fields`); unspecified fields take their + /// `Default` (= cb-basic), so a partial JSON is a delta off the baseline. + pub fn from_json(json: &str) -> Result { + serde_json::from_str(json).map_err(|err| eyre::eyre!("invalid ScenarioSpec JSON: {err}")) + } + + /// Resolve a spec from an optional named base plus a comma-separated + /// `key=value` override string — the deterministic keyword-overlay surface + /// (`--base cb-mux --set get_header=stream,clients=nethermind-prysm`). No + /// model: the base is a named scenario's spec, each override is a typed field. + pub fn from_base_and_overrides(base: Option<&str>, set: Option<&str>) -> Result { + let mut spec = match base { + Some(name) => super::scenario::Scenario::from_name(name) + .ok_or_else(|| eyre::eyre!("unknown base scenario {name:?}"))? + .to_spec(), + None => ScenarioSpec::default(), + }; + if let Some(sets) = set { + for pair in sets.split(',').filter(|s| !s.trim().is_empty()) { + let (k, v) = pair + .split_once('=') + .ok_or_else(|| eyre::eyre!("bad --set entry {pair:?}, want key=value"))?; + spec.apply_override(k.trim(), v.trim())?; + } + } + Ok(spec) + } + + /// Apply one `key=value` override. Unknown keys / values are a loud error; + /// enum values reuse the serde kebab-case names (`nethermind-prysm`, etc.). + pub fn apply_override(&mut self, key: &str, value: &str) -> Result<()> { + fn de(key: &str, v: &str) -> Result { + serde_json::from_value(serde_json::Value::String(v.to_string())) + .map_err(|err| eyre::eyre!("bad value {v:?} for {key}: {err}")) + } + fn de_bool(key: &str, v: &str) -> Result { + match v { + "true" => Ok(true), + "false" => Ok(false), + other => eyre::bail!("bad bool {other:?} for {key}; want true|false"), + } + } + match key { + "clients" => self.clients = de(key, value)?, + "topology" => self.topology = de(key, value)?, + "sigverify" => self.sigverify = de(key, value)?, + "timing_games" => self.timing_games = de_bool(key, value)?, + "extra_validation" => self.extra_validation = de_bool(key, value)?, + "signer" => self.signer = de_bool(key, value)?, + "get_header" => { + self.get_header = match value { + "http" => HeaderTransport::Http, + "stream" => HeaderTransport::Stream { + api_key: KeyPresence::Present, + }, + "stream-nokey" => HeaderTransport::Stream { + api_key: KeyPresence::Absent, + }, + other => { + eyre::bail!("bad get_header {other:?}; want http|stream|stream-nokey") + } + } + } + "min_bid" => { + self.min_bid = match value { + "none" => MinBid::None, + f => MinBid::Floor( + f.parse() + .map_err(|err| eyre::eyre!("bad min_bid floor {f:?}: {err}"))?, + ), + } + } + other => eyre::bail!( + "unknown key {other:?}; want one of clients|topology|get_header|timing_games|\ + extra_validation|signer|sigverify|min_bid" + ), + } + Ok(()) + } + + /// The EL/CL pair this spec runs on. + pub fn el_cl(&self) -> ElCl { + match self.clients { + ClientPair::GethLighthouse => ElCl::DEFAULT, + ClientPair::NethermindPrysm => ElCl::ALT, + ClientPair::GethTeku => ElCl { + el: "geth", + cl: "teku", + }, + ClientPair::GethNimbus => ElCl { + el: "geth", + cl: "nimbus", + }, + ClientPair::GethLodestar => ElCl { + el: "geth", + cl: "lodestar", + }, + } + } + + /// The relay list. Single = one helix; every multi topology = two helix. + fn relays(&self) -> &'static [&'static str] { + match self.topology { + Topology::Single => &["helix"], + Topology::TwoRelays | Topology::DivergentRelays | Topology::Mux => &["helix", "helix"], + } + } + + /// The builder-subsidy YAML value. Derived from topology + min_bid: a floor + /// forces 0; divergent relays use the `[1, 2]` split; else the scalar 1. + fn builder_subsidy(&self) -> &'static str { + match (self.topology, self.min_bid) { + (_, MinBid::Floor(_)) => "0", + (Topology::DivergentRelays, _) => "[1, 2]", + _ => "1", + } + } + + fn network_params(&self) -> &'static str { + match self.topology { + Topology::Mux => MUX_NETWORK_PARAMS, + _ => COMMON_NETWORK_PARAMS, + } + } + + /// Compose the `CbParams` seam lines in a FIXED canonical order. The order + /// is unconstrained by the 13 goldens (none combines two `[pbs]` features), + /// so a dedicated composite test pins it — a reorder there is the silent + /// byte-drift risk. Mux does NOT go through here (it has no injection seam). + fn to_cb_params(&self) -> CbParams { + let mut p = CbParams::basic(); + + // Timeouts ride timing-games. + if self.timing_games { + p.timeout_get_header_ms = 400; + p.timeout_get_payload_ms = 2000; + } + + // [pbs] lines, canonical order: skip_sigverify, extra_validation, min_bid. + let mut pbs = Vec::new(); + if matches!(self.sigverify, Sigverify::Skip | Sigverify::SkipPoisoned) { + pbs.push("skip_sigverify = true".to_string()); + } + if self.extra_validation { + pbs.push("extra_validation_enabled = true".to_string()); + pbs.push(format!(r#"rpc_url = "{}""#, self.el_cl().el_rpc_url())); + } + if let MinBid::Floor(x) = self.min_bid { + pbs.push(format!("min_bid_eth = {x}")); + } + p.extra_pbs_lines = pbs; + + // Per-relay lines, canonical order: timing-games, then ws stream. + let mut per_relay = Vec::new(); + if self.timing_games { + per_relay.extend([ + "enable_timing_games = true".to_string(), + "target_first_request_ms = 100".to_string(), + "frequency_get_header_ms = 200".to_string(), + ]); + } + if let HeaderTransport::Stream { api_key } = self.get_header { + per_relay.push(r#"get_header = "stream""#.to_string()); + if matches!(api_key, KeyPresence::Present) { + per_relay.push(format!(r#"headers = {{ X-Api-Key = "{WS_API_KEY}" }}"#)); + } + } + p.per_relay_lines = per_relay; + + // Fault injection: a wrong-pubkey literal relay replaces the range loop. + if matches!( + self.sigverify, + Sigverify::SkipPoisoned | Sigverify::PoisonedControl + ) { + p.literal_relay_url = Some(poisoned_relay_url()); + } + + if self.signer { + p.signer = Some(SignerParams::devnet()); + } + + p + } + + /// The `feature_fired::Feature`s this spec arms — a pure projection of the + /// config knobs onto the verifier's feature enum. This is what the round-trip + /// test pins against `detect_enabled_features(lower(spec))`. It is 2-valued + /// (armed / not): whether a feature is PROVEN is a runtime outcome the spec + /// cannot know, so it is deliberately not modelled here. `min_bid` and the + /// poison relay live outside the `Feature` enum (separate detectors). + pub fn armed_features(&self) -> Vec { + let mut f = Vec::new(); + if matches!(self.sigverify, Sigverify::Skip | Sigverify::SkipPoisoned) { + f.push(Feature::SkipSigverify); + } + if self.extra_validation { + f.push(Feature::ExtraValidation); + } + if self.timing_games { + f.push(Feature::TimingGames); + } + if matches!(self.get_header, HeaderTransport::Stream { .. }) { + f.push(Feature::WsHeaderStream); + } + f + } + + /// True when this spec sets a `min_bid_eth` floor (detected separately from + /// the `Feature` enum, via `detect_min_bid_eth`). + pub fn arms_min_bid(&self) -> bool { + matches!(self.min_bid, MinBid::Floor(_)) + } + + /// True when this spec injects the wrong-pubkey literal relay (detected via + /// `has_poisoned_relay_pubkey`). + pub fn arms_poison(&self) -> bool { + matches!( + self.sigverify, + Sigverify::SkipPoisoned | Sigverify::PoisonedControl + ) + } + + /// Render the full Kurtosis args-file, with `comment` as the leading block. + /// + /// The comment is a render-time parameter, NOT a spec field: it is + /// hand-written per-scenario prose with no knob preimage, so it is not part + /// of a scenario's structured identity. The 13 named scenarios pass their + /// verbatim `Scenario::comment()`; composed / AI specs pass `auto_comment()`. + /// + /// Total on every constructible spec EXCEPT mux combined with any CB-config + /// injection: `cb_toml_mux` is a structurally different template with no + /// `[pbs]`/per-relay/literal-relay seam, so those combinations cannot be + /// rendered and are rejected loudly rather than silently dropping the + /// injected config. `keys_dir` is read only for mux (the per-node pubkeys). + pub fn render(&self, comment: &str, images: &Images, keys_dir: &Path) -> Result { + let cb_block = if matches!(self.topology, Topology::Mux) { + eyre::ensure!( + !self.timing_games + && !self.extra_validation + && !self.signer + && matches!(self.sigverify, Sigverify::On) + && matches!(self.min_bid, MinBid::None) + && matches!(self.get_header, HeaderTransport::Http), + "mux uses a fixed CB TOML (per-node [[mux]] routing) with no injection seam; \ + it cannot compose with pbs/per-relay/literal-relay features" + ); + let node0 = load_pubkeys(keys_dir, 0)?; + let node1 = load_pubkeys(keys_dir, 1)?; + cb_toml_mux(&node0, &node1) + } else { + cb_toml(&self.to_cb_params()) + }; + + let mev_params = build_mev_params( + self.relays(), + images, + &cb_block, + self.builder_subsidy(), + self.signer, + ); + + Ok([ + comment.to_string(), + self.el_cl().participants(), + COMMON_ADDITIONAL_SERVICES.to_string(), + "mev_type: custom".to_string(), + mev_params, + self.network_params().to_string(), + ] + .join("\n\n") + + "\n") + } + + /// A generated comment for a composed / AI-authored spec (no verbatim prose + /// exists). One `#` header line naming the non-default knobs, so a rendered + /// config is self-describing without a hand-written block. + pub fn auto_comment(&self) -> String { + let mut knobs: Vec = Vec::new(); + if self.clients == ClientPair::NethermindPrysm { + knobs.push("nethermind-prysm".to_string()); + } + match self.topology { + Topology::Single => {} + Topology::TwoRelays => knobs.push("two-relays".to_string()), + Topology::DivergentRelays => knobs.push("divergent-relays".to_string()), + Topology::Mux => knobs.push("mux".to_string()), + } + match self.get_header { + HeaderTransport::Http => {} + HeaderTransport::Stream { + api_key: KeyPresence::Present, + } => knobs.push("ws-stream".to_string()), + HeaderTransport::Stream { + api_key: KeyPresence::Absent, + } => knobs.push("ws-stream-nokey".to_string()), + } + if self.timing_games { + knobs.push("timing-games".to_string()); + } + if self.extra_validation { + knobs.push("extra-validation".to_string()); + } + if self.signer { + knobs.push("signer".to_string()); + } + match self.sigverify { + Sigverify::On => {} + Sigverify::Skip => knobs.push("skip-sigverify".to_string()), + Sigverify::SkipPoisoned => knobs.push("skip-sigverify+poison".to_string()), + Sigverify::PoisonedControl => knobs.push("poison-control".to_string()), + } + if let MinBid::Floor(x) = self.min_bid { + knobs.push(format!("min-bid={x}")); + } + let body = if knobs.is_empty() { + "cb-basic".to_string() + } else { + knobs.join(", ") + }; + format!("# composed scenario: {body}") + } +} + +/// Curated coverage points worth freezing as regression anchors: high-value +/// composed scenarios and the additional CL clients. Each is a `ScenarioSpec` +/// (composed, not a `Scenario` enum variant) with a byte-golden under +/// `tests/fixtures/curated-configs/`, and each has been confirmed to stand up a +/// live devnet (Law 7 / the bench discipline: a golden of a config that has +/// never run is worthless). New entries land WITH a live confirmation. +pub fn curated() -> Vec<(&'static str, ScenarioSpec)> { + let basic_on = |clients: ClientPair| ScenarioSpec { + clients, + ..ScenarioSpec::default() + }; + vec![ + // The additional CL clients (basic MEV pipeline on each — Law 7). + ("cb-basic-teku", basic_on(ClientPair::GethTeku)), + ("cb-basic-nimbus", basic_on(ClientPair::GethNimbus)), + ("cb-basic-lodestar", basic_on(ClientPair::GethLodestar)), + // ws stream on the prysm pair — the exact Law-7 route-coupling concern + // (a prysm-specific ws regression is invisible under geth+lighthouse). + ( + "cb-ws-prysm", + ScenarioSpec { + clients: ClientPair::NethermindPrysm, + get_header: HeaderTransport::Stream { + api_key: KeyPresence::Present, + }, + ..ScenarioSpec::default() + }, + ), + // The composition anchor: two markers must both fire on one run. + ( + "cb-timing-extra-validation", + ScenarioSpec { + topology: Topology::TwoRelays, + timing_games: true, + extra_validation: true, + ..ScenarioSpec::default() + }, + ), + ] +} + +#[cfg(test)] +mod tests { + use std::path::Path; + + use cb_testnet_verifier::checks::feature_fired::{ + detect_enabled_features, detect_min_bid_eth, has_poisoned_relay_pubkey, + }; + + use super::*; + use crate::genmodel::scenario::Scenario; + + /// The keys dir the mux scenario reads (real repo fixture, as the golden + /// tests use). Non-mux specs ignore it. + fn keys() -> &'static Path { + Path::new("keys") + } + + /// Every `ClientPair` variant, for exhaustive coverage in the offline tests. + const ALL_CLIENT_PAIRS: [ClientPair; 5] = [ + ClientPair::GethLighthouse, + ClientPair::NethermindPrysm, + ClientPair::GethTeku, + ClientPair::GethNimbus, + ClientPair::GethLodestar, + ]; + + /// Each client pair renders its own `el_type`/`cl_type` into the participants + /// block (the parametric axis is real end to end — Law 7). Offline shape check; + /// standing the client up on a devnet is a separate live confirmation. + #[test] + fn every_client_pair_renders_its_el_cl() { + let images = Images::default(); + for c in ALL_CLIENT_PAIRS { + let spec = ScenarioSpec { + clients: c, + ..ScenarioSpec::default() + }; + let el = spec.el_cl(); + let out = spec.render("# x", &images, keys()).unwrap(); + assert!( + out.contains(&format!("el_type: {}", el.el)), + "missing el_type {} for {c:?}", + el.el + ); + assert!( + out.contains(&format!("cl_type: {}", el.cl)), + "missing cl_type {} for {c:?}", + el.cl + ); + } + } + + fn sorted_ids(mut fs: Vec) -> Vec<&'static str> { + fs.sort_by_key(|f| f.id()); + fs.dedup(); + fs.iter().map(|f| f.id()).collect() + } + + /// THE MIGRATION CONTRACT: the composable `render` path reproduces every + /// named scenario byte-for-byte against the existing (byte-golden'd) + /// `args_file_in`. If this passes, `render` inherits the goldens' coverage. + #[test] + fn lower_reproduces_every_scenario() { + let images = Images::default(); + for s in Scenario::ALL { + let via_spec = s.to_spec().render(s.comment(), &images, keys()).unwrap(); + let via_assembly = s.args_file_in(&images, keys()).unwrap(); + assert_eq!( + via_spec, + via_assembly, + "render(spec) != args_file_in for {}", + s.name() + ); + } + } + + /// Round-trip: what the config ARMS (per the spec) equals what the verifier's + /// own `detect_enabled_features` sees in the rendered config. This is a + /// RENDERER-DRIFT guard only — it proves emit and detect agree on the toggle + /// keys; it does NOT prove the config is valid CB (both sides share key + /// strings, so a shared typo passes — that is `sim preflight`'s job). + #[test] + fn armed_features_round_trip_over_the_named_set() { + let images = Images::default(); + for s in Scenario::ALL { + let spec = s.to_spec(); + let rendered = spec.render(s.comment(), &images, keys()).unwrap(); + assert_eq!( + sorted_ids(detect_enabled_features(&rendered)), + sorted_ids(spec.armed_features()), + "armed/detected feature mismatch for {}", + s.name() + ); + assert_eq!( + detect_min_bid_eth(&rendered).is_some(), + spec.arms_min_bid(), + "min_bid arm/detect mismatch for {}", + s.name() + ); + assert_eq!( + has_poisoned_relay_pubkey(&rendered), + spec.arms_poison(), + "poison arm/detect mismatch for {}", + s.name() + ); + } + } + + /// Pin the canonical fragment order for a COMPOSITE spec — the 13 goldens + /// each populate at most one `[pbs]` feature, so only a composite catches a + /// canonical-order regression (the silent byte-drift risk the grill flagged). + #[test] + fn composite_fragment_order_is_canonical() { + let images = Images::default(); + // All three [pbs] features + both per-relay features on one spec. + let spec = ScenarioSpec { + sigverify: Sigverify::Skip, + extra_validation: true, + min_bid: MinBid::Floor(0.5), + timing_games: true, + get_header: HeaderTransport::Stream { + api_key: KeyPresence::Present, + }, + topology: Topology::TwoRelays, + ..ScenarioSpec::default() + }; + let out = spec.render("# composite", &images, keys()).unwrap(); + let at = |needle: &str| { + out.find(needle) + .unwrap_or_else(|| panic!("missing {needle}")) + }; + // [pbs] order: skip_sigverify < extra_validation_enabled < min_bid_eth + assert!(at("skip_sigverify = true") < at("extra_validation_enabled = true")); + assert!(at("extra_validation_enabled = true") < at("min_bid_eth = 0.5")); + // per-relay order: enable_timing_games < get_header = "stream" + assert!(at("enable_timing_games = true") < at(r#"get_header = "stream""#)); + } + + /// A representative slice of the composable space. Mux is exclusive, so it is + /// enumerated alone; every other axis combines with the non-mux base. + fn enumerate() -> Vec { + let mut out = Vec::new(); + let clients = ALL_CLIENT_PAIRS; + let transports = [ + HeaderTransport::Http, + HeaderTransport::Stream { + api_key: KeyPresence::Present, + }, + HeaderTransport::Stream { + api_key: KeyPresence::Absent, + }, + ]; + let sigverifies = [ + Sigverify::On, + Sigverify::Skip, + Sigverify::SkipPoisoned, + Sigverify::PoisonedControl, + ]; + let topos = [ + Topology::Single, + Topology::TwoRelays, + Topology::DivergentRelays, + ]; + for &c in &clients { + for &t in &transports { + for &sv in &sigverifies { + for &topo in &topos { + for tg in [false, true] { + for ev in [false, true] { + for mb in [MinBid::None, MinBid::Floor(0.5)] { + out.push(ScenarioSpec { + clients: c, + topology: topo, + get_header: t, + timing_games: tg, + extra_validation: ev, + signer: false, + sigverify: sv, + min_bid: mb, + }); + } + } + } + } + } + } + } + // Mux alone (exclusive): each client pair. + for &c in &clients { + out.push(ScenarioSpec { + clients: c, + topology: Topology::Mux, + ..ScenarioSpec::default() + }); + } + out + } + + /// `render` is total across the non-mux composable space (no panic, always + /// Ok), and the round-trip holds for every enumerated point — the offline + /// "enumerable/testable" deliverable (a test, not a coverage-claiming verb). + #[test] + fn every_enumerated_spec_renders_and_round_trips() { + let images = Images::default(); + for spec in enumerate() { + let rendered = spec + .render(&spec.auto_comment(), &images, keys()) + .unwrap_or_else(|e| panic!("render failed for {spec:?}: {e}")); + assert_eq!( + sorted_ids(detect_enabled_features(&rendered)), + sorted_ids(spec.armed_features()), + "round-trip mismatch for {spec:?}" + ); + } + } + + #[test] + fn from_json_is_a_delta_off_the_default_and_rejects_unknown_fields() { + // Partial JSON: only the named fields move; the rest default to cb-basic. + let spec = ScenarioSpec::from_json(r#"{"clients":"nethermind-prysm","timing_games":true}"#) + .unwrap(); + assert_eq!(spec.clients, ClientPair::NethermindPrysm); + assert!(spec.timing_games); + assert_eq!(spec.topology, Topology::Single); // defaulted + // An invented key is a hard error (deny_unknown_fields), not silently dropped. + assert!(ScenarioSpec::from_json(r#"{"turbo_mode":true}"#).is_err()); + } + + #[test] + fn from_base_and_overrides_composes_onto_a_named_base() { + // No base + no set == default (cb-basic). + assert_eq!( + ScenarioSpec::from_base_and_overrides(None, None).unwrap(), + ScenarioSpec::default() + ); + // A named base resolves to that scenario's spec. + assert_eq!( + ScenarioSpec::from_base_and_overrides(Some("cb-mux"), None).unwrap(), + Scenario::Mux.to_spec() + ); + // Overrides apply onto the base. + let spec = ScenarioSpec::from_base_and_overrides( + Some("cb-basic"), + Some("get_header=stream,clients=nethermind-prysm,timing_games=true"), + ) + .unwrap(); + assert_eq!( + spec.get_header, + HeaderTransport::Stream { + api_key: KeyPresence::Present + } + ); + assert_eq!(spec.clients, ClientPair::NethermindPrysm); + assert!(spec.timing_games); + } + + #[test] + fn override_errors_are_loud() { + let mut s = ScenarioSpec::default(); + assert!(s.apply_override("nonsense", "x").is_err()); // unknown key + assert!(s.apply_override("clients", "solana").is_err()); // bad enum value + assert!(s.apply_override("timing_games", "yes").is_err()); // bad bool + assert!(s.apply_override("min_bid", "abc").is_err()); // bad float + assert!(ScenarioSpec::from_base_and_overrides(Some("no-such-base"), None).is_err()); + } + + /// The curated coverage points render stably against their committed golden. + /// Regenerate the goldens with `BLESS_CURATED=1 cargo test --bin sim + /// every_curated_spec_matches_its_golden` (only after a live devnet run + /// confirms each config actually works). + #[test] + fn every_curated_spec_matches_its_golden() { + let images = Images::default(); + let dir = "tests/fixtures/curated-configs"; + for (name, spec) in curated() { + let rendered = spec.render(&spec.auto_comment(), &images, keys()).unwrap(); + let path = format!("{dir}/{name}.yml"); + if std::env::var("BLESS_CURATED").is_ok() { + std::fs::create_dir_all(dir).unwrap(); + std::fs::write(&path, &rendered).unwrap(); + } else { + let golden = std::fs::read_to_string(&path).unwrap_or_else(|_| { + panic!("missing curated golden {path}; run BLESS_CURATED=1 to create it") + }); + assert_eq!( + rendered, golden, + "curated config {name} drifted from its golden" + ); + } + } + } + + /// Mux composed with any injection feature is rejected loudly (not a silent + /// wrong config) — the one genuinely unrenderable family. + #[test] + fn mux_with_injection_is_rejected() { + let images = Images::default(); + let bad = ScenarioSpec { + topology: Topology::Mux, + timing_games: true, + ..ScenarioSpec::default() + }; + assert!(bad.render("# x", &images, keys()).is_err()); + } +} diff --git a/src/bin/sim/main.rs b/src/bin/sim/main.rs index 724c365..1f38a4f 100644 --- a/src/bin/sim/main.rs +++ b/src/bin/sim/main.rs @@ -7,9 +7,12 @@ //! Sync only: the verbs shell `kurtosis`/`docker` with `std::process::Command`, //! matching `discovery.rs`. No tokio. -use std::path::Path; +use std::path::{Path, PathBuf}; use clap::Parser; +use eyre::WrapErr; + +use genmodel::spec::ScenarioSpec; mod checks_catalog; mod cli; @@ -38,17 +41,82 @@ fn main() { scenario, out_dir, check, - } => generate(scenario.as_deref(), &out_dir, check), + curated, + } => generate(scenario.as_deref(), &out_dir, check, curated), + Command::Scenario { + spec, + base, + set, + out, + show_spec, + } => scenario_cmd(spec, base, set, out, show_spec), + } +} + +/// Render a composable scenario from a `ScenarioSpec` (`--spec `) or a +/// named base with typed overrides (`--base`/`--set`). Implemented via +/// `ScenarioSpec::{from_json, from_base_and_overrides}` + `render`. +fn scenario_cmd( + spec_path: Option, + base: Option, + set: Option, + out: Option, + show_spec: bool, +) { + let result = (|| -> eyre::Result<()> { + let spec = match &spec_path { + Some(path) => { + eyre::ensure!( + base.is_none() && set.is_none(), + "--spec is mutually exclusive with --base/--set" + ); + let json = std::fs::read_to_string(path) + .wrap_err_with(|| format!("reading {}", path.display()))?; + ScenarioSpec::from_json(&json)? + } + None => ScenarioSpec::from_base_and_overrides(base.as_deref(), set.as_deref())?, + }; + if show_spec { + eprintln!("{}", serde_json::to_string_pretty(&spec)?); + let mut arms: Vec = spec + .armed_features() + .iter() + .map(|f| f.id().to_string()) + .collect(); + if spec.arms_min_bid() { + arms.push("feature.min_bid".to_string()); + } + if spec.arms_poison() { + arms.push("poison_relay".to_string()); + } + eprintln!("arms: [{}]", arms.join(", ")); + } + let images = generate::images_from_env(Path::new(".env")); + let rendered = spec.render(&spec.auto_comment(), &images, Path::new("keys"))?; + match &out { + Some(path) => { + std::fs::write(path, &rendered) + .wrap_err_with(|| format!("writing {}", path.display()))?; + println!("Rendered {}", path.display()); + } + None => print!("{rendered}"), + } + Ok(()) + })(); + if let Err(e) = result { + tracing::error!(error = %e, "sim scenario failed"); + eprintln!("scenario error: {e:?}"); + std::process::exit(1); } } /// Generate Kurtosis args-files (Task 1), or `--check` them (P2 drift gate). /// Implemented in `generate::run` / `generate::check`. -fn generate(scenario: Option<&str>, out_dir: &Path, check: bool) { +fn generate(scenario: Option<&str>, out_dir: &Path, check: bool, curated: bool) { let result = if check { - generate::check(scenario, out_dir) + generate::check(scenario, out_dir, curated) } else { - generate::run(scenario, out_dir) + generate::run(scenario, out_dir, curated) }; if let Err(e) = result { tracing::error!(error = %e, "sim generate failed"); diff --git a/src/checks/feature_fired.rs b/src/checks/feature_fired.rs index 6d8a7c2..208a8d3 100644 --- a/src/checks/feature_fired.rs +++ b/src/checks/feature_fired.rs @@ -711,7 +711,10 @@ mod tests { // check owns the inconclusive flag, this must not double-flag let r = classify_ws_fallback(0, 64); assert_eq!(r.status, CheckStatus::Warn); - assert!(!r.inconclusive, "double-flagging would make one failure two"); + assert!( + !r.inconclusive, + "double-flagging would make one failure two" + ); } // Contract: the marker strings match CB main's actual log lines (pinned diff --git a/tests/fixtures/curated-configs/cb-basic-lodestar.yml b/tests/fixtures/curated-configs/cb-basic-lodestar.yml new file mode 100644 index 0000000..c4e1366 --- /dev/null +++ b/tests/fixtures/curated-configs/cb-basic-lodestar.yml @@ -0,0 +1,160 @@ +# composed scenario: cb-basic + +participants: + - el_type: geth + cl_type: lodestar + +additional_services: + - dora + - spamoor + - prometheus + +mev_type: custom + +mev_params: + mev_relay: helix + mev_sidecar: commit-boost + mev_builder: flashbots + + helix_relay_image: ghcr.io/gattaca-com/helix-relay:main + mev_boost_image: commit-boost/commit-boost:kurtosis + mev_builder_image: ethpandaops/reth-rbuilder:develop + mev_builder_cl_image: sigp/lighthouse:latest + + mev_builder_subsidy: 1 + + helix_relay_config: | + instance_id: "helix-kurtosis-test" + + # NOTE: no network/genesis section. Current helix-relay:main removed the old + # `network_config: !Custom {dir_path, genesis_validator_root, genesis_time}` + # field entirely; the relay now fetches the chain spec + genesis from the + # beacon node at startup (GET eth/v1/config/spec + eth/v1/beacon/genesis, see + # beacon_client.get_chain_info -> main.rs load_chain_info). A `!Custom` YAML + # tag on that now-unknown key makes serde_yaml panic with + # "untagged and internally tagged enums do not support enum input". + + postgres: + hostname: "{{ .POSTGRES_HOST_NAME }}" + port: {{ .POSTGRES_PORT }} + db_name: "{{ .POSTGRES_DB }}" + user: "{{ .POSTGRES_USER }}" + password: "{{ .POSTGRES_PASS }}" + region: 0 + region_name: "LOCAL" + + beacon_clients: + - url: "{{ .BEACON_URI }}" + + gossip_payload_on_header: false + + simulators: + - url: "{{ .BLOCKSIM_URI }}" + namespace: flashbots + is_merging_simulator: false + max_concurrent_tasks: 32 + + router_config: + enabled_routes: + - route: GetValidators + - route: SubmitBlock + - route: GetTopBid + - route: GetHeader + rate_limit: + replenish_ms: 50 + burst_size: 20 + - route: GetPayload + # The builder-spec v2 proposer route (submitBlindedBlockV2). REQUIRED for + # any CL that submits via v2 -- prysm does. Without it helix 404s + # /eth/v2/builder/blinded_blocks, CB (correctly) refuses to downgrade to v1 + # (v2 semantics: the relay publishes the block, so a v1 payload would be + # silently dropped), returns 502, and EVERY builder block the proposer + # chose is lost. That read as "prysm can't do MEV" until the route list was + # checked (Law 7 first dividend). + - route: GetPayloadV2 + - route: HeaderStream + - route: RegisterValidators + - route: Status + - route: ProposerPayloadDelivered + - route: BuilderBidsReceived + - route: ValidatorRegistration + shutdown_delay_ms: 12000 + + timing_game_config: + max_header_delay_ms: 400 + latest_header_delay_ms_in_slot: 1500 + default_client_latency_ms: 50 + + target_get_payload_propagation_duration_ms: 500 + + is_submission_instance: true + is_registration_instance: true + + admin_token: "test_admin_token" + + logging: + type: Console + + # CoresConfig (helix_common::config CoresConfig, 10 fields in current + # :main). The old block used `sub_workers: [0]` and omitted the per-tile + # core assignments; current :main removed sub_workers and added + # decoder/simulator/top_bid/data_gatherer/block_merging/housekeeper. The + # outer RelayConfigExt flattens RelayConfig, so a wrong/missing cores field + # surfaces as a top-level "missing field `decoder`" / "invalid type ... + # expected usize|sequence" serde error. Verified against the binary: + # `decoder` is Vec ([0]); the other five new tile fields are usize. + cores: + auctioneer: 0 + tokio: [0] + reg_workers: [0] + tcp_bid_submissions_tile: 2 + decoder: [0] + simulator: 0 + top_bid: 0 + data_gatherer: 0 + block_merging: 0 + housekeeper: 0 + + is_local_dev: false + + commit_boost_config: | + chain = { genesis_time_secs = {{ .Timestamp }}, path = "{{ .Network }}" } + + [pbs] + host = "0.0.0.0" + port = {{ .Port }} + timeout_get_header_ms = 950 + timeout_get_payload_ms = 4000 + late_in_slot_time_ms = 2000 + + + [metrics] + enabled = true + host = "0.0.0.0" + start_port = 9090 + + {{ range $index, $relay := .Relays }} + [[relays]] + id = "mev_relay_{{$index}}" + url = "{{ $relay }}" + {{- end }} + + [logs.stdout] + level = "debug" + + [logs.file] + enabled = false + +network_params: + network: kurtosis + network_id: "3151908" + deposit_contract_address: "0x00000000219ab540356cBB839Cbe05303d7705Fa" + seconds_per_slot: 12 + slot_duration_ms: 12000 + num_validator_keys_per_node: 128 + preregistered_validator_keys_mnemonic: + "giant issue aisle success illegal bike spike + question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy + very lucky have athlete" + prefunded_accounts: '{"0xb9e79d19f651a941757b35830232E7EFC77E1c79": {"balance": "100000ETH"}}' + diff --git a/tests/fixtures/curated-configs/cb-basic-nimbus.yml b/tests/fixtures/curated-configs/cb-basic-nimbus.yml new file mode 100644 index 0000000..d88a852 --- /dev/null +++ b/tests/fixtures/curated-configs/cb-basic-nimbus.yml @@ -0,0 +1,160 @@ +# composed scenario: cb-basic + +participants: + - el_type: geth + cl_type: nimbus + +additional_services: + - dora + - spamoor + - prometheus + +mev_type: custom + +mev_params: + mev_relay: helix + mev_sidecar: commit-boost + mev_builder: flashbots + + helix_relay_image: ghcr.io/gattaca-com/helix-relay:main + mev_boost_image: commit-boost/commit-boost:kurtosis + mev_builder_image: ethpandaops/reth-rbuilder:develop + mev_builder_cl_image: sigp/lighthouse:latest + + mev_builder_subsidy: 1 + + helix_relay_config: | + instance_id: "helix-kurtosis-test" + + # NOTE: no network/genesis section. Current helix-relay:main removed the old + # `network_config: !Custom {dir_path, genesis_validator_root, genesis_time}` + # field entirely; the relay now fetches the chain spec + genesis from the + # beacon node at startup (GET eth/v1/config/spec + eth/v1/beacon/genesis, see + # beacon_client.get_chain_info -> main.rs load_chain_info). A `!Custom` YAML + # tag on that now-unknown key makes serde_yaml panic with + # "untagged and internally tagged enums do not support enum input". + + postgres: + hostname: "{{ .POSTGRES_HOST_NAME }}" + port: {{ .POSTGRES_PORT }} + db_name: "{{ .POSTGRES_DB }}" + user: "{{ .POSTGRES_USER }}" + password: "{{ .POSTGRES_PASS }}" + region: 0 + region_name: "LOCAL" + + beacon_clients: + - url: "{{ .BEACON_URI }}" + + gossip_payload_on_header: false + + simulators: + - url: "{{ .BLOCKSIM_URI }}" + namespace: flashbots + is_merging_simulator: false + max_concurrent_tasks: 32 + + router_config: + enabled_routes: + - route: GetValidators + - route: SubmitBlock + - route: GetTopBid + - route: GetHeader + rate_limit: + replenish_ms: 50 + burst_size: 20 + - route: GetPayload + # The builder-spec v2 proposer route (submitBlindedBlockV2). REQUIRED for + # any CL that submits via v2 -- prysm does. Without it helix 404s + # /eth/v2/builder/blinded_blocks, CB (correctly) refuses to downgrade to v1 + # (v2 semantics: the relay publishes the block, so a v1 payload would be + # silently dropped), returns 502, and EVERY builder block the proposer + # chose is lost. That read as "prysm can't do MEV" until the route list was + # checked (Law 7 first dividend). + - route: GetPayloadV2 + - route: HeaderStream + - route: RegisterValidators + - route: Status + - route: ProposerPayloadDelivered + - route: BuilderBidsReceived + - route: ValidatorRegistration + shutdown_delay_ms: 12000 + + timing_game_config: + max_header_delay_ms: 400 + latest_header_delay_ms_in_slot: 1500 + default_client_latency_ms: 50 + + target_get_payload_propagation_duration_ms: 500 + + is_submission_instance: true + is_registration_instance: true + + admin_token: "test_admin_token" + + logging: + type: Console + + # CoresConfig (helix_common::config CoresConfig, 10 fields in current + # :main). The old block used `sub_workers: [0]` and omitted the per-tile + # core assignments; current :main removed sub_workers and added + # decoder/simulator/top_bid/data_gatherer/block_merging/housekeeper. The + # outer RelayConfigExt flattens RelayConfig, so a wrong/missing cores field + # surfaces as a top-level "missing field `decoder`" / "invalid type ... + # expected usize|sequence" serde error. Verified against the binary: + # `decoder` is Vec ([0]); the other five new tile fields are usize. + cores: + auctioneer: 0 + tokio: [0] + reg_workers: [0] + tcp_bid_submissions_tile: 2 + decoder: [0] + simulator: 0 + top_bid: 0 + data_gatherer: 0 + block_merging: 0 + housekeeper: 0 + + is_local_dev: false + + commit_boost_config: | + chain = { genesis_time_secs = {{ .Timestamp }}, path = "{{ .Network }}" } + + [pbs] + host = "0.0.0.0" + port = {{ .Port }} + timeout_get_header_ms = 950 + timeout_get_payload_ms = 4000 + late_in_slot_time_ms = 2000 + + + [metrics] + enabled = true + host = "0.0.0.0" + start_port = 9090 + + {{ range $index, $relay := .Relays }} + [[relays]] + id = "mev_relay_{{$index}}" + url = "{{ $relay }}" + {{- end }} + + [logs.stdout] + level = "debug" + + [logs.file] + enabled = false + +network_params: + network: kurtosis + network_id: "3151908" + deposit_contract_address: "0x00000000219ab540356cBB839Cbe05303d7705Fa" + seconds_per_slot: 12 + slot_duration_ms: 12000 + num_validator_keys_per_node: 128 + preregistered_validator_keys_mnemonic: + "giant issue aisle success illegal bike spike + question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy + very lucky have athlete" + prefunded_accounts: '{"0xb9e79d19f651a941757b35830232E7EFC77E1c79": {"balance": "100000ETH"}}' + diff --git a/tests/fixtures/curated-configs/cb-basic-teku.yml b/tests/fixtures/curated-configs/cb-basic-teku.yml new file mode 100644 index 0000000..de87ef9 --- /dev/null +++ b/tests/fixtures/curated-configs/cb-basic-teku.yml @@ -0,0 +1,160 @@ +# composed scenario: cb-basic + +participants: + - el_type: geth + cl_type: teku + +additional_services: + - dora + - spamoor + - prometheus + +mev_type: custom + +mev_params: + mev_relay: helix + mev_sidecar: commit-boost + mev_builder: flashbots + + helix_relay_image: ghcr.io/gattaca-com/helix-relay:main + mev_boost_image: commit-boost/commit-boost:kurtosis + mev_builder_image: ethpandaops/reth-rbuilder:develop + mev_builder_cl_image: sigp/lighthouse:latest + + mev_builder_subsidy: 1 + + helix_relay_config: | + instance_id: "helix-kurtosis-test" + + # NOTE: no network/genesis section. Current helix-relay:main removed the old + # `network_config: !Custom {dir_path, genesis_validator_root, genesis_time}` + # field entirely; the relay now fetches the chain spec + genesis from the + # beacon node at startup (GET eth/v1/config/spec + eth/v1/beacon/genesis, see + # beacon_client.get_chain_info -> main.rs load_chain_info). A `!Custom` YAML + # tag on that now-unknown key makes serde_yaml panic with + # "untagged and internally tagged enums do not support enum input". + + postgres: + hostname: "{{ .POSTGRES_HOST_NAME }}" + port: {{ .POSTGRES_PORT }} + db_name: "{{ .POSTGRES_DB }}" + user: "{{ .POSTGRES_USER }}" + password: "{{ .POSTGRES_PASS }}" + region: 0 + region_name: "LOCAL" + + beacon_clients: + - url: "{{ .BEACON_URI }}" + + gossip_payload_on_header: false + + simulators: + - url: "{{ .BLOCKSIM_URI }}" + namespace: flashbots + is_merging_simulator: false + max_concurrent_tasks: 32 + + router_config: + enabled_routes: + - route: GetValidators + - route: SubmitBlock + - route: GetTopBid + - route: GetHeader + rate_limit: + replenish_ms: 50 + burst_size: 20 + - route: GetPayload + # The builder-spec v2 proposer route (submitBlindedBlockV2). REQUIRED for + # any CL that submits via v2 -- prysm does. Without it helix 404s + # /eth/v2/builder/blinded_blocks, CB (correctly) refuses to downgrade to v1 + # (v2 semantics: the relay publishes the block, so a v1 payload would be + # silently dropped), returns 502, and EVERY builder block the proposer + # chose is lost. That read as "prysm can't do MEV" until the route list was + # checked (Law 7 first dividend). + - route: GetPayloadV2 + - route: HeaderStream + - route: RegisterValidators + - route: Status + - route: ProposerPayloadDelivered + - route: BuilderBidsReceived + - route: ValidatorRegistration + shutdown_delay_ms: 12000 + + timing_game_config: + max_header_delay_ms: 400 + latest_header_delay_ms_in_slot: 1500 + default_client_latency_ms: 50 + + target_get_payload_propagation_duration_ms: 500 + + is_submission_instance: true + is_registration_instance: true + + admin_token: "test_admin_token" + + logging: + type: Console + + # CoresConfig (helix_common::config CoresConfig, 10 fields in current + # :main). The old block used `sub_workers: [0]` and omitted the per-tile + # core assignments; current :main removed sub_workers and added + # decoder/simulator/top_bid/data_gatherer/block_merging/housekeeper. The + # outer RelayConfigExt flattens RelayConfig, so a wrong/missing cores field + # surfaces as a top-level "missing field `decoder`" / "invalid type ... + # expected usize|sequence" serde error. Verified against the binary: + # `decoder` is Vec ([0]); the other five new tile fields are usize. + cores: + auctioneer: 0 + tokio: [0] + reg_workers: [0] + tcp_bid_submissions_tile: 2 + decoder: [0] + simulator: 0 + top_bid: 0 + data_gatherer: 0 + block_merging: 0 + housekeeper: 0 + + is_local_dev: false + + commit_boost_config: | + chain = { genesis_time_secs = {{ .Timestamp }}, path = "{{ .Network }}" } + + [pbs] + host = "0.0.0.0" + port = {{ .Port }} + timeout_get_header_ms = 950 + timeout_get_payload_ms = 4000 + late_in_slot_time_ms = 2000 + + + [metrics] + enabled = true + host = "0.0.0.0" + start_port = 9090 + + {{ range $index, $relay := .Relays }} + [[relays]] + id = "mev_relay_{{$index}}" + url = "{{ $relay }}" + {{- end }} + + [logs.stdout] + level = "debug" + + [logs.file] + enabled = false + +network_params: + network: kurtosis + network_id: "3151908" + deposit_contract_address: "0x00000000219ab540356cBB839Cbe05303d7705Fa" + seconds_per_slot: 12 + slot_duration_ms: 12000 + num_validator_keys_per_node: 128 + preregistered_validator_keys_mnemonic: + "giant issue aisle success illegal bike spike + question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy + very lucky have athlete" + prefunded_accounts: '{"0xb9e79d19f651a941757b35830232E7EFC77E1c79": {"balance": "100000ETH"}}' + diff --git a/tests/fixtures/curated-configs/cb-timing-extra-validation.yml b/tests/fixtures/curated-configs/cb-timing-extra-validation.yml new file mode 100644 index 0000000..8f9e438 --- /dev/null +++ b/tests/fixtures/curated-configs/cb-timing-extra-validation.yml @@ -0,0 +1,168 @@ +# composed scenario: two-relays, timing-games, extra-validation + +participants: + - el_type: geth + cl_type: lighthouse + +additional_services: + - dora + - spamoor + - prometheus + +mev_type: custom + +mev_params: + mev_relay: + - helix + - helix + mev_sidecar: commit-boost + mev_builder: flashbots + + helix_relay_image: ghcr.io/gattaca-com/helix-relay:main + mev_relay_image: ethpandaops/mev-boost-relay:main + mev_boost_image: commit-boost/commit-boost:kurtosis + mev_builder_image: ethpandaops/reth-rbuilder:develop + mev_builder_cl_image: sigp/lighthouse:latest + + mev_builder_subsidy: 1 + + helix_relay_config: | + instance_id: "helix-kurtosis-test" + + # NOTE: no network/genesis section. Current helix-relay:main removed the old + # `network_config: !Custom {dir_path, genesis_validator_root, genesis_time}` + # field entirely; the relay now fetches the chain spec + genesis from the + # beacon node at startup (GET eth/v1/config/spec + eth/v1/beacon/genesis, see + # beacon_client.get_chain_info -> main.rs load_chain_info). A `!Custom` YAML + # tag on that now-unknown key makes serde_yaml panic with + # "untagged and internally tagged enums do not support enum input". + + postgres: + hostname: "{{ .POSTGRES_HOST_NAME }}" + port: {{ .POSTGRES_PORT }} + db_name: "{{ .POSTGRES_DB }}" + user: "{{ .POSTGRES_USER }}" + password: "{{ .POSTGRES_PASS }}" + region: 0 + region_name: "LOCAL" + + beacon_clients: + - url: "{{ .BEACON_URI }}" + + gossip_payload_on_header: false + + simulators: + - url: "{{ .BLOCKSIM_URI }}" + namespace: flashbots + is_merging_simulator: false + max_concurrent_tasks: 32 + + router_config: + enabled_routes: + - route: GetValidators + - route: SubmitBlock + - route: GetTopBid + - route: GetHeader + rate_limit: + replenish_ms: 50 + burst_size: 20 + - route: GetPayload + # The builder-spec v2 proposer route (submitBlindedBlockV2). REQUIRED for + # any CL that submits via v2 -- prysm does. Without it helix 404s + # /eth/v2/builder/blinded_blocks, CB (correctly) refuses to downgrade to v1 + # (v2 semantics: the relay publishes the block, so a v1 payload would be + # silently dropped), returns 502, and EVERY builder block the proposer + # chose is lost. That read as "prysm can't do MEV" until the route list was + # checked (Law 7 first dividend). + - route: GetPayloadV2 + - route: HeaderStream + - route: RegisterValidators + - route: Status + - route: ProposerPayloadDelivered + - route: BuilderBidsReceived + - route: ValidatorRegistration + shutdown_delay_ms: 12000 + + timing_game_config: + max_header_delay_ms: 400 + latest_header_delay_ms_in_slot: 1500 + default_client_latency_ms: 50 + + target_get_payload_propagation_duration_ms: 500 + + is_submission_instance: true + is_registration_instance: true + + admin_token: "test_admin_token" + + logging: + type: Console + + # CoresConfig (helix_common::config CoresConfig, 10 fields in current + # :main). The old block used `sub_workers: [0]` and omitted the per-tile + # core assignments; current :main removed sub_workers and added + # decoder/simulator/top_bid/data_gatherer/block_merging/housekeeper. The + # outer RelayConfigExt flattens RelayConfig, so a wrong/missing cores field + # surfaces as a top-level "missing field `decoder`" / "invalid type ... + # expected usize|sequence" serde error. Verified against the binary: + # `decoder` is Vec ([0]); the other five new tile fields are usize. + cores: + auctioneer: 0 + tokio: [0] + reg_workers: [0] + tcp_bid_submissions_tile: 2 + decoder: [0] + simulator: 0 + top_bid: 0 + data_gatherer: 0 + block_merging: 0 + housekeeper: 0 + + is_local_dev: false + + commit_boost_config: | + chain = { genesis_time_secs = {{ .Timestamp }}, path = "{{ .Network }}" } + + [pbs] + host = "0.0.0.0" + port = {{ .Port }} + extra_validation_enabled = true + rpc_url = "http://el-1-geth-lighthouse:8545" + timeout_get_header_ms = 400 + timeout_get_payload_ms = 2000 + late_in_slot_time_ms = 2000 + + + [metrics] + enabled = true + host = "0.0.0.0" + start_port = 9090 + + {{ range $index, $relay := .Relays }} + [[relays]] + id = "mev_relay_{{$index}}" + url = "{{ $relay }}" + enable_timing_games = true + target_first_request_ms = 100 + frequency_get_header_ms = 200 + {{- end }} + + [logs.stdout] + level = "debug" + + [logs.file] + enabled = false + +network_params: + network: kurtosis + network_id: "3151908" + deposit_contract_address: "0x00000000219ab540356cBB839Cbe05303d7705Fa" + seconds_per_slot: 12 + slot_duration_ms: 12000 + num_validator_keys_per_node: 128 + preregistered_validator_keys_mnemonic: + "giant issue aisle success illegal bike spike + question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy + very lucky have athlete" + prefunded_accounts: '{"0xb9e79d19f651a941757b35830232E7EFC77E1c79": {"balance": "100000ETH"}}' + diff --git a/tests/fixtures/curated-configs/cb-ws-prysm.yml b/tests/fixtures/curated-configs/cb-ws-prysm.yml new file mode 100644 index 0000000..f547730 --- /dev/null +++ b/tests/fixtures/curated-configs/cb-ws-prysm.yml @@ -0,0 +1,162 @@ +# composed scenario: nethermind-prysm, ws-stream + +participants: + - el_type: nethermind + cl_type: prysm + +additional_services: + - dora + - spamoor + - prometheus + +mev_type: custom + +mev_params: + mev_relay: helix + mev_sidecar: commit-boost + mev_builder: flashbots + + helix_relay_image: ghcr.io/gattaca-com/helix-relay:main + mev_boost_image: commit-boost/commit-boost:kurtosis + mev_builder_image: ethpandaops/reth-rbuilder:develop + mev_builder_cl_image: sigp/lighthouse:latest + + mev_builder_subsidy: 1 + + helix_relay_config: | + instance_id: "helix-kurtosis-test" + + # NOTE: no network/genesis section. Current helix-relay:main removed the old + # `network_config: !Custom {dir_path, genesis_validator_root, genesis_time}` + # field entirely; the relay now fetches the chain spec + genesis from the + # beacon node at startup (GET eth/v1/config/spec + eth/v1/beacon/genesis, see + # beacon_client.get_chain_info -> main.rs load_chain_info). A `!Custom` YAML + # tag on that now-unknown key makes serde_yaml panic with + # "untagged and internally tagged enums do not support enum input". + + postgres: + hostname: "{{ .POSTGRES_HOST_NAME }}" + port: {{ .POSTGRES_PORT }} + db_name: "{{ .POSTGRES_DB }}" + user: "{{ .POSTGRES_USER }}" + password: "{{ .POSTGRES_PASS }}" + region: 0 + region_name: "LOCAL" + + beacon_clients: + - url: "{{ .BEACON_URI }}" + + gossip_payload_on_header: false + + simulators: + - url: "{{ .BLOCKSIM_URI }}" + namespace: flashbots + is_merging_simulator: false + max_concurrent_tasks: 32 + + router_config: + enabled_routes: + - route: GetValidators + - route: SubmitBlock + - route: GetTopBid + - route: GetHeader + rate_limit: + replenish_ms: 50 + burst_size: 20 + - route: GetPayload + # The builder-spec v2 proposer route (submitBlindedBlockV2). REQUIRED for + # any CL that submits via v2 -- prysm does. Without it helix 404s + # /eth/v2/builder/blinded_blocks, CB (correctly) refuses to downgrade to v1 + # (v2 semantics: the relay publishes the block, so a v1 payload would be + # silently dropped), returns 502, and EVERY builder block the proposer + # chose is lost. That read as "prysm can't do MEV" until the route list was + # checked (Law 7 first dividend). + - route: GetPayloadV2 + - route: HeaderStream + - route: RegisterValidators + - route: Status + - route: ProposerPayloadDelivered + - route: BuilderBidsReceived + - route: ValidatorRegistration + shutdown_delay_ms: 12000 + + timing_game_config: + max_header_delay_ms: 400 + latest_header_delay_ms_in_slot: 1500 + default_client_latency_ms: 50 + + target_get_payload_propagation_duration_ms: 500 + + is_submission_instance: true + is_registration_instance: true + + admin_token: "test_admin_token" + + logging: + type: Console + + # CoresConfig (helix_common::config CoresConfig, 10 fields in current + # :main). The old block used `sub_workers: [0]` and omitted the per-tile + # core assignments; current :main removed sub_workers and added + # decoder/simulator/top_bid/data_gatherer/block_merging/housekeeper. The + # outer RelayConfigExt flattens RelayConfig, so a wrong/missing cores field + # surfaces as a top-level "missing field `decoder`" / "invalid type ... + # expected usize|sequence" serde error. Verified against the binary: + # `decoder` is Vec ([0]); the other five new tile fields are usize. + cores: + auctioneer: 0 + tokio: [0] + reg_workers: [0] + tcp_bid_submissions_tile: 2 + decoder: [0] + simulator: 0 + top_bid: 0 + data_gatherer: 0 + block_merging: 0 + housekeeper: 0 + + is_local_dev: false + + commit_boost_config: | + chain = { genesis_time_secs = {{ .Timestamp }}, path = "{{ .Network }}" } + + [pbs] + host = "0.0.0.0" + port = {{ .Port }} + timeout_get_header_ms = 950 + timeout_get_payload_ms = 4000 + late_in_slot_time_ms = 2000 + + + [metrics] + enabled = true + host = "0.0.0.0" + start_port = 9090 + + {{ range $index, $relay := .Relays }} + [[relays]] + id = "mev_relay_{{$index}}" + url = "{{ $relay }}" + get_header = "stream" + headers = { X-Api-Key = "9d5c2f4e-1b7a-4c3d-8e6f-0a1b2c3d4e5f" } + {{- end }} + + [logs.stdout] + level = "debug" + + [logs.file] + enabled = false + +network_params: + network: kurtosis + network_id: "3151908" + deposit_contract_address: "0x00000000219ab540356cBB839Cbe05303d7705Fa" + seconds_per_slot: 12 + slot_duration_ms: 12000 + num_validator_keys_per_node: 128 + preregistered_validator_keys_mnemonic: + "giant issue aisle success illegal bike spike + question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy + very lucky have athlete" + prefunded_accounts: '{"0xb9e79d19f651a941757b35830232E7EFC77E1c79": {"balance": "100000ETH"}}' + diff --git a/tests/fixtures/golden-configs/cb-basic-nethermind-prysm.yml b/tests/fixtures/golden-configs/cb-basic-nethermind-prysm.yml index 68a1fa2..01625ad 100644 --- a/tests/fixtures/golden-configs/cb-basic-nethermind-prysm.yml +++ b/tests/fixtures/golden-configs/cb-basic-nethermind-prysm.yml @@ -74,7 +74,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-basic.yml b/tests/fixtures/golden-configs/cb-basic.yml index b5ddd33..e673827 100644 --- a/tests/fixtures/golden-configs/cb-basic.yml +++ b/tests/fixtures/golden-configs/cb-basic.yml @@ -73,7 +73,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-extra-validation.yml b/tests/fixtures/golden-configs/cb-extra-validation.yml index 8b8b9a7..24b44cf 100644 --- a/tests/fixtures/golden-configs/cb-extra-validation.yml +++ b/tests/fixtures/golden-configs/cb-extra-validation.yml @@ -74,7 +74,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-min-bid.yml b/tests/fixtures/golden-configs/cb-min-bid.yml index e6ead9c..71a0ff1 100644 --- a/tests/fixtures/golden-configs/cb-min-bid.yml +++ b/tests/fixtures/golden-configs/cb-min-bid.yml @@ -80,7 +80,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-multiple-relays.yml b/tests/fixtures/golden-configs/cb-multiple-relays.yml index b1d5e67..cb89ea0 100644 --- a/tests/fixtures/golden-configs/cb-multiple-relays.yml +++ b/tests/fixtures/golden-configs/cb-multiple-relays.yml @@ -80,7 +80,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-mux.yml b/tests/fixtures/golden-configs/cb-mux.yml index 87b4d4a..2a27ba8 100644 --- a/tests/fixtures/golden-configs/cb-mux.yml +++ b/tests/fixtures/golden-configs/cb-mux.yml @@ -78,7 +78,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-signer.yml b/tests/fixtures/golden-configs/cb-signer.yml index a520320..171188c 100644 --- a/tests/fixtures/golden-configs/cb-signer.yml +++ b/tests/fixtures/golden-configs/cb-signer.yml @@ -82,7 +82,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-sigverify-diff-control.yml b/tests/fixtures/golden-configs/cb-sigverify-diff-control.yml index 1dbe5af..b753834 100644 --- a/tests/fixtures/golden-configs/cb-sigverify-diff-control.yml +++ b/tests/fixtures/golden-configs/cb-sigverify-diff-control.yml @@ -74,7 +74,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-sigverify-diff.yml b/tests/fixtures/golden-configs/cb-sigverify-diff.yml index 7aac423..cec355e 100644 --- a/tests/fixtures/golden-configs/cb-sigverify-diff.yml +++ b/tests/fixtures/golden-configs/cb-sigverify-diff.yml @@ -77,7 +77,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-skip-sigverify.yml b/tests/fixtures/golden-configs/cb-skip-sigverify.yml index a602e46..38a7ea6 100644 --- a/tests/fixtures/golden-configs/cb-skip-sigverify.yml +++ b/tests/fixtures/golden-configs/cb-skip-sigverify.yml @@ -74,7 +74,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-timing-games.yml b/tests/fixtures/golden-configs/cb-timing-games.yml index bb6f363..00d81b3 100644 --- a/tests/fixtures/golden-configs/cb-timing-games.yml +++ b/tests/fixtures/golden-configs/cb-timing-games.yml @@ -77,7 +77,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-ws-stream-nokey.yml b/tests/fixtures/golden-configs/cb-ws-stream-nokey.yml index 70092f5..5a0c8ed 100644 --- a/tests/fixtures/golden-configs/cb-ws-stream-nokey.yml +++ b/tests/fixtures/golden-configs/cb-ws-stream-nokey.yml @@ -77,7 +77,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators diff --git a/tests/fixtures/golden-configs/cb-ws-stream.yml b/tests/fixtures/golden-configs/cb-ws-stream.yml index 0d83e89..e241dfd 100644 --- a/tests/fixtures/golden-configs/cb-ws-stream.yml +++ b/tests/fixtures/golden-configs/cb-ws-stream.yml @@ -77,7 +77,7 @@ mev_params: # (v2 semantics: the relay publishes the block, so a v1 payload would be # silently dropped), returns 502, and EVERY builder block the proposer # chose is lost. That read as "prysm can't do MEV" until the route list was - # checked -- see .agent/SWEEP-BACKLOG.md (Law 7 first dividend). + # checked (Law 7 first dividend). - route: GetPayloadV2 - route: HeaderStream - route: RegisterValidators