feat(p2p): add opt-in discv5 peer discovery - #579
Draft
MegaRedHand wants to merge 7 commits into
Draft
Conversation
Lean nodes could only meet through a static bootnode list, so every new node needed an operator to hand it peers. This wires ethrex's discv5 stack in behind `--discovery.enable`: the node builds and signs its own ENR, joins the DHT on its own UDP socket, and dials what it finds over libp2p QUIC. Static bootnode dialing is untouched and discovery is off by default, so nothing changes for an operator who does not ask for it. Admission follows the beacon phase0 p2p spec, mirroring lighthouse's `eth2_fork_predicate`: the `eth2` fork digest must match, a differing `next_fork_version`/`next_fork_epoch` is explicitly tolerated, and the peer must advertise a `quic` port. The checks live in a `LeanFilter` that ethrex's peer table runs as each ENR arrives, so a record is judged where it lands rather than at dial time, and is judged afresh whenever the peer publishes a higher-`seq` record. Survivors are ranked by how many attestation subnets they cover that no connected peer does, so discovery fills subnet gaps first. A peer's `attnets` is self-reported, so subnet ids at or beyond the local committee count are dropped before ranking sees them. `ethrex-p2p` is pinned to the unmerged `feat/discovery-peer-requirements` branch, which carries the unified `DiscoveryServer`, the peer table, and the `PeerFilter` seam. Repoint it at a main revision once that merges. Known gap: `DiscoveryServer::spawn` builds its own local record and offers no way to seed the consensus entries, so the ENR ethrex answers queries with carries `ip`/`udp`/`secp256k1` but not `eth2`, `attnets` or `quic`. Discovery is one-sided until `spawn` can take a prepared record: we find and admit lean peers, but a lean peer applying these same rules to what ethrex serves would refuse us. See `docs/discovery.md`.
MegaRedHand
added a commit
that referenced
this pull request
Aug 13, 2026
## What Adds the three operator-facing flags the discv5 work needs, on their own, so the implementation PR (#579) is confined to the p2p crate. | Flag | Default | Meaning | | --- | --- | --- | | `--discovery.enable` | `false` | turn discv5 peer discovery on | | `--discovery.port` | `9000` | UDP port for the discv5 socket | | `--discovery.advertise-ip` | unset | IP to advertise in the ENR | The flags parse and validate here. **Nothing reads them yet**, which is the point of splitting them out: this is reviewable on its own and cannot change runtime behaviour of a node that does not pass them. ## Why the port validation `--discovery.port` and `--gossipsub-port` are both UDP and both default to 9000, so enabling discovery without moving one of them collides. Left unchecked, that surfaces at bind time as an opaque `EADDRINUSE` on whichever socket loses the race, pointing at neither flag. `CliOptions::validate_discovery` rejects it at startup with a message naming both flags and their values. The check only fires when discovery is enabled, so the shared default is harmless for every existing deployment. ## Why `--discovery.advertise-ip` The node binds the wildcard `0.0.0.0`, which is not dialable as published. A node whose reachable address differs from what it listens on (a devnet on `127.0.0.1`, or a host behind NAT) needs to say so explicitly. discv5's PONG-based IP voting may still learn and substitute the real external address at runtime; this only sets what the ENR carries at startup. ## Testing - `make lint` clean. - Colliding ports are rejected by name: ``` $ ethlambda ... --discovery.enable Error: --discovery.port (9000) must differ from --gossipsub-port (9000): both bind UDP and cannot share a port ``` - Distinct ports pass validation and startup proceeds: ``` $ ethlambda ... --discovery.enable --discovery.port 9010 Error: failed to load node key from /nonexistent/node.key ``` - The group renders under `--help` with its dotted prefixes intact. ## Relationship to #579 #579 carries the discv5 implementation and currently includes these same flags. If this lands first, #579 rebases onto it and drops the `cli.rs` hunk.
Quality pass over the discovery feature. No behaviour change; the one observable difference is that a malformed bootnode ENR now warns once instead of twice, because the file is parsed once. Reuse and layering: - Merge `ethlambda-types::enr` into `p2p::discovery::enr`. The shared types crate grew an SSZ container and a `libssz_derive` use for a single consumer crate, and split `encode_attnets` from the `ATTNETS_ENR_KEY` that gives it meaning. `FORK_DIGEST` stays in `types::constants`, where a second crate does use it. - Move the dial loop out of `lib.rs` into `discovery::dial`, matching how `gossipsub::handler` and `req_resp::handlers` already keep their bodies out of the shared actor file. `DiscoveryState`, `covered_subnets` and `local_peer_id` go with it, so dial policy is editable without touching shared actor state. - Add `P2PServer::forget_discovered_peer` so the two teardown paths (`ConnectionClosed` and `OutgoingConnectionError`) share a seam instead of both reaching into `peer_attnets`. - One `quic_multiaddr()` for the two dial paths that were building the same `ip / udp / quic-v1 / p2p` chain, and `ethrex_p2p::utils::public_key_from_signing_key` in place of the hand-rolled uncompressed-SEC1 conversion (which had three copies). - Fold the `!= 0` filter into `read_quic_port` and have `parse_enr` call it. The two spellings of "no dialable quic port" had already drifted. - Drop `read_extra`: ethrex's `pairs.extra()` already returns `Bytes`, so the wrapper only added a copy on a path that runs per arriving ENR. Simplification: - `subnets_from_attnets(bits, committee_count)` replaces decode-everything-then-clamp. Iterating our own committee makes the clamp unforgettable rather than documented in three places, and stops a padded hostile bitfield allocating ~18 KB before being discarded. - `DiscoveryError` via `thiserror` replaces 12 hand-rolled `String` errors; p2p was the only crate in the workspace without it. `main.rs` loses its `map_err(|err| eyre::eyre!(err))` bridge. - Delete `DiscoveredPeer::label` and `DiscoveryHandle::bound_addr`: both were read only by tests, and `label` allocated a base58 string on every admission while the one log line uses `%peer_id`. The ENR-vs-bound-port test now asserts on the record's `udp` entry, which is the invariant. - Delete `RejectReason::as_str`, whose five strings restated the five variant docs for one `debug!`. - Read the bootnode file once (`#[derive(Clone)] Bootnode`) and inline the locals copied out of `options.discovery`. Move the unspecified-IP warning into `spawn_discovery`, next to the code that picks the value. Dependencies: - `DiscoverySpawnConfig::node_key` takes `Vec<u8>` like its sibling `SwarmConfig::node_key`, which removes the binary's direct `secp256k1` dependency and its version-coupling to ethrex's workspace. - p2p: drop the unused `recovery` feature, move `bytes` and `rand` to dev-dependencies (both are test-only). Revert `pub mod req_resp` / `pub mod encoding` to private: they were widened for an `examples/mainnet_gossip.rs` that is not in the tree, and making `req_resp` public also exposed the actor-facing `handlers` module. `NodeIdentity` reaches the identity route behind an `Arc`, so a polled endpoint stops cloning two startup-fixed strings per request.
The merge with main placed it after a blank line, outside the list it belongs to and flush against the new Development heading.
Our lock pinned 669de531, which is no longer on the branch: it was rebased away, so the build only kept working because the old commit was still in the local Cargo cache. A fresh clone would not have resolved it. Three API changes come with f30b16d5: - `PeerTableServer::spawn_with_filter` takes `impl PeerFilter + 'static` instead of `Box<dyn PeerFilter>`, so the call site drops its `Box::new`. - `NodeRecordPairs::set_extra_int` takes a `u64` rather than any `RLPEncode`, which is deliberate upstream: a generic bound under a method named for integers would re-open the encode-a-`Vec<u8>`-as-a-list footgun that `set_extra` exists to close. - Both setters now answer whether the entry was stored, `false` for a key the record already has a typed field for. `attnets`, `eth2` and `quic` are all outside that dictionary and the tests assert each one lands in the built record, so `local_pairs` does not check the answers. `PeerFilter::accepts` is unchanged, so `LeanFilter` needed no edit.
The helpers predate `f7fddb9dc` upstream, which added the `set_extra*` accessors so callers stop writing `extra_fields` directly. They still assigned the whole bag and hand-rolled the RLP for each entry, which made these tests the one place an ENR was assembled differently from the way `build_local_enr` assembles one: a `pair()` returning `(Bytes, Bytes)`, `Bytes::from(..).encode_to_vec()` per payload, and a comment explaining which of the two encodings that produced. `record_with` now takes a closure over `NodeRecordPairs` and the entries go through `set_extra`/`set_extra_int`, so a record these tests accept is one built the way production builds it, encoding included. Assertions are unchanged. Since nothing names `Bytes` any more, the `bytes` dev-dependency and the `ethrex_rlp::encode::RLPEncode` import go with it. `set_extra_encoded` stays unused: it exists for values the typed setters cannot express, such as a deliberately malformed RLP list, and no test wants one yet.
Bumps ethrex to the feat/discovery-peer-requirements tip (f30b16d5 -> bf401280, rebased onto main 24.0.0), which reworks `DiscoveryServer::spawn` to take a prepared `NodeRecord` instead of a `Store` it derived one from. That closes the gap docs/discovery.md called "the record ethrex serves is not the record we report": ethrex built its own copy from the local `Node`, so what it answered discv5 queries with carried `ip`, `udp` and `secp256k1` but none of `eth2`, `attnets` or `quic`. A lean peer applying our own admission rules to that record rejected us for the missing `quic` entry, so discovery found peers but could not be found by them. We now hand `spawn` the same record `enr_url` reports, and ethrex edits and re-signs it on IP voting rather than rebuilding, so the consensus entries survive a sequence bump. The empty in-memory ethrex `Store` existed only to satisfy the old signature, so both it and the `ethrex-storage` dependency go, along with the `DiscoveryError::Store` variant that could no longer be constructed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds opt-in discv5 peer discovery, so a lean node can find peers instead of
being handed them. Off by default;
--discovery.enableturns it on and--discovery.portgives it its own UDP socket (it must differ from--gossipsub-port, and the node refuses to start otherwise rather thanfailing later with an opaque
EADDRINUSE). Static bootnode dialing isuntouched.
Built on ethrex's discovery stack:
DiscoveryServerruns discv5-only andwrites what it finds into a
PeerTable, whichP2PServerpolls, filters anddials over libp2p QUIC. We build the local ENR ourselves and hand it to
spawn, so the record ethrex answers queries with is the one this nodereports.
How peers are judged
Admission follows the beacon phase0 p2p spec, mirroring lighthouse's
eth2_fork_predicate:eth2entryfork_digestnext_fork_version/next_fork_epochquicportsecp256k1,ip/ip6These live in a
LeanFilterhanded to the peer table as itsPeerFilter, soeach record is judged the moment it arrives rather than at dial time. No
rejection is final: the peer table re-runs the filter as soon as the peer
publishes a higher-
seqENR, so a node that adds aquicentry or gains anaddress through discv5's IP voting is reconsidered without a restart.
Admitted peers are ranked by how many attestation subnets they advertise that
no connected peer covers, so discovery fills coverage gaps first.
attnetsisself-reported and unauthenticated, so subnet ids at or beyond the local
committee count are dropped before ranking sees them: otherwise an ENR padding
its bitfield with a few hundred bytes of
0xFFwould outrank every honest peerforever.
Also here
--discovery.advertise-ipseparates the bound address from the advertisedone, for a node behind NAT or on a host whose public IP is not what it binds.
GET /lean/v0/node/identityreports the local ENR alongside the peer id,grouped into a
NodeIdentitystruct.quicport: one with only audpentryis kept as a discv5 seed even though it cannot be dialed over libp2p.
docs/discovery.mdcovers the ENR layout, the admission rules, the operatorflags and the known limitations.
Dependency
ethrex-p2pis pinned to the unmergedfeat/discovery-peer-requirementsbranch, which carries the unified
DiscoveryServer, the peer table and thePeerFilterseam.Cargo.lockpins the exact commit (currentlybf401280),so builds are reproducible. This should be repointed at a main revision
before merge.
Note that ethrex still uses libssz 0.2.2 while ethlambda is on 0.3.0, so the
dependency graph now carries both. Nothing SSZ-typed crosses the boundary
(lean's
EnrForkIdis its own type), but it is worth knowing.Known limitation: one lean devnet is not separated from another
Lean's
fork_digestis the hardcoded cross-client dummy0x12345678, so theeth2check separates lean from non-lean but not one lean devnet fromanother. Two devnets running this code will peer with each other. Closing
that needs lean to adopt a genesis-derived fork digest, which is a
cross-client change to gossip topic names.
Resolved: discovery is no longer one-sided
An earlier revision of this PR shipped with the ENR we reported and the ENR
we served being different records.
DiscoveryServer::spawntook an ethrexStoreand derived its own record from the localNode, with no way to seedthe consensus entries, so what it answered queries with carried
ip,udpand
secp256k1but none ofeth2,attnetsorquic. We found and admittedlean peers, but a lean peer applying these same rules to what ethrex served
would have refused us for a missing
quicentry.spawnnow takes a preparedNodeRecord, and we pass the onebuild_local_enrproduces, so the two are the same bytes. Discv5's IP voting
edits andre-signs that record rather than rebuilding it, so the consensus entries
survive a sequence bump. The empty in-memory
Storethat existed only tosatisfy the old signature is gone, along with the
ethrex-storagedependency.Testing
make lintclean.cargo test --workspace --profile release-fast --no-fail-fast: 601 passed,0 failed, including the forkchoice, signature, STF and SSZ spec tests.
oversized-
attnetsranking attack, subnet ranking, andspawn_discoverybinding a real socket (including
--discovery.advertise-ipand a busy port).Draft because the ethrex dependency is still an unmerged branch.