Skip to content

feat(discv5)!: prefer IPv4 for dual-stack ENRs and fail requests on send error - #341

Open
MysticRyuujin wants to merge 1 commit into
ChainSafe:masterfrom
MysticRyuujin:fix/dual-stack-ipv4-first
Open

MysticRyuujin wants to merge 1 commit into
ChainSafe:masterfrom
MysticRyuujin:fix/dual-stack-ipv4-first

Conversation

@MysticRyuujin

Copy link
Copy Markdown

Motivation

A node that binds both 0.0.0.0 and :: contacts every dual-stack ENR over IPv6 and never falls back. getSocketAddressMultiaddrOnENR prefers udp6 whenever ipMode.ip6 is set, the chosen address is frozen into the NodeContact, and transport/udp.ts calls socket.send without a callback, so a failed send is never observed. With requestRetries: 1 the request dies on the 1 s timeout with zero retransmits.

In a Docker container without an IPv6 route this means zero peers forever once the bootnode list has no IPv4-only entry. Lodestar v1.48.0 hit exactly this on sepolia and hoodi after the built-in bootnodes moved to a dual-stack fleet (ChainSafe/lodestar#10104 is the caller-side mitigation). Fresh nodes sent no discv5 packets at all; the same image with an IPv4-only bind discovered peers at once.

A second defect surfaced while tracing this: verifyEnr joins the family comparisons with ??, so for a dual-stack ENR observed over IPv6 the IPv4 comparison returns false and the IPv6 endpoint is never checked. With the default allowUnverifiedSessions: false the session is dropped. Dual-stack peers that reach a dual-stack node over IPv6 are rejected today.

Description

  • Dual-stack ENRs are contacted over IPv4 first. The order is configurable through the new optional ISessionConfig.dualStackPreferredFamily (4 | 6, default 4). IPMode stays a pure capability. This matches go-ethereum, which prefers IPv4 on a tie for dual-stack records.
  • UDPTransportService.send resolves or rejects through the socket.send callback. dgram reports send errors only that way. A rejected request send fails the request at once with the new RequestErrorType.SendFailed, guarded against late callbacks for a request that was already answered or replaced. Response and challenge sends keep the log-only path.
  • Discv5 remembers the socket address each node last sent an authenticated message from (confirmedAddrs, written on decrypted requests and verified responses only, bounded like the session cache, cleared on Disconnected and stop). Later requests to that node reuse it through contactFor, so a peer that dialed us over IPv6 keeps getting IPv6 and IPv6 address votes keep flowing under the IPv4-first default. An outgoing handshake never writes the cache: it proves nothing about the address until something comes back. createNodeContactAt builds an Enr contact at a given bare address; a Raw contact would trigger the internal FINDNODE(0) ENR exchange after session expiry and carry a /p2p suffix that fails the reply address check.
  • The IPv6 vote-harvest ping (bucket-full outgoing peers) prefers IPv6 explicitly and is marked as a probe. A probe failure does not fail requests queued behind it (failRequest hands the queue to sendNextRequest instead of failSession) and does not mark the peer Disconnected in rpcFailure. The probe flag survives the pending queue.
  • verifyEnr accepts a dual-stack ENR observed over either family.
  • New metric discv5_request_failed_count{reason}.

Breaking

Dual-stack peers are now contacted over IPv4 first. Set dualStackPreferredFamily: 6 to restore the previous order. A host with working IPv6 and broken IPv4 loses connectivity to dual-stack peers until it sets that option; a follow-up adds per-request fallback to the other family.

Tests

  • test/unit/util/ip.test.ts, test/unit/session/nodeInfo.test.ts: selection order and the endpoint-override contact.
  • test/unit/transport/udp.test.ts: send rejects on an undeliverable datagram, before start, and for an unbound family.
  • test/unit/session/service.test.ts: stub transport; SendFailed without waiting for the timeout, queue failure semantics, probe isolation, probe flag through the queue, verifyEnr over either family.
  • test/e2e/dualStack.test.ts: loopback nodes where the peer advertises an unreachable IPv6 address next to reachable 127.0.0.1. Default reaches it over IPv4 (2001:db8::1, which some kernels blackhole and others refuse); IPv6-first fails fast with SendFailed (::ffff:127.0.0.1, which an IPv6-only socket refuses on every platform); a peer that dialed us over ::1 is contacted back over ::1.

The existing mainnetBootnodes e2e cannot show this regression because the mainnet list still has IPv4-only entries.

Follow-up

Sequential endpoint fallback in SessionService: try the other family after a SendFailed or a timeout, with a bounded success cache. This PR leaves the single request-send chokepoint, the probe flag in both layers and contactFor in place for it.

Verification on an IPv4-only Docker host

Lodestar built from a throwaway branch that overrides @chainsafe/discv5 with this branch, run on a Linux host whose Docker bridge has no IPv6 route, with --listenAddress 0.0.0.0 --listenAddress6 :: so both sockets are bound, built-in bootnodes only (SKIP_FETCH_NETWORK_BOOTNODES=1), 90 seconds after start:

node discv5 connected kad table ENRs decoded request failures
sepolia, this branch 15 20 138 147 Timeout, 8 SendFailed
sepolia, unpatched v1.48.0 0 7 (bootnodes only) 0 metric not available
hoodi, this branch 17 20 3 6 Timeout

The unpatched node never decodes a single ENR: every request goes to an IPv6 address and dies on the timeout. The SendFailed count on the patched node is the IPv6 vote-harvest probes, which are isolated from the routing table by design. The unit and e2e suites also pass on that host inside a node:24 container both with host networking (IPv6 route present) and on the bridge network (none).

AI Assistance Disclosure

This PR was written primarily by Claude Code: the investigation, the code, the tests and this description. I directed the work and validated the fix on our own infrastructure. A second model (Codex) reviewed the design twice and the diff twice before submission. Replies in this PR may also be drafted with AI assistance.

…end error

Dual-stack ENRs are contacted over IPv4 first (configurable via dualStackPreferredFamily). UDP send errors fail the request at once with SendFailed. The address a node last sent an authenticated message from is reused for later requests. The IPv6 vote-harvest ping is an isolated probe. verifyEnr accepts a dual-stack ENR observed over either family.
@MysticRyuujin
MysticRyuujin requested a review from a team as a code owner September 16, 2026 18:35
nflaig added a commit to ChainSafe/lodestar that referenced this pull request Sep 19, 2026
…10104)

## Motivation

A fresh v1.48.0 beacon node on sepolia or hoodi that runs in a Docker
container without an IPv6 route finds zero peers. Discovery never sends
a UDP packet.

`parseListenArgs` binds `::` when the user sets neither
`--listenAddress` nor `--listenAddress6`. With an IPv6 socket bound,
`@chainsafe/discv5` contacts every dual-stack ENR over IPv6 and does not
fall back to IPv4 (`getSocketAddressMultiaddrOnENR`, same rule as
sigp/discv5 `DualStack`). In a container without IPv6 connectivity every
send to those peers fails.

Until v1.47.0 the built-in sepolia and hoodi bootnode lists contained
IPv4-only EF bootnodes, which masked this. #10049 replaced them with the
NodeOps fleet, which is dual-stack. The only IPv4-only entry left (a
Teku node) does not answer. The v1.48.0 lists therefore contain no
bootnode a default-configured IPv4-only container can reach. Mainnet
still has IPv4-only bootnodes and is not affected.

Verified on a v4-only Docker host on sepolia with the fleet ENRs as
`--bootnodes`:

| image | flags | result after 90s |
| --- | --- | --- |
| v1.48.0 | default | 0 peers, 0 ENRs decoded, 0 UDP packets sent |
| v1.48.0 | `--listenAddress 0.0.0.0` | syncing, hundreds of ENRs
discovered |
| v1.48.0 | `--bootnodes <IPv4-only EF ENR>` | syncing |
| v1.47.0 | default | syncing (built-in IPv4-only EF bootnodes) |

## Description

Bind IPv6 by default only if the host has a global unicast IPv6 address.
`hasGlobalIPv6Address` reads `os.networkInterfaces()` and accepts only
addresses inside the IANA global unicast block `2000::/3`, minus the
special-purpose prefixes allocated inside it that the IANA registry
marks not globally reachable (IETF protocol assignments including
Teredo, 6to4, documentation, SRv6 SIDs), through two `net.BlockList`s.
Everything else (loopback, IPv4-mapped, NAT64, unique-local, link-local,
site-local) falls outside `2000::/3`. An explicit `--listenAddress6`
still forces IPv6 on. An explicit `--listenAddress` still turns the IPv6
default off, as before.

`parseListenArgs` takes the detection result as a second parameter so
tests stay independent of the test host. Adds unit tests, updates the
`--listenAddress6` description and the networking docs.

Validated with an image built from this branch on the same IPv4-only
Docker host: default flags bind IPv4 only
(`bindAddr4=/ip4/0.0.0.0/udp/...`, no `bindAddr6`) and discover peers on
sepolia and hoodi; `--listenAddress6 ::` still binds IPv6 as before.

Lodestar warns at startup when it skips the IPv6 default for this
reason. A persisted ENR loses its `ip6`/`udp6`/`tcp6`/`quic6` fields
when no IPv6 listener is configured and no `--enr.*6` flag is given, so
peers stop dialing a dead IPv6 endpoint and the beacon handler's
`enrUpdate` gate is no longer blocked by a stale `ip6`.

The library-side fix, IPv4-first selection and fast failure on send
errors in `@chainsafe/discv5`, is ChainSafe/discv5#341.

## AI Assistance Disclosure

- [x] External Contributors: I have read the [contributor
guidelines](https://github.com/ChainSafe/lodestar/blob/unstable/CONTRIBUTING.md#ai-assistance-notice)
and disclosed my usage of AI below.

This PR was written primarily by Claude Code, including the
investigation, the code, the tests and this description. I directed the
work and validated the fix on our own infrastructure. A second model
(Codex) reviewed the diff before submission. Replies in this PR may also
be drafted with AI assistance.

---------

Co-authored-by: Nico Flaig <nflaig@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant