Merge reth v2.8.0

.agents/skills/reth-rust-style/SKILL.md

@@ -0,0 +1,102 @@
+---
+name: reth-rust-style
+description: Review rules reth reviewers enforce beyond AGENTS.md. Use when writing, refactoring, or reviewing Rust in reth, before opening or updating a PR.
+---
+
+# Reth Rust Style
+
+These are review comments reth reviewers leave often that AGENTS.md doesn't spell out. AGENTS.md still applies; this file only adds to it.
+
+## Scope
+
+- Before pushing, read the full diff and revert every hunk the PR description doesn't explain.
+- Refactors keep behavior identical. If behavior must change, say so in the PR description.
+- When wrapping or moving a type, keep existing call sites unchanged (`Deref` or delegating methods).
+
+## Reuse before writing
+
+- Prefer upstream APIs to manual byte handling:
+  - `B256::from(U256)`, `U256::from(B256)`, `Address::into_word()`
+  - `Log::topic0()`, `Signature::as_bytes()`, `Bytes::from([u8; N])`
+  - `PayloadId::from_str`, alloy `From` impls between RPC and consensus types
+  - `alloy_rlp::decode_exact`, `abi_decode_validate`, `sol!` for ABI types
+- Use alloy's fixed-bytes maps (`B256Map`, `AddressMap`) for maps keyed by hashes or addresses.
+- Use the constructors a type already has (`ExecOutput::done`, `ExecOutput::in_progress`, `StorageEntry::new`) instead of struct literals.
+- When the same logic appears twice, extract one helper.
+
+## Keep it small
+
+- No wrapper, trait, enum, or type alias with a single use. The one exception is a type alias that replaces `#[allow(clippy::type_complexity)]`.
+- No caches, interior mutability, or format versioning without a measured need.
+- Don't re-check an invariant that is already enforced upstream. If correctness depends on it, say where it's enforced in a one-line comment.
+- Remove unused parameters instead of `_`-prefixing them.
+
+## Public API
+
+- reth is a library. Don't remove or rename public items without a reason in the PR description.
+- A new trait method needs a default or no-op impl unless every implementor must provide it.
+- Public types need public fields or accessors for the data callers need.
+- Destructive or write operations belong on the mutable transaction (`DbTxMut`), not on `DbTx`. Say in their docs that they are destructive.
+- Re-export types that appear in public signatures from the crate that exposes the signature.
+
+## Comments and docs
+
+- A comment states one non-obvious fact. Don't restate the code or narrate the change.
+- Keep existing comments and TODOs when moving code.
+- When behavior changes, update everything that describes it in the same PR: doc comments, numbered step lists, `expect` and `unreachable!` messages, metric names.
+
+## Naming
+
+- Names say what a thing does or holds, including side effects (`*_blocking`).
+- Match sibling names (`*_with_nodes` next to the trait that uses it).
+- Rename an item when its meaning changes. Leave unrelated names alone.
+- Don't reuse names of alloy types for different things (prefix with `Pooled*` or similar).
+- Don't alias imports with `use X as Y`; it breaks grep.
+
+## Errors
+
+- Carry the source error in the variant, and don't report provider errors as consensus errors.
+- Never discard a fallible result with `.ok()` or `let _ =`. Ignore only the specific variant you expect.
+- Don't turn values a spec leaves open into hard decode errors. Decode, then truncate or reject at the layer that owns the limit.
+- Use `checked_*` and return an error for gas and index math. Use `try_from` for conversions; `as` only when it can't truncate.
+
+## Types
+
+- Make invalid states unrepresentable. Put variant-specific data inside enum variants, not in correlated `bool`/`Option` fields.
+- Use `usize` for limits that size collections, so they need no casts.
+- Check serde compatibility of changed public types.
+- A function that doesn't use `self` is a free function. Logic that belongs to a type goes in its `impl`.
+- Reject mutually exclusive CLI flags at parse time with clap's `conflicts_with`, and update the parse-all test.
+
+## Tests
+
+- Every new branch, guard, error path, and unwind or reopen path gets a test in the same PR.
+- Assert the exact error or message, not just `is_err()`.
+- Fixtures must be coherent: request the same root the fixture verifies, and cover every match arm the loop claims to.
+- Don't commit `#[ignore]` manual perf tests. Write a bench.
+
+## Async and observability
+
+- Every wait is bounded and observes shutdown. Every queue and retained map has a bound.
+- A spawned task enters the caller's span for the whole closure, not only as the parent of a `DEBUG` child; check it survives an `INFO` filter.
+- Log `debug!` or `trace!` on state transitions in background tasks.
+- Count skips with a dedicated metric, not by recording zeros.
+
+## Layout and visibility
+
+- When dropping an import, remove the whole line and any blank line it leaves.
+- Keep items `pub(crate)` or private unless another crate uses them.
+- Use `#[expect(lint, reason = "...")]` instead of `#[allow]`.
+
+## Performance
+
+- Don't clone large values to read them. Borrow, `take`, or share through `Arc`. Getters return references.
+- Don't read the same DB value twice, and build optional data lazily (`Option`) when only some paths need it.
+- Cap any length that comes from untrusted input before `Vec::with_capacity` or a loop.
+
+## Dependencies and CI
+
+- Workspace dependencies have default features off. Check that each touched crate builds on its own, with and without its features.
+- Keep `no_std` crates free of `std`; use `core` and `alloc`.
+- Dependency bumps respect the MSRV.
+- Pin reusable workflows by SHA only after the 7-day soak, with a trailing `# repo#PR` comment. Never reference `@main` or expose secrets to cargo builds.

.claude/skills

@@ -0,0 +1 @@
+../.agents/skills

.config/zepter.yaml

@@ -12,7 +12,7 @@ workflows:
         # Check that `A` activates the features of `B`.
         "propagate-feature",
         # These are the features to check:
-        "--features=std,dev,asm-keccak,jemalloc,jemalloc-prof,tracy-allocator,tracy,serde-bincode-compat,serde,test-utils,arbitrary,min-error-logs,min-warn-logs,min-info-logs,min-debug-logs,min-trace-logs,otlp,otlp-logs,js-tracer,portable,keccak-cache-global,trie-debug,secp256k1",
+        "--features=std,dev,account-ext,asm-keccak,jemalloc,jemalloc-prof,tracy-allocator,tracy,serde-bincode-compat,serde,test-utils,arbitrary,min-error-logs,min-warn-logs,min-info-logs,min-debug-logs,min-trace-logs,otlp,otlp-logs,js-tracer,portable,keccak-cache-global,trie-debug,secp256k1",
         # Do not try to add a new section to `[features]` of `A` only because `B` exposes that feature. There are edge-cases where this is still needed, but we can add them manually.
         "--left-side-feature-missing=ignore",
         # Ignore the case that `A` it outside of the workspace. Otherwise it will report errors in external dependencies that we have no influence on.

.github/dependabot.yml

@@ -8,13 +8,16 @@ updates:
       time: "09:00"
       timezone: UTC
     cooldown:
-      default-days: 7
+      # Buffer the scanner's seven-day minimum: Dependabot reads date-only tag metadata.
+      default-days: 8
     groups:
       ci-weekly:
         patterns:
           - '*'
   - package-ecosystem: cargo
     directory: /
+    allow:
+      - dependency-type: all
     schedule:
       interval: weekly
       day: monday

.github/scripts/bench-args-test.js

@@ -0,0 +1,90 @@
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const path = require('node:path');
+const vm = require('node:vm');
+const { test } = require('node:test');
+
+const workflow = fs.readFileSync(path.join(__dirname, '../workflows/bench.yml'), 'utf8');
+const source = workflow.match(/            const validBalModes = [\s\S]*?(?=\n      - name: Acknowledge request)/)[0];
+
+async function parse(args = {}, eventName = 'workflow_dispatch') {
+  const outputs = {};
+  const failures = [];
+  await vm.runInNewContext(`(async () => {${source}\n})()`, {
+    process: { env: Object.fromEntries(Object.entries(args).map(([key, value]) => [
+      `INPUT_${key.toUpperCase().replaceAll('-', '_')}`, value,
+    ])) },
+    context: {
+      eventName,
+      repo: { owner: 'paradigmxyz', repo: 'reth' },
+      issue: { number: 27928 },
+      payload: { comment: { user: { login: 'tester' }, body: `derek bench ${Object.entries(args).map(([key, value]) => `${key}=${value}`).join(' ')}` } },
+    },
+    github: { rest: { pulls: {
+      list: async () => ({ data: [] }),
+      get: async () => ({ data: { head: { sha: 'test-sha', ref: 'test-ref', repo: { full_name: 'paradigmxyz/reth' } } } }),
+    } } },
+    core: {
+      setOutput: (key, value) => { outputs[key] = value; },
+      setFailed: message => failures.push(message),
+      info: () => {},
+    },
+  });
+  return { outputs, failures };
+}
+
+for (const event of ['workflow_dispatch', 'issue_comment']) {
+  for (const bal of ['true', 'baseline', 'feature']) {
+    test(`${event}: bal=${bal} uses the existing big-block format in both arms`, async () => {
+      const { outputs, failures } = await parse({ bal }, event);
+      assert.deepEqual(failures, []);
+      assert.equal(outputs['big-blocks'], 'true');
+      assert.equal(outputs['big-blocks-target-gas'], '1');
+      assert.equal(outputs['node-bin'], 'reth-bb');
+      assert.equal(outputs.bal, bal);
+      assert.equal(outputs.blocks, '500');
+      assert.equal(outputs.warmup, '125');
+      assert.equal(outputs['run-pairs'], '6');
+    });
+    for (const extra of [{ reorg: '5' }, { mempool: 'true' }]) {
+      test(`${event}: rejects bal=${bal} with ${Object.keys(extra)[0]}`, async () => {
+        const { outputs, failures } = await parse({ bal, ...extra }, event);
+        assert.deepEqual(outputs, {});
+        assert.deepEqual(failures, [extra.reorg
+          ? 'reorg is not supported with big-blocks or BAL (reth-bb cannot build reorg blocks)'
+          : 'mempool is only supported in Engine mode without big-blocks or reorg']);
+      });
+    }
+  }
+
+  test(`${event}: retains explicit big-block targets and run sizes`, async () => {
+    const { outputs, failures } = await parse({ bal: 'true', 'big-blocks': '2G', blocks: '17', warmup: '3', 'run-pairs': '2' }, event);
+    assert.deepEqual(failures, []);
+    assert.equal(outputs['big-blocks-target-gas'], '2G');
+    assert.equal(outputs.blocks, '17');
+    assert.equal(outputs.warmup, '3');
+    assert.equal(outputs['run-pairs'], '2');
+  });
+
+  test(`${event}: non-BAL reorg replay continues using reth`, async () => {
+    const { outputs, failures } = await parse({ reorg: '5' }, event);
+    assert.deepEqual(failures, []);
+    assert.equal(outputs['node-bin'], 'reth');
+    assert.equal(outputs['big-blocks'], 'false');
+    assert.equal(outputs.reorg, '5');
+  });
+
+  test(`${event}: rejects big-block reorgs even without BAL`, async () => {
+    const { outputs, failures } = await parse({ 'big-blocks': '1', reorg: '5' }, event);
+    assert.deepEqual(outputs, {});
+    assert.deepEqual(failures, ['reorg is not supported with big-blocks or BAL (reth-bb cannot build reorg blocks)']);
+  });
+
+  for (const mode of ['rpc', 'call']) {
+    test(`${event}: rejects BAL in ${mode} mode`, async () => {
+      const { outputs, failures } = await parse({ mode, bal: 'true' }, event);
+      assert.deepEqual(outputs, {});
+      assert.deepEqual(failures, [`${mode} mode does not support big-blocks, reorg, or BAL`]);
+    });
+  }
+}

.github/scripts/bench-scheduled-refs.sh

@@ -123,17 +123,16 @@ if [ "$MODE" = "release" ]; then
 
   # --- Step 1: Resolve feature ref from latest nightly Docker build ---
   echo "::group::Querying latest nightly docker build"
+  # Filter client-side: the API's event/status filters can return stale runs.
   RUNS_JSON=$(gh run list \
     -R "$REPO" \
     --workflow=docker.yml \
-    --event=schedule \
-    --status=completed \
-    --limit 5 \
-    --json headSha,createdAt,conclusion)
+    --limit 200 \
+    --json headSha,createdAt,conclusion,event)
 
-  LATEST=$(echo "$RUNS_JSON" | jq -r '[.[] | select(.conclusion == "success")] | first // empty')
+  LATEST=$(echo "$RUNS_JSON" | jq -r '[.[] | select(.event == "schedule" and .conclusion == "success")] | first // empty')
   if [ -z "$LATEST" ]; then
-    echo "::error::No successful scheduled docker.yml run found in the last 5 runs"
+    echo "::error::No successful scheduled docker.yml run found in the last 200 runs"
     exit 1
   fi
 
@@ -217,19 +216,18 @@ fi
 # --- Step 1: Query latest successful scheduled docker.yml run ---
 echo "::group::Querying latest nightly docker build"
 
+# Filter client-side: the API's event/status filters can return stale runs.
 RUNS_JSON=$(gh run list \
   -R "$REPO" \
   --workflow=docker.yml \
-  --event=schedule \
-  --status=completed \
-  --limit 5 \
-  --json headSha,createdAt,conclusion)
+  --limit 200 \
+  --json headSha,createdAt,conclusion,event)
 
 # Find the most recent successful run
-LATEST=$(echo "$RUNS_JSON" | jq -r '[.[] | select(.conclusion == "success")] | first // empty')
+LATEST=$(echo "$RUNS_JSON" | jq -r '[.[] | select(.event == "schedule" and .conclusion == "success")] | first // empty')
 
 if [ -z "$LATEST" ]; then
-  echo "::error::No successful scheduled docker.yml run found in the last 5 runs"
+  echo "::error::No successful scheduled docker.yml run found in the last 200 runs"
   echo "Runs found: $RUNS_JSON"
   exit 1
 fi

.github/scripts/bench-state-test.sh

@@ -57,7 +57,7 @@ gh() {
       if [[ "$*" == *'--status=in_progress'* ]]; then
         printf '0\n'
       else
-        printf '[{"headSha":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","createdAt":"%s","conclusion":"success"}]\n' "$STATE_TEST_DATE"
+        printf '[{"headSha":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","createdAt":"%s","conclusion":"success","event":"schedule"}]\n' "$STATE_TEST_DATE"
       fi
       ;;
     *) return 1 ;;

.github/scripts/bench-txgen-build-test.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+script="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/bench-txgen-build.sh"
+scratch=$(mktemp -d)
+trap 'rm -rf "$scratch"' EXIT
+export BUILD_TEST_LOG="$scratch/build-args"
+
+cargo() {
+  printf '%s\n' "$*" > "$BUILD_TEST_LOG"
+}
+rustup() { :; }
+export -f cargo rustup
+
+for arm in baseline feature; do
+  for bal in false true baseline feature; do
+    for big_blocks in false true; do
+      BENCH_BAL="$bal" BENCH_BIG_BLOCKS="$big_blocks" BENCH_TRACY=off \
+        bash "$script" "$arm" "$scratch" test-ref > /dev/null 2>&1
+      if [ "$big_blocks" = true ]; then
+        expected='build --locked --profile profiling -p reth-bb'
+      else
+        expected='build --locked --profile profiling --bin reth'
+      fi
+      [[ "$(cat "$BUILD_TEST_LOG")" == "$expected" ]]
+    done
+  done
+done
+echo 'PASS: both arms build the selected node for all BAL modes'

.github/scripts/bench-txgen-run.sh

@@ -14,7 +14,7 @@
 # Optional env: BENCH_EXECUTION_MODE, BENCH_BIG_BLOCKS, BENCH_BIG_BLOCKS_TARGET_GAS,
 #               BENCH_CALL_CLASS, BENCH_CALL_METHODS, BENCH_CALL_RPS,
 #               BENCH_CALL_DURATION, BENCH_CALL_PASSES, BENCH_CALL_CONCURRENCY,
-#               BENCH_REORG, BENCH_BAL,
+#               BENCH_REORG, BENCH_BAL, BENCH_MEMPOOL,
 #               BENCH_WORK_DIR, BENCH_WAIT_TIME, BENCH_BLOCK_TIME, BENCH_BASELINE_ARGS,
 #               BENCH_FEATURE_ARGS, BENCH_OTLP_TRACES_ENDPOINT,
 #               BENCH_OTLP_LOGS_ENDPOINT, BENCH_OTLP_DISABLED,
@@ -516,6 +516,19 @@ fi
 if [ -n "${BENCH_REORG:-}" ]; then
   TXGEN_SEND_ARGS+=(--reorg "$BENCH_REORG")
 fi
+# Submits each block's transactions to the txpool before its payload, so
+# that features acting on pool transactions see them. BENCH_MEMPOOL is
+# `true` for txgen's defaults, DELAY, or DELAY,FRACTION.
+if [ "$EXECUTION_MODE" = "engine" ] && [ -n "${BENCH_MEMPOOL:-}" ]; then
+  TXGEN_SEND_ARGS+=(--mempool)
+  if [ "$BENCH_MEMPOOL" != "true" ]; then
+    IFS=, read -r MEMPOOL_DELAY MEMPOOL_FRACTION <<< "$BENCH_MEMPOOL"
+    TXGEN_SEND_ARGS+=(--mempool-delay "$MEMPOOL_DELAY")
+    if [ -n "$MEMPOOL_FRACTION" ]; then
+      TXGEN_SEND_ARGS+=(--mempool-fraction "$MEMPOOL_FRACTION")
+    fi
+  fi
+fi
 
 WARMUP="${BENCH_WARMUP_BLOCKS:-0}"
 BLOCKS="${BENCH_BLOCKS:?BENCH_BLOCKS must be set}"

.github/scripts/hive/no_sim_build.diff

@@ -1,5 +1,5 @@
 diff --git a/internal/libdocker/builder.go b/internal/libdocker/builder.go
-index 1c9714df..2aa493ca 100644
+index 20f5060..50a9194 100644
 --- a/internal/libdocker/builder.go
 +++ b/internal/libdocker/builder.go
 @@ -8,7 +8,6 @@ import (
@@ -13,32 +13,32 @@ index 1c9714df..2aa493ca 100644
 @@ -50,25 +49,8 @@ func (b *Builder) BuildClientImage(ctx context.Context, client libhive.ClientDes
 
  // BuildSimulatorImage builds a docker image of a simulator.
- func (b *Builder) BuildSimulatorImage(ctx context.Context, name string, buildArgs map[string]string) (string, error) {
--	dir := b.config.Inventory.SimulatorDirectory(name)
+ func (b *Builder) BuildSimulatorImage(ctx context.Context, sim libhive.SimulatorDesignator) (string, error) {
+-	dir := b.config.Inventory.SimulatorDirectory(sim.Simulator)
 -	buildContextPath := dir
--	buildDockerfile := "Dockerfile"
+-	buildDockerfile := sim.Dockerfile()
 -
 -	// build context dir of simulator can be overridden with "hive_context.txt" file containing the desired build path
 -	if contextPathBytes, err := os.ReadFile(filepath.Join(filepath.FromSlash(dir), "hive_context.txt")); err == nil {
 -		buildContextPath = filepath.Join(dir, strings.TrimSpace(string(contextPathBytes)))
 -		if strings.HasPrefix(buildContextPath, "../") {
 -			return "", fmt.Errorf("cannot access build directory outside of Hive root: %q", buildContextPath)
 -		}
--		if p, err := filepath.Rel(buildContextPath, filepath.Join(filepath.FromSlash(dir), "Dockerfile")); err != nil {
+-		if p, err := filepath.Rel(buildContextPath, filepath.Join(filepath.FromSlash(dir), sim.Dockerfile())); err != nil {
 -			return "", fmt.Errorf("failed to derive relative simulator Dockerfile path: %v", err)
 -		} else {
 -			buildDockerfile = p
 -		}
 -	}
- 	tag := fmt.Sprintf("hive/simulators/%s:latest", name)
--	err := b.buildImage(ctx, buildContextPath, buildDockerfile, tag, buildArgs)
+ 	tag := fmt.Sprintf("hive/simulators/%s:latest", sim.Simulator)
+-	err := b.buildImage(ctx, buildContextPath, buildDockerfile, tag, sim.BuildArgs)
 -	return tag, err
 +	return tag, nil
  }
 
  // BuildImage creates a container by archiving the given file system,
 diff --git a/internal/libdocker/proxy.go b/internal/libdocker/proxy.go
-index 386b4224..7545fda0 100644
+index 386b422..7545fda 100644
 --- a/internal/libdocker/proxy.go
 +++ b/internal/libdocker/proxy.go
 @@ -16,10 +16,6 @@ const hiveproxyTag = "hive/hiveproxy"

.github/scripts/verify_image_arch.sh

@@ -43,7 +43,6 @@ verify_image() {
 if [[ "$TARGETS" == *"nightly"* ]]; then
     verify_image "${REGISTRY}/reth:nightly" amd64 arm64
     verify_image "${REGISTRY}/reth:nightly-profiling" amd64
-    verify_image "${REGISTRY}/reth:nightly-edge-profiling" amd64
 else
     for tag in $(echo "$ETHEREUM_TAGS" | tr ',' ' '); do
         verify_image "$tag" amd64 arm64