Skip to content

test(rvc): add builder-flag eviction tests through the store writers (W4-E6b3) - #1012

Merged
rootwarp merged 2 commits into
developfrom
test/w4-e6b3-877
Nov 18, 2026
Merged

rootwarp merged 2 commits into
developfrom
test/w4-e6b3-877

Conversation

@rootwarp

@rootwarp rootwarp commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

Closes #877
Closes #654
Closes #696

This PR completes #654 AC10 and AC11, and #696 item 5, as #877 states.

The branch is on develop c82da1c5993d26169ee010418784dc3fcf7db1a1.

Summary

W4-E5 added cache eviction and new signatures for keys that return. W4-E6a added the registration call to the epoch boundary runner. This PR adds two integration tests for that behavior.

The runner removes a cached registration when its key leaves the builder set. When a key becomes builder-enabled again, the runner signs a new registration. The tests use the two runtime store writers:

  • One test uses ValidatorManagerAdapter::remove_validator to remove key A from the store. The keymanager DELETE uses this adapter.
  • The other test uses ProposerConfigApplier::apply to set builder_enabled for key A.

This PR changes only tests. It has no production change and no behavior change.

Changes

  • crates/rvc/tests/epoch_boundary_restart.rs:
    • New harness orchestrator_at_slot(slot, OrchestratorOpts):
      • orchestrator_at_slot starts DutyOrchestrator::run and the epoch boundary runner from OrchestratorDeps::for_test.
      • BuilderService sends its requests to MockBeaconNodeClient. The mock records each registration POST and its arrival time.
      • The ValidatorStore holds keys A and B in memory.
      • CountingSigner records each call to sign a registration. The wrapper records the registration before each call. SignerService handles the call. Then the wrapper adds the signature to the record.
      • PausedSlotClock gives the current slot. The duty phase offsets are zero.
      • A tracing layer records the epoch of each Epoch builder registration completed event of the runner.
    • New helper Orchestrated::registration_epoch(epoch):
      • It waits for the end of the registration step of epoch. The wait has a time limit of half a slot.
      • It gets each registration POST and each signer call after its last check.
      • Each POST must arrive in the first slot of the epoch. The helper reads the arrival time from MockBeaconNodeClient::call_stamps.
      • The epoch must have one POST.
    • New test test_builder_flag_off_via_keymanager_evicts_registration.
    • New test test_builder_flag_off_via_proposer_config_url_evicts_registration.
    • The module documentation describes the second harness.

Tests

Each test uses a single-thread Tokio runtime with paused time. Each call to tokio::time::advance moves virtual time by one slot. The new harness uses no files or sockets. At the first slot of each checked epoch, the helper waits at most half a slot for the end of the registration step. The reported assertion failures occur before this time limit.

Acceptance criterion Test
654-AC10, 696-5: after the keymanager DELETE, the next POST does not contain the removed key, and the service makes no new signature calls test_builder_flag_off_via_keymanager_evicts_registration
654-AC11, 696-5: while builder_enabled is false, the next POST does not contain the key, and the service makes no new signature calls. After the test sets builder_enabled to true, the service signs a new registration and the next POST contains the key test_builder_flag_off_via_proposer_config_url_evicts_registration

registration_epoch applies these conditions to each checked epoch of both tests:

  • Each registration POST must arrive in the first slot of its epoch.
  • The epoch must have one registration POST.

test_builder_flag_off_via_keymanager_evicts_registration:

  • The epoch E POST must contain keys A and B.
  • In epoch E, the service must make one call to sign a registration for each of keys A and B.
  • The epoch E POST must contain the registrations that the signer signs in epoch E.
  • After the adapter removes key A, the epoch E+1 POST must contain only the epoch E registration for key B.
  • The service must make no calls to sign a registration in epoch E+1.

test_builder_flag_off_via_proposer_config_url_evicts_registration:

  • The epoch E POST must contain keys A and B.
  • In epoch E, the service must make one call to sign a registration for each of keys A and B.
  • The epoch E POST must contain the registrations that the signer signs in epoch E.
  • While builder_enabled is false, the epoch E+1 POST must contain only the epoch E registration for key B.
  • The service must make no calls to sign a registration in epoch E+1.
  • After the test sets builder_enabled to true, the epoch E+2 POST must contain key A.
  • In epoch E+2, the service must make one call to sign a registration for key A.
  • The epoch E+2 POST must contain only the new registration for key A and the epoch E registration for key B.

Assertion results for hand mutants

W4-E5 and W4-E6a add the behavior, so the new tests pass on the base. I applied one change at a time to the production code. I ran the two new tests. I removed the change after each run with git checkout -- <file>. After each removal, git status --short showed no production file. No change is in a commit. The table identifies each assertion that fails. No result comes from a timeout, the half-slot time limit, a compile error, or a panic outside an assertion. Each new test fails an assertion under at least one mutant.

RSA M6 and Harper M7b are the review mutants of round 1. M1 to M5 are the five hand mutants of the first commit. This round ran M1 to M5 again on the new head.

Mutant File and symbol Change Test Result Assertion text
RSA M6 crates/builder/src/service.rs, BuilderService::submit_epoch_registrations After the usual POST, send a second POST with the cached registrations of keys that are not in the store. keymanager Assertion failure In epoch 4, the service must send one registration POST. The POSTs are [...]. (left: 2, right: 1)
Harper M7b crates/rvc/src/orchestrator/epoch_boundary.rs, EpochBoundaryTask::run let slot = slot.saturating_add(SLOTS_PER_EPOCH - 1); before the epoch selection keymanager Assertion failure Registration POST 2 must arrive in slot 128. The recorded slot is 98.
Harper M7b same same config URL Assertion failure Registration POST 2 must arrive in slot 128. The recorded slot is 98.
M1 keep-removed-key crates/builder/src/service.rs, BuilderService::submit_epoch_registrations No cache.retain. The POST contains each cached entry. keymanager Assertion failure After the keymanager DELETE, the epoch E+1 POST must not contain key A.
M1 keep-removed-key same same config URL Assertion failure While builder_enabled is false, the epoch E+1 POST must not contain key A.
M2 keep-flag-off-key crates/builder/src/service.rs, BuilderService::builder_keys No is_builder_enabled filter. config URL Assertion failure While builder_enabled is false, the epoch E+1 POST must not contain key A.
M3 resign-every-epoch crates/builder/src/service.rs, BuilderService::submit_epoch_registrations let unchanged = false; keymanager Assertion failure The service must make no calls to sign a registration in epoch E+1.
M3 resign-every-epoch same same config URL Assertion failure The service must make no calls to sign a registration in epoch E+1.
M4 no-eviction crates/builder/src/service.rs, BuilderService::submit_epoch_registrations No cache.retain. Key A keeps its cache entry while builder_enabled is false. config URL Assertion failure In epoch E+2, the service must make one call to sign a registration for key A.
M5 flag-on-ignored crates/rvc/src/background_tasks/config_url.rs, ValidatorConfigUpdate::to_store_update builder_proposals: self.builder_enabled.map(|_| false) config URL Assertion failure After the test sets builder_enabled to true, the epoch E+2 POST must contain key A.

The config URL test passes under RSA M6. Key A stays in the store in that test, so the mutant finds no key to send. The keymanager test passes under M2, M4, and M5, because it does not use the builder flag or the URL writer.

A mutant causes an assertion failure for each condition below:

  • RSA M6 detects a second registration POST in epoch E+1.
  • Harper M7b detects a registration POST before the first slot of its epoch.
  • M1 detects a key that stays in the POST after DELETE.
  • M1 and M2 detect a key that stays in the POST while its builder flag is false.
  • M3 detects a new signature call in epoch E+1.
  • M4 detects a missing signature call for key A in epoch E+2.
  • M5 detects a POST without key A in epoch E+2.

These runs do not reach the comparison with the unchanged epoch E registration of key B. Harper's round 1 review gives separate failure evidence for that comparison: a review mutant that changes the cached timestamp causes an assertion failure.

Clean test results

After the last restore, all seven tests in epoch_boundary_restart pass. Each new test also passes 50 repeat runs.

Gates

Final head: 920a7afacce843426a54f719a198c5bc886a07e4. Toolchain: 1.100.0. Each cargo gate used a private target directory.

Command Result
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets --all-features -- -D warnings PASS
cargo nextest run -p rvc --all-features --no-fail-fast PASS: 1880 passed, 7 skipped
cargo nextest run -p rvc --all-features --test epoch_boundary_restart --stress-count 50 test_builder_flag_off_via_keymanager_evicts_registration PASS: 50 of 50
cargo nextest run -p rvc --all-features --test epoch_boundary_restart --stress-count 50 test_builder_flag_off_via_proposer_config_url_evicts_registration PASS: 50 of 50
cargo deny check advisories sources PASS: advisories ok, sources ok
cargo mutants Not run: the diff has no production code. Hand mutants M1 to M5, RSA M6, and Harper M7b give the assertion results.
gitleaks 8.21.2, strict config, working tree (--no-git) PASS: no leaks found
scripts/gitleaks_range_scan.sh <strict config> c82da1c5993d26169ee010418784dc3fcf7db1a1 PASS: 2 commits scanned, no leaks found
git diff --name-only c82da1c5993d26169ee010418784dc3fcf7db1a1..HEAD PASS: only crates/rvc/tests/epoch_boundary_restart.rs
Extra: cargo nextest run -p rvc-architecture-tests --all-features --no-fail-fast PASS: 409 passed

Pre-PR self-check

  • Red check: each new acceptance test was shown failing with the fix removed (hand mutants M1 to M5, RSA M6, and Harper M7b; each result is an assertion failure).
  • Measurement claims ("no regression") come from a scenario where unfixed code scores worse. This PR makes no measurement claim.
  • Repo-wide grep for every changed symbol or pattern that the issue targets: remove_validator, ProposerConfigApplier::apply, builder_enabled, submit_epoch_registrations, the keymanager routes, Epoch builder registration completed, and MockBeaconNodeClient::call_stamps. The source checks appear in the list below.
  • Signing-root and hash_tree_root tests are KAT-anchored or kat_exempt with a reason. This PR adds no test with such a name.
  • No production code change: git diff --name-only c82da1c5993d26169ee010418784dc3fcf7db1a1..HEAD shows only crates/rvc/tests/epoch_boundary_restart.rs.
  • fmt, clippy and tests pass on toolchain 1.100.0 before the PR opens.

Source checks on c82da1c5:

  • The keymanager router has no route that sets builder_proposals.
  • ValidatorManagerAdapter::remove_validator calls ValidatorStore::remove_validator_durable.
  • ProposerConfigApplier::apply writes builder_enabled as builder_proposals.
  • BuilderService::submit_epoch_registrations evicts a key that left the builder set. The runner step builder_registration calls it in each epoch.
  • The runner logs Epoch builder registration completed with the epoch after submit_epoch_registrations returns Ok (crates/rvc/src/orchestrator/epoch_boundary.rs:748).
  • MockBeaconNodeClient::call_stamps records one MockMethod::RegisterValidators stamp, in virtual time, before each registration call.
  • The old BuilderService::register_validators method is absent from the base.

Decisions (intentional)

  • Keymanager DELETE. No keymanager route sets builder_proposals. The keymanager test uses ValidatorManagerAdapter::remove_validator to remove key A from the store. The keymanager DELETE uses this adapter. The test keeps its assigned name.
  • Orchestrator harness, not the bootstrap harness. The tests need access to the live store and the signer, so they start the orchestrator directly. They use the production store writers.
  • Tests only. W4-E5 added cache eviction and new signatures for keys that return. W4-E6a added the registration call to the epoch boundary runner. This PR adds two integration tests for that behavior.
  • Hand mutants. The old BuilderService::register_validators method is absent from the base, so the earlier failure at 64c8ccb7 cannot run here. Each new test fails a behavior assertion under at least one hand mutant.
  • Completion event as the wait signal. The tests wait for the existing Epoch builder registration completed event of the runner. A tracing layer in the test records it. This needs no production change.
  • Time limit of half a slot. The wait keeps this time limit as a guard. No reported failure comes from it.
  • Key order. The tests compare POST bodies in the order of the public keys. The acceptance criteria do not specify the key order in a POST.

Fix round 1

Review item Change
RSA B1 registration_epoch waits for the end of the registration step. Then it gets every POST and every signer call after its last check. Each checked epoch must have one POST. The tests check the complete body of each POST and the signer calls before the next store change. RSA M6 now causes an assertion failure in the keymanager test.
Harper B1 registration_epoch reads each MockMethod::RegisterValidators stamp from MockBeaconNodeClient::call_stamps. It uses the paused clock origin to get the slot, and compares it with the first slot of the epoch. Harper M7b now causes the slot assertion failure in both tests.
RSA N1 The test documentation, the assertion messages, and the CountingSigner text use the rewrites of the review.
RSA N2 The test conditions in this body are vertical lists.
RSA N3 The summary, the test text, and the mutant sections use the rewrites of the review.
Harper N1 The test documentation and the messages use the rewrites of the STE record.
Harper N2 The message of the fix commit contains the corrected description of commit 0100bbe6.
Harper N3 The summary has shorter sentences. The mutant sections use the new headings, and the matrix text states only what the runs show.

Release note

None. This PR has no behavior change. The W4-E6a note covers eviction.

Size

git diff --shortstat c82da1c5993d26169ee010418784dc3fcf7db1a1...HEAD: 1 file changed, 509 insertions(+), 5 deletions(-). Gross: 514 lines. Cargo.lock has no change.

Fix round 1 (git diff --shortstat 0100bbe653283486fe8e9198121145de00000586..HEAD): 1 file changed, 254 insertions(+), 121 deletions(-). Gross: 375 lines.

…(W4-E6b3)

Add the orchestrator_at_slot harness to epoch_boundary_restart.rs. It
starts DutyOrchestrator::run and the epoch boundary runner with
in-process doubles and paused time. A counting RegistrationSigner wraps
the real SignerService.

The keymanager DELETE test removes key A through
ValidatorManagerAdapter::remove_validator. The epoch E+1 POST must not
carry key A, and the signer must make no new signature.

The proposer-config URL test sets the builder flag of key A to off
through ProposerConfigApplier::apply. The epoch E+1 POST must not carry
key A, and the signer must make no new signature. Then the test sets the
flag to on. The signer must make one new signature, for key A, and the
epoch E+2 POST must carry key A.

The behavior shipped in W4-E5 and W4-E6a, thus these tests are a
backfill. Hand mutants of production code gave an assertion failure for
each test. The mutants are not in this commit.
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Report

Total: 94.70% (140581 / 148449 lines)

File Coverage Lines
src/bls_to_execution.rs 92.5% 246/266
src/deposit.rs 100.0% 295/295
src/existing_mnemonic.rs 86.3% 120/139
src/exit.rs 94.2% 259/275
src/fs_util.rs 98.5% 64/65
src/main.rs 0.0% 0/51
src/network.rs 100.0% 138/138
src/new_mnemonic.rs 92.7% 460/496
src/password.rs 92.6% 212/229
src/verify.rs 97.6% 82/84
src/main.rs 95.1% 887/933
src/cli.rs 90.2% 542/601
commands/doppelganger.rs 96.7% 327/338
commands/health.rs 100.0% 5/5
commands/prepare_exit.rs 90.1% 182/202
commands/signed_exit.rs 97.5% 1101/1129
commands/slashing.rs 93.5% 144/154
commands/submit_exit.rs 75.3% 73/97
commands/voluntary_exit.rs 63.1% 41/65
src/logging.rs 90.4% 641/709
src/main.rs 100.0% 44/44
bin/generate_architecture_md.rs 0.0% 0/8
src/lib.rs 83.5% 192/230
src/rust_source.rs 93.9% 1381/1471
src/admission.rs 86.4% 51/59
src/client.rs 88.3% 1884/2133
src/error.rs 100.0% 57/57
src/http_caps.rs 89.6% 43/48
src/prepare_check.rs 85.0% 306/360
src/retry.rs 100.0% 53/53
src/shape.rs 100.0% 30/30
src/ssz_deser.rs 95.3% 631/662
src/types.rs 95.7% 1882/1967
src/v4_wire.rs 100.0% 87/87
src/error.rs 100.0% 22/22
src/metrics.rs 98.9% 89/90
service/mod.rs 88.4% 1305/1477
src/traits.rs 73.1% 68/93
src/types.rs 100.0% 7/7
src/validation.rs 99.0% 191/193
src/admission.rs 98.5% 194/197
src/broadcast.rs 98.0% 146/149
src/error.rs 100.0% 13/13
src/fan_out.rs 94.8% 2581/2723
src/header_quorum.rs 100.0% 238/238
src/health.rs 86.9% 292/336
src/manager.rs 95.5% 8800/9214
src/metrics.rs 98.4% 186/189
src/mock.rs 93.6% 1132/1210
src/revocation_commit.rs 86.4% 51/59
src/sse.rs 87.3% 1440/1649
sse/frame.rs 98.1% 363/370
src/submit.rs 96.9% 557/575
src/sync_status.rs 98.3% 920/936
src/traits.rs 96.1% 395/411
src/types.rs 99.2% 245/247
src/service.rs 98.9% 2411/2439
src/traits.rs 73.9% 17/23
src/bls.rs 90.2% 220/244
src/composite_signer.rs 92.9% 469/505
src/eip2333.rs 97.1% 233/240
src/error.rs 100.0% 63/63
src/insecure.rs 100.0% 131/131
src/key_manager.rs 94.7% 791/835
src/keystore.rs 98.2% 1471/1498
src/mnemonic.rs 100.0% 91/91
src/signer_trait.rs 97.8% 89/91
src/signing.rs 97.7% 126/129
src/signing_root.rs 98.3% 456/464
src/test_utils.rs 100.0% 41/41
src/typed_signer.rs 94.4% 656/695
src/voluntary_exit_signing.rs 100.0% 129/129
src/enablement.rs 100.0% 26/26
src/epoch_clock.rs 97.7% 84/86
src/forward_window.rs 97.6% 1051/1077
src/lib.rs 100.0% 34/34
src/marker_file.rs 96.3% 129/134
src/restart_skip.rs 100.0% 34/34
src/duty.rs 100.0% 47/47
src/metrics.rs 100.0% 26/26
src/tracker.rs 98.6% 3401/3450
src/aggregation.rs 100.0% 307/307
src/aggregator.rs 96.4% 81/84
src/block.rs 99.4% 831/836
src/block_body.rs 94.3% 1508/1599
src/block_body_json.rs 96.4% 397/412
src/builder.rs 99.0% 96/97
src/builder_request_auth.rs 90.8% 108/119
canonical/gvr_hex.rs 100.0% 25/25
canonical/pubkey_hex.rs 98.4% 63/64
canonical/signing_root_hex.rs 100.0% 13/13
src/deposit.rs 98.2% 161/164
src/domains.rs 100.0% 64/64
src/duties.rs 98.4% 63/64
src/fork.rs 99.3% 435/438
src/hex_fixed.rs 100.0% 108/108
src/lib.rs 96.3% 131/136
src/networks.rs 100.0% 91/91
src/payload_attestation.rs 98.5% 64/65
src/proposer_preferences.rs 86.4% 19/22
src/serde_signature.rs 97.1% 68/70
src/signed_block_contents_json.rs 100.0% 154/154
src/ssz_helpers.rs 95.9% 396/413
src/sync_committee.rs 99.0% 307/310
src/tree_hash_utils.rs 93.4% 253/271
src/client.rs 91.0% 998/1097
src/lib.rs 100.0% 212/212
src/auth.rs 95.4% 334/350
src/error.rs 95.2% 258/271
src/handlers.rs 81.6% 678/831
src/lifecycle.rs 93.5% 1551/1658
src/server.rs 84.7% 400/472
src/traits.rs 97.6% 40/41
src/types.rs 96.4% 80/83
src/url_validator.rs 91.5% 193/211
src/definitions.rs 93.7% 177/189
src/lib.rs 95.3% 241/253
src/server.rs 95.3% 306/321
src/hex.rs 100.0% 39/39
src/logging.rs 99.3% 288/290
src/pubkey.rs 100.0% 62/62
src/span_names.rs 100.0% 70/70
src/client.rs 80.5% 124/154
src/wire.rs 95.1% 705/741
src/error.rs 100.0% 6/6
src/lib.rs 100.0% 10/10
src/network.rs 100.0% 43/43
sections/beacon.rs 0.0% 0/12
sections/builder.rs 98.2% 109/111
sections/builder_limits.rs 54.5% 12/22
sections/duties.rs 100.0% 55/55
sections/fork_schedule.rs 79.3% 46/58
sections/grpc_signer.rs 0.0% 0/8
sections/keymanager.rs 95.8% 114/119
sections/keys.rs 0.0% 0/9
sections/logfile.rs 62.5% 15/24
sections/mod.rs 99.3% 139/140
sections/monitoring.rs 46.2% 6/13
sections/network.rs 0.0% 0/8
sections/proposer_config.rs 62.5% 20/32
sections/safety.rs 61.9% 13/21
sections/secret_provider.rs 24.0% 6/25
sections/server.rs 0.0% 0/3
sections/slashing.rs 0.0% 0/6
sections/timing.rs 100.0% 94/94
sections/tracing.rs 61.0% 25/41
containers/attestation.rs 93.9% 123/131
containers/block_body.rs 97.9% 94/96
containers/body_leaves.rs 95.1% 214/225
containers/builder_requests.rs 95.6% 65/68
containers/envelope.rs 95.9% 94/98
containers/execution_payload.rs 94.4% 51/54
containers/leaves.rs 95.9% 162/169
containers/payload.rs 93.6% 103/110
containers/requests.rs 97.8% 44/45
src/kat.rs 100.0% 49/49
src/lib.rs 100.0% 71/71
src/merkle.rs 94.6% 88/93
src/roots.rs 96.5% 390/404
bin/gen_spec_kat.rs 81.3% 1091/1342
src/loader.rs 87.1% 196/225
src/lib.rs 92.0% 150/163
background_tasks/config_url.rs 95.8% 2917/3046
background_tasks/monitoring.rs 93.3% 280/300
src/beacon_adapter.rs 98.0% 484/494
bootstrap/beacon.rs 97.9% 512/523
bootstrap/enablement.rs 98.2% 1222/1244
bootstrap/executor.rs 95.6% 739/773
bootstrap/keys.rs 78.4% 333/425
bootstrap/mod.rs 100.0% 60/60
bootstrap/run.rs 93.4% 1559/1669
bootstrap/services.rs 97.8% 750/767
bootstrap/signer_probe.rs 90.7% 603/665
bootstrap/slashing.rs 91.1% 205/225
bootstrap/tasks.rs 94.6% 930/983
config/builder.rs 93.2% 2739/2940
config/error.rs 100.0% 136/136
config/network.rs 100.0% 118/118
config/types.rs 96.1% 3752/3906
src/deletion_denylist.rs 92.3% 372/403
src/detection_marker.rs 97.3% 1982/2038
src/detection_stop.rs 98.8% 251/254
src/index_resolver.rs 97.4% 1216/1248
src/key_admission.rs 92.7% 612/660
keymanager_adapters/config.rs 100.0% 86/86
keymanager_adapters/doppelganger.rs 56.5% 78/138
keymanager_adapters/import_window.rs 96.6% 1004/1039
keymanager_adapters/kdf_budget.rs 100.0% 173/173
keymanager_adapters/keystore.rs 82.4% 742/901
keymanager_adapters/notifier.rs 84.0% 21/25
keymanager_adapters/remote_keys.rs 96.3% 105/109
keymanager_adapters/slashing.rs 83.1% 143/172
keymanager_adapters/spawn.rs 91.5% 182/199
keymanager_adapters/validator.rs 52.9% 46/87
keymanager_adapters/voluntary_exit.rs 99.3% 398/401
src/liveness_loop.rs 98.2% 2910/2962
src/metrics.rs 99.6% 242/243
orchestrator/aggregation.rs 92.4% 2341/2534
orchestrator/attestation.rs 86.4% 1216/1407
block_proposal/mod.rs 98.2% 56/57
coordinator/mod.rs 93.7% 947/1011
orchestrator/dispatch.rs 89.8% 150/167
orchestrator/duty_management.rs 98.5% 705/716
orchestrator/epoch_boundary.rs 99.0% 3822/3862
orchestrator/error.rs 100.0% 31/31
orchestrator/head_events.rs 99.4% 334/336
orchestrator/payload_attestation.rs 97.1% 2311/2380
orchestrator/slot_anchor.rs 94.9% 111/117
orchestrator/slot_context.rs 94.7% 1297/1369
orchestrator/sync_committee.rs 95.0% 2313/2436
orchestrator/utils.rs 97.6% 486/498
validation/attestation_data.rs 100.0% 149/149
src/pubkey_index.rs 98.1% 414/422
src/quiesce.rs 98.7% 298/302
src/slashing_monitor.rs 94.3% 280/297
src/startup.rs 97.2% 650/669
src/test_metrics.rs 100.0% 49/49
src/format.rs 82.8% 82/99
src/key_source_manager.rs 96.4% 1363/1414
src/lib.rs 97.6% 82/84
src/metrics.rs 100.0% 74/74
src/refresh.rs 97.7% 1231/1260
src/lib.rs 78.9% 310/393
src/lib.rs 100.0% 20/20
audit/cn.rs 83.6% 127/152
audit/log.rs 63.3% 31/49
audit/mod.rs 100.0% 41/41
backend/basic.rs 93.7% 311/332
backend/signer_adapter.rs 50.0% 6/12
src/config.rs 99.8% 823/825
src/error.rs 56.2% 27/48
src/gate_shared_across_transports.rs 100.0% 97/97
src/grpc_common.rs 99.5% 370/372
grpc_tls/mod.rs 96.7% 118/122
grpc_tls/server_builder.rs 100.0% 12/12
accept_loop/mod.rs 94.4% 117/124
http_api/dispatch.rs 98.9% 366/370
http_api/mod.rs 97.8% 132/135
http_api/pubkey.rs 98.7% 77/78
http_api/request.rs 95.1% 77/81
http_api/response.rs 98.8% 241/244
routes/mod.rs 96.5% 193/200
tls_config/mod.rs 93.8% 61/65
src/insecure_startup.rs 100.0% 65/65
src/metrics.rs 98.5% 658/668
src/reload.rs 78.8% 216/274
server/backend.rs 97.9% 94/96
server/grpc.rs 88.2% 150/170
server/http.rs 90.0% 235/261
server/mod.rs 92.9% 1168/1257
server/slashing.rs 84.9% 225/265
src/service.rs 95.4% 1879/1969
src/sign_plan.rs 98.7% 833/844
slashing/config.rs 97.3% 73/75
src/tls_io.rs 100.0% 11/11
src/trace_ctx.rs 100.0% 41/41
src/core.rs 92.5% 1853/2004
src/error.rs 98.1% 154/157
src/fail_closed.rs 100.0% 3/3
src/gate.rs 92.1% 314/341
src/lib.rs 96.5% 4228/4381
src/locks.rs 85.1% 40/47
src/metrics.rs 92.6% 225/243
service_util/circuit_breaker.rs 100.0% 129/129
src/test_utils.rs 100.0% 112/112
src/traits.rs 75.0% 12/16
src/audit.rs 91.7% 22/24
db/interchange.rs 96.5% 1285/1331
db/migrations.rs 80.6% 412/511
db/mod.rs 98.3% 788/802
db/open.rs 87.9% 326/371
db/records.rs 99.2% 492/496
db/watermarks.rs 97.1% 726/748
src/error.rs 100.0% 7/7
src/evaluate.rs 97.9% 94/96
src/group_commit.rs 89.4% 388/434
src/history.rs 97.9% 466/476
src/metrics.rs 95.7% 90/94
src/reader.rs 76.4% 42/55
src/rules.rs 92.6% 390/421
src/scoped.rs 100.0% 203/203
src/stage.rs 83.4% 267/320
src/types.rs 92.0% 380/413
src/config.rs 100.0% 190/190
src/file_appender.rs 98.3% 119/121
src/format.rs 92.6% 386/417
src/health_probe.rs 97.7% 128/131
src/init.rs 96.1% 370/385
src/propagation.rs 94.1% 256/272
src/rolling.rs 99.4% 330/332
src/shutdown.rs 90.6% 29/32
src/trace_id.rs 98.1% 158/161
src/clock.rs 96.5% 556/576
src/error.rs 100.0% 12/12
src/lib.rs 100.0% 19/19
src/block_selection.rs 100.0% 84/84
src/config.rs 100.0% 15/15
src/error.rs 94.7% 198/209
src/store.rs 97.1% 5652/5823
src/hex_serde.rs 90.2% 46/51
src/lib.rs 97.9% 568/580

@rootwarp rootwarp left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1012 R1 security review

The review ends at 2026-11-19 02:27:26 KST (Asia/Seoul). Blake (RSA) reviews W4-E6b3, issue #877, Class T.

Verdict for 0100bbe6 — BLOCK

The review has 1 BLOCKING and 3 NIT findings.

Pins and lineage

Ref Commit SHA Tree SHA
Assigned head 0100bbe653283486fe8e9198121145de00000586 153fffc7a4ea81715a17586338ae9ddf74bf32e7
Assigned parent and PR base c82da1c5993d26169ee010418784dc3fcf7db1a1 5263875a091c63c9ac543b1fef87bdca1d501cd6

The head branch is test/w4-e6b3-877. The base branch is develop.
The PR is a draft and open at both checks.
The local and live head and base match the assigned pins at the start and before this report.
The detached worktree is clean at both checks and after each restore.

The head has one parent, the assigned develop commit.
git merge-base --is-ancestor exits 0. The symmetric range count is 0 1.
This history permits a fast-forward.

Git verifies the head's ED25519 SSH signature for rootwarp@gmail.com.
The stable patch ID is 79d354629baf07d2aec777bf6cb99b3947d024db.
The range has one file, 376 insertions, and five deletions. Cargo.lock has no changes.
The assignment has no earlier review head or rebase pair. Local start pins, live start pins, local final pins, live final pins, assigned diff.

B1 — BLOCKING. Check every POST in the epoch

At crates/rvc/tests/epoch_boundary_restart.rs:1083, the helper waits for at least the requested count.
At line 1091 it returns only one body by position.
T1 checks POST number two at lines 244–259. It leaves the rest of the mock's POST log unchecked.

M6 sends a second POST in epoch E+1 after the expected POST for B.
The mock accepts the extra POST with the cached registration for removed key A. T1 and T2 both pass.
A regression can send another registration for a removed key while the test passes.

Wait for the existing completion event for the epoch registration step before you check its effects.
Record every POST and every call to the registration signer for that epoch.
With this fixture, require one POST and check its complete body. Require the expected set of signer calls before the next store change.
The runner has a completion event at crates/rvc/src/orchestrator/epoch_boundary.rs:748. Source excerpts, M6 log.

N1 — NIT. Describe the calls that the tests make

Location Rewrite
epoch_boundary_restart.rs:21–25 and :973–974 “orchestrator_at_slot starts DutyOrchestrator::run and its epoch boundary runner. The harness uses a mock beacon node in the test process. The tests update the live ValidatorStore and count calls to the registration signer.”
epoch_boundary_restart.rs:223–227 and :242. PR body lines 11–14 and 104. “Use ValidatorManagerAdapter::remove_validator to remove key A from the store. The keymanager DELETE uses this adapter.” Use this assertion message at line 242. “The validator manager must remove key A from the store.”
epoch_boundary_restart.rs:238 and :279 “The epoch E POST must contain keys A and B.”
epoch_boundary_restart.rs:288 “While builder_enabled is false, the epoch E+1 POST must not contain key A.”
epoch_boundary_restart.rs:308 “After the test sets builder_enabled to true, the epoch E+2 POST must contain key A.”
epoch_boundary_restart.rs:253 and :293 “The service must make no calls to sign a registration in epoch E+1.”
epoch_boundary_restart.rs:313 “In epoch E+2, the service must make one call to sign a registration for key A.”
epoch_boundary_restart.rs:1116–1117 and :1128–1134. PR body line 25. “The wrapper records the public key before each call. SignerService handles the call.” For total, use “Return the number of calls to sign a registration.” For since, use “Skip the first from calls. Return the public key for each subsequent call.”

N2 — NIT. Use lists for test conditions

Replace the prose in the two Assertions cells at PR body lines 37–38 with vertical lists.

For T1, use this text.

  • The epoch E POST must contain keys A and B.
  • After the adapter removes key A, the epoch E+1 POST must contain only the epoch E registration for key B.
  • The service must make no calls to sign a registration in epoch E+1.

For T2, use this text.

  • The epoch E POST must contain keys A and B.
  • While builder_enabled is false, the epoch E+1 POST must contain only the epoch E registration for key B.
  • The service must make no calls to sign a registration in epoch E+1.
  • After the test sets builder_enabled to true, the epoch E+2 POST must contain key A.
  • In epoch E+2, the service must make one call to sign a registration for key A.

N3 — NIT. Name the test results directly

Location in the PR body Rewrite
Lines 11 and 106 “W4-E5 added cache eviction and new signatures for keys that return. W4-E6a added the registration call to the epoch boundary runner. This PR adds two integration tests for that behavior.”
Line 33 “Advance virtual time by one slot per call. At the first slot of the epoch, wait at most half a slot for the registration POST. The reported assertion failures occur before this deadline.”
Lines 40–42 Use “Assertion results for hand mutants” as the heading. Use “Each new test fails an assertion under at least one mutant.”
Lines 44–54 Use “Assertion failure” in each affected Result cell. Use “A mutant causes an assertion failure for each condition below.”
Lines 62–64 Use “Clean test results” as the heading. Use “After the last restore, all seven tests pass. Each new test also passes 50 repeat runs.”
Lines 87 and 107 Use “Each new test fails a behavior assertion under at least one hand mutant.”

The assertions give a clear meaning for the cited prose. The review applies Issue 9 to new or changed text. Test language notes, live PR body, issue #877.

Store writers, signatures, and secret material

T1 is test_builder_flag_off_via_keymanager_evicts_registration. T2 is test_builder_flag_off_via_proposer_config_url_evicts_registration.

T1 calls the validator manager at crates/rvc/tests/epoch_boundary_restart.rs:241.
The adapter calls remove_validator_durable at crates/rvc/src/keymanager_adapters/validator.rs:52.
The store removes the row in memory for this fixture. The keymanager router has no route that sets builder_proposals.

T2 calls ProposerConfigApplier::apply at crates/rvc/tests/epoch_boundary_restart.rs:282 and :302.
The mapper writes builder_enabled as builder_proposals at crates/rvc/src/background_tasks/config_url.rs:194.
The applier passes the update to the store at line 832.

The pre-Gloas runner calls submit_epoch_registrations at crates/rvc/src/orchestrator/epoch_boundary.rs:746.
At crates/builder/src/service.rs:316–321, the builder set excludes removed keys and keys with the builder flag off.
The service evicts other cache entries at line 238. It signs eligible entries before it sends the POST at line 282.

The test keeps both keys in a local key manager. The signer can therefore report attempts that the builder service must prevent.
The wrapper records the public key before it delegates to the real signer at crates/rvc/tests/epoch_boundary_restart.rs:1147–1149.
On the clean tree, neither test calls the registration signer for A while A is absent or the flag is false.
Both tests check for zero new calls in epoch E+1. T2 requires one new call for A after the flag changes to true.
The fixture supplies no proposer duties and uses a fork schedule without Gloas. Source excerpts, test map.

The diff changes only crates/rvc/tests/epoch_boundary_restart.rs. It adds no production change or test hook in production code.
The tests use the existing test-utils fixture at crates/rvc/src/orchestrator/coordinator/mod.rs:315–317.
The new keys are two synthetic arrays of repeated bytes at test lines 106–108. The new code adds no credentials or tokens.
Gitleaks 8.21.2 finds no leaks in the changed file or the one-commit range. Both scans use the strict CI config.
The first scan fetches public tooling directly. The repeat fetches both pinned inputs through read-only gh. Production check, first Gitleaks log, repeat Gitleaks log.

Hand mutants

Each run applies one source edit and restores the full file from HEAD. Each restore yields an empty git status --porcelain.
The table names the assertion locations in crates/rvc/tests/epoch_boundary_restart.rs.
Command time includes the wait for the lock. Test time comes from nextest.

Mutant Change Result Test evidence Command time Files
M1 Skip cache eviction. Send all cached entries. ASSERTION_KILL T1 fails at :246:5 (0.014 s). “The epoch E+1 POST must not carry key A after the keymanager DELETE.” T2 fails at :286:5 (0.014 s). “The epoch E+1 POST must not carry key A after builder_enabled false.” 881.067 s Log, diff, JSON, restore
M2 Remove the builder flag filter. ASSERTION_KILL T1 passes (0.010 s). T2 fails at :286:5 (0.011 s). “The epoch E+1 POST must not carry key A after builder_enabled false.” 14.920 s Log, diff, JSON, restore
M3 Sign each eligible key each epoch. ASSERTION_KILL T1 fails at :250:5 (0.012 s). “The signer must make no new registration signature in epoch E+1.” T2 fails at :290:5 (0.012 s). “The signer must make no new registration signature in epoch E+1.” 37.340 s Log, diff, JSON, restore
M4 Skip cache eviction. Keep the POST filter. ASSERTION_KILL T1 passes (0.011 s). T2 fails at :310:5 (0.011 s). “The signer must make one new registration signature in epoch E+2, for key A.” 36.187 s Log, diff, JSON, restore
M5 Write false for both builder flag values. ASSERTION_KILL T1 passes (0.012 s). T2 fails at :306:5 (0.015 s). “The epoch E+2 POST must carry key A after builder_enabled true.” 36.406 s Log, diff, JSON, restore
M6 Send a cached entry for a removed key in a separate POST. SURVIVED T1 passes (0.011 s). T2 passes (0.012 s). 36.617 s Log, diff, JSON, restore

M1 through M5 are assertion kills. No result depends on a build error, timeout, or shutdown guard.
T1 passes M2 because the store removes A from the enabled list.
T1 passes M4 because the POST still uses that list and the test does not add A again.
T1 passes M5 because the test does not use the config URL writer.
The five applied diffs match the supplied diffs byte for byte. Diff comparison.

M6 survives. The log shows one extra batch with one removed entry in epoch 4 (E+1) and zero failures.
This changes the mock's received POSTs and is a non-equivalent mutant.
T2 leaves A in the store, so M6 has no effect on its result.
M6 has no assertion failure. It demonstrates B1.
The log parser now keeps assertion details when nextest repeats a result in its summary.

Clean runs and commands

Both clean runs pass 7 of 7 tests in the assigned file. Each run includes T1 and T2. green before, green after.

T1 and T2 each pass all 50 repeat runs on the clean tree. Repeat test log.

The toolchain reports rustc 1.99.0 (b940084d7 2026-09-28).
The runner sets PROTOC=/usr/local/bin/protoc and CARGO_INCREMENTAL=0.
Every cargo call uses the shared lock and the private target. The lowest guard readings are 48 GB and 384,334 free inodes.
This review ran only targeted nextest. It ran no full-package or workspace-wide nextest suite, Clippy, Docker, or cargo deny. Rust version, lock runner, cargo environment, nextest config, test runner, result parser, progress.

PATH=/home/box/.cargo/bin:/usr/local/bin:$PATH rustc +1.99.0 --version
df -BG /
df -i /
review_root=/workspace/rvc/pr1012-rsa-r1
flock -x /tmp/harper-build.lock timeout 600 "$review_root/scratch/lc.sh" \
  /tmp/pr1012-rsa-r1-target nextest run \
  --config-file "$review_root/scratch/nextest.toml" \
  -p rvc --all-features --test epoch_boundary_restart --no-fail-fast

The first clean run uses a 1,200 s time limit. The mutant commands append -E 'test(/^test_builder_flag_off_via_/)'.
M6 also appends --success-output final. The repeat run uses the filter and --stress-count 50.

CI on the assigned head

The final GitHub check ends at 2026-11-19 02:27:26 KST.
The PR rollup uses 0100bbe653283486fe8e9198121145de00000586 and shows 9 success.

Actions run ID Workflow Status Conclusion headSha
44353609562 CI completed success 0100bbe653283486fe8e9198121145de00000586

GitGuardian Security Checks (134975671749) reports success.

CI passes on this head.

PR check rollup, branch Actions runs, commit checks, CI commands.

Model: gpt-6.2-sol, effort high

@rootwarp rootwarp left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1012 (W4-E6b3 / #877) — Harper Formal R1

Overall verdict: REQUEST_CHANGES (1 BLOCKING, 3 NIT). Both new tests pass on the pinned head. The five supplied production mutants, and one extra payload mutant, fail by assertion. A further production change shows that both tests accept a registration POST from an earlier epoch.

  • Head: 0100bbe653283486fe8e9198121145de00000586 (re-checked at FINAL against gh pr view 1012 and the review worktree; unchanged)
  • Tree: 153fffc7a4ea81715a17586338ae9ddf74bf32e7
  • Develop / parent: c82da1c5993d26169ee010418784dc3fcf7db1a1 — 1 ahead / 0 behind (rev-list --left-right --count = 0 1). Live develop stayed at this SHA during the review.
  • Branch test/w4-e6b3-877. DRAFT, MERGEABLE, OPEN. One signed commit sits directly on develop. This is the expected R1 shape. It is not a finding. Author = committer = Joonkyo Kim <rootwarp@gmail.com>; no trailers. The commit has a Good "git" signature for that address with ED25519 key SHA256:66/BtMChdDUV09Eu2LXyZuu5NU8naVYDPt2RRruMcWo. Authored and committed 2026-11-19 01:50:20 KST (Asia/Seoul).

Diff versus the assigned parent: 1 file, crates/rvc/tests/epoch_boundary_restart.rs, +376/−5. Cargo.lock is unchanged. Production code is not in the diff.

This review used one Codex run: gpt-6.2-sol with model_reasoning_effort=high. The wrapper started at 2026-11-19 02:05:13 KST (Asia/Seoul) and exited 0 at 02:37:04 KST.

BLOCKING

B1 — The tests accept a POST from an earlier epoch

registration_post waits until the POST count reaches the requested number, then reads that body. The watch can already hold that count when the wait starts. The tests do not check the arrival slot, so they still treat an early POST as the epoch E+1 (or E+2) POST.

M7 adds one to the processed slot before epoch selection. Both tests still pass (exit 0), and that run does not prove an early arrival. M7b adds SLOTS_PER_EPOCH - 1 (31). Diagnostic lines in that run show:

  • The registration step for epoch 4 (E+1) ended in slot 99. Epoch 4 starts at slot 128.
  • In the proposer-config test, the registration step for epoch 5 (E+2) ended in slot 131. Epoch 5 starts at slot 160.

Both tests passed (exit 0). The half-slot wait guard did not fire. This is a claim-bearing, non-equivalent survivor.

The tests must read the matching MockMethod::RegisterValidators timestamp from MockBeaconNodeClient::call_stamps, convert it to a slot with the paused clock origin, and compare it with the expected slot. Use: "Registration POST {count} must arrive in slot {expected_slot}. The recorded slot is {actual_slot}." Keep the half-slot time limit for the wait.

Evidence: pr1012-r1/scratch/harper-logs/mutants/M7b-register-early-confirmation/.

NIT

N1 — Rewrite new test prose

Some new assertion and helper text uses words outside the approved meanings, for example "carry" for POST contents. The meanings are clear. Concrete rewrites are in pr1012-r1/scratch/harper-logs/ste-review.md.

N2 — Rewrite the commit message

The commit message uses "wraps," "shipped," and "backfill." The meaning is clear. A full replacement is in the STE record.

N3 — Shorten the PR body and limit the mutant claim

The first summary sentence has 28 words. The proof heading uses informal terms. The matrix sentence states more than the five supplied runs show. M6 supplies separate evidence for the reused key-B payload. Concrete rewrites are in the STE record.

Tests, mutants, and CI

The unmutated head passes the targeted epoch_boundary_restart suite under rustc 1.99.1 (58cf8f9c3 2026-10-13): 7 passed, 0 skipped, exit 0. The selection includes both new tests. The shared 1.99.1 rustup path was missing, so Harper used a private official 1.99.1 toolchain (commit 58cf8f9c35b355199727627d07ae34a4e2846d67) and did not change /home/box/.rustup.

Hand mutants of production code (original patches applied with no adaptation for M1–M5):

  • M1 keep-removed-key: assertion kill, exit 100. T1 at :246:5, T2 at :286:5.
  • M2 keep-flag-off-key: assertion kill, exit 100. T1 passes (DELETE removes A from list_enabled_pubkeys). T2 fails at :286:5.
  • M3 resign-every-epoch: assertion kill, exit 100. T1 at :250:5, T2 at :290:5.
  • M4 no-eviction: assertion kill, exit 100. T1 passes (the send still selects current builder keys). T2 fails at :310:5.
  • M5 flag-on-ignored: assertion kill, exit 100. T1 passes (DELETE path). T2 fails at :306:5.
  • M6 change-cached-timestamp: assertion kill, exit 100. T1 at :255:5, T2 at :295:5. This extra mutant covers the reused key-B payload claim.
  • M7 / M7b: both tests pass, exit 0. See B1. No half-slot guard fired in any run.

Generated cargo mutants --in-diff (test-only diff): exit 0, No mutants to filter. Zero production mutants, zero assertion kills, zero survivors, zero compile failures. The hand mutants are the strength evidence.

The new orchestrator_at_slot harness uses one Tokio thread with paused time, deterministic one-slot advances, a mock beacon node, a store without a config path, and an in-memory slashing database. It uses no network connections, file writes, or wall-clock sleeps. T1 writes through ValidatorManagerAdapter::remove_validator. T2 writes twice through production ProposerConfigApplier::apply on the same applier. Harper agrees with the plan choices: builder_keys supplies the fixture keys; the integration test uses a local counting signer because the coordinator helper is private; the old inherent register_validators path is gone, and the mutants use submit_epoch_registrations.

Same-head CI run 44353609562 is success on 0100bbe653283486fe8e9198121145de00000586. Harper retrieved the metadata and logs through bash -ic with gh, and rechecked them at FINAL. The Test job reports 7,781 passed, one slow, and ten skipped, in 146.945 s, at 2026-11-19 02:08:11 KST (Asia/Seoul) (job 134975670107). Doctests report 29 passed and 6 ignored. The Check job runs fmt and workspace Clippy with all features (job 134975670640). Architecture Gates (job 134975670769), Coverage, Scripts, Secret Scan, Spec Vectors, Trace continuity, and GitGuardian also pass. All nine required checks pass. CI uses rustc 1.100.0. This review does not rerun the full workspace suite or Clippy locally.

Every cargo command used /workspace/rvc/bin/locked-cargo, flock -x /tmp/harper-build.lock, rustc 1.99.1, CARGO_INCREMENTAL=0, PROTOC=/usr/local/bin/protoc, CARGO_BUILD_JOBS=1, and private /tmp/harper-pr1012-r1-target. The review did not use /workspace/rvc/target-shared. The private target is absent after the run. One cargo job ran at a time. Harper waited for the shared lock and left other processes alone. The worktree is clean at the pinned head and tree. No comment, review, push, merge, or source fix was made. Harper verified the pins, the assertion-only kills, the M7b survivor, the STE items, and the CI counts, then wrote this FINAL.

Copies: lead/handoff/pr1012, pr1012-r1/out, /home/box/rvc-review-archive/pr1012. First line <!-- prepared by Harper for Ash -->. Not posted. Did not message Ash.

This text replaces the description in commit 0100bbe:

Add orchestrator_at_slot to epoch_boundary_restart.rs. The harness runs
the orchestrator and its epoch boundary runner with paused time. It uses
a mock beacon node and records calls to the local signer.

The keymanager DELETE test removes key A with
ValidatorManagerAdapter::remove_validator. The epoch E+1 POST must
contain only the epoch E registration for key B. The signer must not
sign a new registration in epoch E+1.

The proposer configuration test sets builder_enabled to false for key A
with ProposerConfigApplier::apply. The same epoch E+1 conditions apply.
The test then sets builder_enabled to true for key A. In epoch E+2, the
signer must sign one new registration for key A. The epoch E+2 POST must
contain a registration for key A.

W4-E5 and W4-E6a add this behavior. Temporary changes to the production
code cause assertion failures in each new test. Commit 0100bbe contains
the tests.

This commit makes these changes to the two tests:

A tracing layer records the "Epoch builder registration completed" event
of the epoch boundary runner. The new helper registration_epoch waits
for this event for one epoch. The wait has a time limit of half a slot.
Then the helper gets each registration POST and each signer call after
its last check.

Each POST must arrive in the first slot of its epoch. The helper reads
the arrival time from MockBeaconNodeClient::call_stamps. It uses the
paused clock origin to change the time to a slot. Each epoch must have
one POST.

The signer wrapper records the registration and the signature of each
call. The epoch E POST must contain the registrations that the signer
signs in epoch E. The epoch E+1 POST must contain only the epoch E
registration for key B. The epoch E+2 POST must contain only the new
registration for key A and the epoch E registration for key B.

A temporary production change sends a second POST in epoch E+1 with the
cached registration for removed key A. The keymanager test then fails on
the one-POST assertion. A second temporary change makes the runner use
an epoch 31 slots early. Both tests then fail on the slot assertion. The
test text now uses ASD-STE100 wording.

@rootwarp rootwarp left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1012 (W4-E6b3 / #877) — Harper fix1 recheck

Overall verdict: CLEARED (B1 PASS; N1 PASS, N2 PASS, N3 PASS). No new findings outside the assigned items.

  • Head: 920a7afacce843426a54f719a198c5bc886a07e4 (re-checked at FINAL against gh pr view 1012 and the review worktree; unchanged)
  • Tree: 911477cb3bf504b2ac541df3cfe621c4fefa43c0
  • Develop / parent: c82da1c5993d26169ee010418784dc3fcf7db1a1 — 2 ahead / 0 behind (rev-list --left-right --count = 0 2). Live develop stayed at this SHA during the recheck.
  • R1 head: 0100bbe653283486fe8e9198121145de00000586
  • Branch test/w4-e6b3-877. DRAFT, MERGEABLE, OPEN. The reviewed commit 0100bbe6 remains, plus one additive fix commit 920a7afa. This shape is expected. It is not a finding. Author = committer = Joonkyo Kim <rootwarp@gmail.com>; no trailers. Both commits have a Good "git" signature for that address with ED25519 key SHA256:66/BtMChdDUV09Eu2LXyZuu5NU8naVYDPt2RRruMcWo. Recheck scope: Harper R1 B1 and N1–N3 only.

An independent git range-diff of c82da1c5..0100bbe6 against c82da1c5..920a7afa matches the supplied range-diff byte for byte. Diff of the fix commit: 1 file, crates/rvc/tests/epoch_boundary_restart.rs, +254/−121. Cargo.lock is unchanged.

This recheck used one Codex run: gpt-6.2-sol with model_reasoning_effort=high. The wrapper started at 2026-11-19 03:16:19 KST (Asia/Seoul) and exited 0 at 03:37:32 KST.

Item verdicts

B1 — PASS

Both new tests call registration_epoch, which waits for the registration step to end, reads new MockMethod::RegisterValidators stamps from MockBeaconNodeClient::call_stamps, converts each stamp to a slot with the paused clock origin, and compares that slot with the first slot of the epoch. The assertion message is “Registration POST {count} must arrive in slot {expected_slot}. The recorded slot is {actual_slot}.”

The original R1 M7b patch applies with both hunks and no adaptation. It adds SLOTS_PER_EPOCH - 1 to the processed slot before epoch selection. Both targeted tests fail by assertion at crates/rvc/tests/epoch_boundary_restart.rs:1176:13: “Registration POST 2 must arrive in slot 128. The recorded slot is 98.” The counts are left: 98 and right: 128. Nextest exits 100: 0 passed, 2 failed, 5 skipped. The half-slot wait guard does not fire. This is an assertion kill, not a timeout. The tree was restored clean after the run.

Evidence: pr1012-fix1/scratch/harper-logs/m7b/.

N1 — PASS

The named test-prose sites now use the R1 STE rewrites or equivalent wording. The messages use “contain” for POST contents and “sign” for signer actions. The slot assertion uses the R1 wording. The site map is in pr1012-fix1/scratch/harper-logs/ste/recheck.md.

N2 — PASS

The new fix commit 920a7afa uses the replacement description with STE wording. The reviewed 0100bbe6 message stays, as required.

N3 — PASS

The live PR body replaces the 28-word summary with three shorter sentences. The proof heading is “Assertion results for hand mutants.” The matrix identifies the five supplied runs, and it describes M6 as separate evidence.

Tests and CI

The unmutated head passes the targeted epoch_boundary_restart suite under rustc 1.99.1 (58cf8f9c3 2026-10-13): 7 passed, 0 skipped, exit 0. The selection includes both new tests. The shared 1.99.1 rustup path was missing, so Harper used a private official 1.99.1 toolchain (commit 58cf8f9c35b355199727627d07ae34a4e2846d67) and did not change /home/box/.rustup. This recheck did not run cargo mutants --in-diff.

Same-head CI run 44367055012 is success on 920a7afacce843426a54f719a198c5bc886a07e4. Harper retrieved the metadata and logs through bash -ic with gh, and rechecked them at FINAL. The Test job reports 7,781 passed, one slow, and ten skipped, in 145.556 s, at 2026-11-19 03:19:29 KST (Asia/Seoul) (job 135023517583). Both new tests pass in that job. Doctests report 29 passed and 6 ignored. The Check job runs fmt and workspace Clippy with all features (job 135023517352). Architecture Gates (job 135023516838), Coverage, Scripts, Secret Scan, Spec Vectors, Trace continuity, and GitGuardian also pass. All nine required checks pass. CI uses rustc 1.100.0. This review does not rerun the full workspace suite or Clippy locally.

Every cargo command used /workspace/rvc/bin/locked-cargo, flock -x /tmp/harper-build.lock, rustc 1.99.1, CARGO_INCREMENTAL=0, PROTOC=/usr/local/bin/protoc, CARGO_BUILD_JOBS=1, and private /tmp/harper-pr1012-fix1-target. The review did not use /workspace/rvc/target-shared. The private target is absent after the run. One cargo job ran at a time. Harper waited for the shared lock and left other processes alone. The worktree is clean at the pinned head and tree. No comment, review, push, merge, or source fix was made. Harper verified the pins, the M7b assertion kill, the STE items, and the CI counts, then wrote this FINAL.

Copies: lead/handoff/pr1012, pr1012-fix1/out, /home/box/rvc-review-archive/pr1012. First line <!-- prepared by Harper for Ash -->. Not posted. Did not message Ash.

@rootwarp
rootwarp marked this pull request as ready for review November 18, 2026 18:44
@rootwarp
rootwarp merged commit 920a7af into develop Nov 18, 2026
10 checks passed
@rootwarp
rootwarp deleted the test/w4-e6b3-877 branch November 18, 2026 18:44

@rootwarp rootwarp left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1012 fix1 security recheck

Blake (RSA) completes the review at 2026-11-19 03:38:54 KST (Asia/Seoul).

Verdict for 920a7afa — SECURITY-CLEAR

The review covers W4-E6b3, issue #877, Class T. B1 is FIXED. The two remaining nits do not block this verdict.

Pins and lineage

Ref Commit SHA Tree SHA
Assigned head 920a7afacce843426a54f719a198c5bc886a07e4 911477cb3bf504b2ac541df3cfe621c4fefa43c0
Head parent and R1 head 0100bbe653283486fe8e9198121145de00000586 153fffc7a4ea81715a17586338ae9ddf74bf32e7
Develop base c82da1c5993d26169ee010418784dc3fcf7db1a1 5263875a091c63c9ac543b1fef87bdca1d501cd6

PR #1012 is open and in draft. The head branch is test/w4-e6b3-877. The base branch is develop.
The local and live head and base match the pins at the start and before this report.
The detached worktree is clean at both checks.
The range 0100bbe6..920a7afa adds one commit. The range c82da1c5..920a7afa contains two commits.
git merge-base --is-ancestor exits 0. git rev-list --left-right --count returns 0 and 2, separated by a tab.
Develop is an ancestor of the head. The history permits a fast-forward.
Git verifies the ED25519 signature of 920a7afa for rootwarp@gmail.com. Initial local pins, initial live pins, final local pins, final live pins, signature verification.

Range diff and patch IDs

The regenerated range diff matches both Glen copies byte for byte.
It shows 0100bbe6 = 0100bbe6 and adds 920a7afa as entry 2. The first commit is identical to R1.
The fix changes only crates/rvc/tests/epoch_boundary_restart.rs. Git reports 254 insertions and 121 deletions.
The Cargo.lock diff is empty. The change is test-only. Range diff, range comparison, stable patch IDs, R1 patch ID comparison, file counts, source delta.

Commit Stable patch ID
0100bbe6 79d354629baf07d2aec777bf6cb99b3947d024db
920a7afa 3335715c39720ee4b92c9421a74e35806ccfe270

B1 — FIXED. Check every POST in the epoch

T1 is test_builder_flag_off_via_keymanager_evicts_registration. T2 is test_builder_flag_off_via_proposer_config_url_evicts_registration.

The service awaits the signer calls and the POST batches at crates/builder/src/service.rs:260 and :282.
The batch loop awaits the mock at line 401. The mock records each arrival and each body at crates/bn-manager/src/mock.rs:1280–1281.
The runner emits the completion event at crates/rvc/src/orchestrator/epoch_boundary.rs:748 after the service returns success.
RegistrationSteps records the epoch at crates/rvc/tests/epoch_boundary_restart.rs:1210–1216.

registration_epoch waits for this event at lines 1163–1169. It reads every unchecked POST at line 1170.
It checks each arrival slot at lines 1171–1180 and requires one POST at line 1182.
It reads every unchecked signer call at lines 1188–1191.
CountingSigner records each registration before it calls the signer. It adds each successful signature at lines 1271–1281. Test source, registration call sites.

T1 and T2 check each epoch before the next store change. The body comparisons include every message field and the signature.

  • In epoch E, both tests require one signer call per key and a POST with both signed entries at lines 318–333.
  • In epoch E+1, both tests require zero signer calls and the complete epoch E entry for key B at lines 338–348.
  • In T2 epoch E+2, the test requires one signer call for A at lines 304–308. It compares the full POST at lines 309–314. The POST must contain the new entry for A and the epoch E entry for B.

T1 changes the store at line 247. T2 changes it at lines 286 and 296.
The checks catch a second POST that contains a cached registration for a removed key. Epoch E checks, epoch E+1 checks, T2 epoch E+2 checks, source notes.

M6 — Assertion kill

M6 sends a cached registration for a removed key in a separate POST. The applied diff matches the R1 diff byte for byte.
The mock reports one extra batch in epoch 4 with one removed entry and zero send failures.

T1 fails in 0.026 s at crates/rvc/tests/epoch_boundary_restart.rs:1182:9. The assertion assert_eq!(posts.len(), 1) reports left: 2 and right: 1.
The message starts, “In epoch 4, the service must send one registration POST.” The log also includes both bodies.
This result is an assertion kill. The completion guard passes before the count check.
T2 passes in 0.031 s. It keeps A in the store.

Nextest exits 100 and reports 7 tests run: 6 passed, 1 failed, 0 skipped.
The command takes 982.877 s, which includes the wait for the lock.
Git restores the full source file from HEAD. The next git status --porcelain gives no output. M6 log, applied diff, JSON result, restore log, diff comparison.

N1 — NOT FIXED. Describe the calls that the tests make

The module and harness descriptions name the orchestrator and the mock at epoch_boundary_restart.rs:21–27 and :1050–1054.
The T1 description names the adapter at lines 227–229. The removal assertion names the validator manager at line 248.
The shared POST and signer assertions use the R1 wording at lines 322 and 342. T2 uses it at lines 292, 302, and 307.
CountingSigner::calls replaces total and since. This helper gives access to the signer records at lines 1257–1260.
The clean log reports PASS for T1 and T2. Test source, clean run.

Two changed passages still need more precise words.

  • Line 254 names a keymanager DELETE. T1 calls the adapter directly at line 247. Use “After the adapter removes key A, the epoch E+1 POST must not contain key A.”
  • The signer description at lines 1249–1251 and PR body line 27 says the wrapper adds the signature. At lines 1278–1281, this depends on a successful call. Use “If SignerService returns a signature, the wrapper adds it to the record.”

These changes keep the context and the requirements clear. They concern the wording in the R1 nit. T1 assertion, signer source, live PR body.

N2 — FIXED. Use lists for test conditions

The live PR body gives T1 assertions as a vertical list at lines 55–59. It gives T2 assertions as a vertical list at lines 63–70.
The test descriptions also use vertical lists at epoch_boundary_restart.rs:232–235 and :264–273.
The comparison with Glen's PR body copy exits 0. The clean log reports PASS for both tests. T1 list, T2 list, body comparison, test descriptions, clean run.

N3 — NOT FIXED. Name the test results directly

The live PR body uses the R1 summary at lines 11 and 150. Line 41 describes the half-slot limit and the assertion failures.
Line 72 uses “Assertion results for hand mutants.” The Result cells at lines 80–89 say “Assertion failure.”
Line 105 uses “Clean test results.” Line 107 gives the clean test count.
Line 151 uses the direct hand-mutant sentence. The independent M6 and clean runs report the assertion failure and pass counts for T1 and T2. Live PR body, M6 log, clean run.

The self-check at line 129 still says the tests fail “with the fix removed.” The runs use temporary changes to the production code.
Use “Each new test fails a behavior assertion under at least one hand mutant.” This sentence names the test result and its condition. Self-check text.

Clean run and commands

After the restore, nextest exits 0 and reports 7 tests run: 7 passed, 0 skipped. Both T1 and T2 pass.
The worktree is clean after the run. Clean run.

The toolchain reports rustc 1.99.0 (b940084d7 2026-09-28).
The cargo wrapper sets PROTOC=/usr/local/bin/protoc and CARGO_INCREMENTAL=0.
Both calls use the shared flock and the private target. The lowest disk and inode guard readings are 49 GB and 409,775 free inodes.
The review runs this test file twice, once with M6 and once after the restore.
The review runs zero full nextest suites, Clippy commands, or Docker commands.
A shell quoting error omits paths from the first M6 progress entries. The journal now includes those paths. Rust version, cargo wrapper, lock runner, test runner, nextest config, progress.

The following command uses the M6 limit of 1,200 s. The clean call uses 600 s and the same test selection.

PATH=/home/box/.cargo/bin:/usr/local/bin:$PATH rustc +1.99.0 --version
df -BG /
df -i /
review_root=/workspace/rvc/pr1012-rsa-fix1
flock -x /tmp/harper-build.lock timeout 1200 "$review_root/scratch/lc.sh" \
  /tmp/pr1012-rsa-fix1-target nextest run \
  --config-file "$review_root/scratch/nextest.toml" \
  -p rvc --all-features --test epoch_boundary_restart --no-fail-fast

CI on the assigned head

The final CI and pin checks run at 2026-11-19 03:38:54 KST.
The PR rollup shows nine completed checks with conclusion SUCCESS. The commit check API shows the same nine successes on the pinned SHA.

Actions run ID Event Status Conclusion headSha
44367055012 pull_request completed success 920a7afacce843426a54f719a198c5bc886a07e4

GitGuardian Security Checks also reports success on this head. Its check ID is 135023505769. PR rollup, Actions runs, commit checks, CI commands.

Model: gpt-6.2-sol, effort high

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant