Skip to content

Integrate negotiated Heft child reporting - #6000

Open
Sean Larkin (TheLarkInn) wants to merge 101 commits into
copilot/reporter-r5b-demo-reportersfrom
copilot/reporter-r7a-heft-reporting
Open

Integrate negotiated Heft child reporting#6000
Sean Larkin (TheLarkInn) wants to merge 101 commits into
copilot/reporter-r5b-demo-reportersfrom
copilot/reporter-r7a-heft-reporting

Conversation

@TheLarkInn

Copy link
Copy Markdown
Member

Part of #5980

Stack

Parent: #5997 (copilot/reporter-r5b-demo-reporters)

This PR is based directly on the latest R5B demo tip and intentionally excludes the parallel R6 #5993 bootstrap work. Auto-merge remains disabled while stack ancestors are open.

Negotiated Heft reporting

Rush now allocates a pair of private inherited descriptors for operation children: one carries child-to-parent NDJSON events and the other returns the parent hello acknowledgement. The handshake negotiates protocol major, optional capabilities, required features, and parent-owned reporter context (reporter, log level, color, and terminal width).

Compatible Heft versions:

  • advertise heft-child-events-v1 and reporter-context-v1;
  • emit bounded externalOutput and structured diagnostics without selecting or constructing the parent's reporters;
  • preserve child session, parent request/session/operation correlation, privacy classification, and child sourceSequence;
  • propagate structured warnings back into Rush operation status calculation.

The Heft-side bootstrap uses a node-only frozen wire client rather than a runtime dependency on @rushstack/rush-reporter, avoiding a Heft/self-hosting dependency cycle. It activates only when Rush supplies both private descriptors; direct standalone Heft behavior is unchanged.

Fallback and problem matchers

Older Heft and opaque external children continue through ordered raw stdout/stderr. Missing or capability-incompatible negotiation falls back without losing output. Unsupported protocol majors or required features produce RUSH_PROTOCOL_UPDATE_REQUIRED; malformed, oversized, or truncated streams produce RUSH_PROTOCOL_INVALID_CHILD_STREAM.

Raw output is published before matcher diagnostics. Matchers consume ANSI-normalized copies keyed independently by operation, child record, and stdout/stderr stream, retain bounded partial lines, cap duplicates, and never mutate, suppress, or reorder the source output. They are disabled only after heft-child-events-v1 is actually accepted, preventing duplicate recovery when structured diagnostics exist.

Validation

  • rush build --only @rushstack/heft --only @rushstack/rush-reporter --only @microsoft/rush-lib --only @microsoft/rush --verbose
  • reporter protocol/matcher/Heft integration suite: 317 tests
  • Rush child-process cross-version fixtures: 5 tests
  • Rush operation stream/sharding/raw-order suite: 15 tests
  • Heft child bootstrap fixtures: 3 tests
  • apps/rush frontend suite: 28 tests
  • node apps/rush/src/test/sandbox/reporter-demo/run.mjs
  • rush check
  • rush change --verify --no-fetch

Coverage includes new Rush + new Heft, old/capability-less child fallback, unsupported major/required features, unknown optional and required events, malformed/truncated/oversized records, descriptor close and child crash, stdout/stderr interleaving, UTF-8 chunk limits, sourceSequence, request/session/operation correlation, privacy, matcher recovery and duplicate caps, sharded operations, warning status propagation, and parent context.

Non-goals

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Create the authoritative frontend reporter host before version selection, register global reporter controls, and preserve legacy output unless a non-legacy reporter is explicitly selected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Consume the repository experiment before Rush version selection, keep agent detection out of pre-major defaults, strip frontend-only controls before engine handoff, and preserve legacy verbosity compatibility.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
@TheLarkInn

Copy link
Copy Markdown
Member Author

Independent /pr-agentmerge review completed.

Findings fixed in 1f910c2

  • Structured Heft externalOutput bypassed Rush project logs and build-cache log chunks. Structured output now enters the archival pipeline through a no-reemit terminal provider, preserving child correlation while avoiding duplicate reporter output and problem matching.
  • Heft trusted inherited descriptor numbers and could write to or close unrelated descriptors when an intermediate launcher preserved environment variables but dropped fd mappings. Heft now requires pipe/socket descriptors before negotiation; Windows lifecycle commands explicitly retain safe raw stdout/stderr fallback because the shell launch path does not preserve Node fd mappings.
  • The parent host now rejects child-session changes, non-monotonic source sequence, malformed externalOutput payloads, and chunks above the 64 KiB protocol limit.
  • The reporter demo now accepts the valid fromCache operation terminal state when comparing rendered output with raw events.

Validation

  • rush test --only @rushstack/rush-reporter --only @microsoft/rush-lib --only @microsoft/rush: 6/6 operations passed.
  • Focused Heft HeftChildReporter suite: 4/4 passed.
  • Reporter demo passed, covering negotiated current-Heft events, parent correlation/sourceSequence order, chunk bounds, legacy rollback parity, raw grouping, and duplicate suppression.
  • rush check, rush change --verify, API Extractor, lint, and the focused post-fix review passed.
  • The full Heft suite reached four unrelated pre-existing GitUtilities.checkIgnoreAsync failures caused by this hidden worktree path; the changed Heft suite passes directly.

There are no review threads to resolve. #6000 remains stacked on open #5997 with auto-merge disabled. Merge is blocked until the ancestor chain reaches main, the PR is retargeted/rebased one step at a time with an R7-only diff, and full required CI/review gates pass.

@TheLarkInn

Copy link
Copy Markdown
Member Author

Combined deep review of current head 1f910c26e8.

  1. HIGH extra reporter descriptors are attached to every Unix shell operation. Forked grandchildren can retain the pipes and hang operation completion. Allocate descriptors only for negotiated children.
  2. MEDIUM the acknowledgement stream lacks an error handler. A closed child pipe can raise an unhandled EPIPE.
  3. MEDIUM forwarded child envelopes retain child-chosen source and privacy. Override both from trusted parent context before telemetry or reporter fan-out.
  4. MEDIUM handshake/context exceptions can escape the stream data handler and terminate Rush. Validate once and reject attachAsync() instead.

Stop reporter control scans at the pass-through separator and add an exactly-once frontend close contract across success, failure, and termination paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Keep reporter controls out of ts-command-line globals, gate incompatible engines before initialization, and enforce bounded signal and close-error behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Preserve unsupported custom reporter values until frontend ownership is unambiguous, and narrow emergency legacy stripping to the reporter selection flag.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Probe reporter ownership without requiring a value, then enforce strict reporter parsing only after frontend ownership is established.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
@TheLarkInn
Sean Larkin (TheLarkInn) force-pushed the copilot/reporter-r5b-demo-reporters branch from e30aa36 to 7159901 Compare August 28, 2026 20:08
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Track explicit operation graph iteration identities across lifecycle events and isolate reporter totals, diagnostics, and output spools by watch cycle.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
@TheLarkInn
Sean Larkin (TheLarkInn) force-pushed the copilot/reporter-r5b-demo-reporters branch from 9e90abf to 4d141b8 Compare September 7, 2026 18:00
Preserve the existing feature commits and content. Resolve only history-induced conflicts where the parent file blobs are unchanged from the original parent; verify the full merged tree and feature patch against the pre-restack snapshot. The only aggregate content change is the existing R2A fixture isolation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
The newly rerun CI for #5989 exposed the existing reporter-package change without its release note. Describe the already-implemented pass-through separator behavior; no runtime code or gate changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Track explicit operation graph iteration identities across lifecycle events and isolate reporter totals, diagnostics, and output spools by watch cycle.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
@TheLarkInn
Sean Larkin (TheLarkInn) force-pushed the copilot/reporter-r5b-demo-reporters branch from 4d141b8 to 1c84401 Compare September 7, 2026 18:46
Only the existing R2B reporter change note is new content; all runtime and test blobs are unchanged. Preserve both parent histories and verify the child feature patch exactly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
@TheLarkInn
Sean Larkin (TheLarkInn) force-pushed the copilot/reporter-r5b-demo-reporters branch from 1c84401 to 26c2ac0 Compare September 7, 2026 20:07
Preserve the existing feature history and scope after the backed-up native stack root alignment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs triage

Development

Successfully merging this pull request may close these issues.

1 participant