From c964dc017a316ad5eb3c519866e8f85e94637a7c Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 01:53:14 +0100 Subject: [PATCH 01/34] WL-0MSQ3FH1K000MMJW: Herdr plugin: expand second-level (and deeper) children MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - getFlattenedItems() now recurses through any depth: expanding a child inserts its own children at depth = parent depth + 1 (depth derived from hierarchy position, never hardcoded) - select (Enter) and toggle-expand (Tab) handlers no longer gate on depth === undefined — any item with childCount > 0 toggles, matching top-level behavior - fetchChildrenForItem(parentId, depth) assigns the caller-provided depth; the on-demand fetch and refresh re-fetch pass parent depth + 1 so grandchildren render at depth 2+ - doRefresh attaches fresh children with a full tree walk (nested expanded parents re-fetch correctly) and attachChildren() lands on-demand fetches on the live tree object, so nested expansion survives auto-refresh - new WorkItemListState.getItemDepth()/attachChildren() helpers - unit + TUI-level tests: 3-level flattening, depth assignment, Tab/Enter gates, on-demand grandchild fetch, collapse, refresh persistence - README documents expand-at-any-depth; regenerate src/version.ts to match package.json 1.0.10 (stale compiled artifacts shadowing version.ts were removed so tests resolve one version consistently) --- packages/herdr/README.md | 6 +- packages/herdr/src/fetcher.ts | 11 +- .../herdr/src/worklist-nested-expand.test.ts | 325 ++++++++++++++++++ packages/herdr/src/worklist.test.ts | 152 ++++++++ packages/herdr/src/worklist.ts | 116 ++++++- src/version.ts | 2 +- 6 files changed, 590 insertions(+), 22 deletions(-) create mode 100644 packages/herdr/src/worklist-nested-expand.test.ts diff --git a/packages/herdr/README.md b/packages/herdr/README.md index 41b985b96..ce33496a6 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -4,7 +4,7 @@ A Herdr plugin that provides a keyboard-navigable work item selection list for b ## Features -- **Browse work items** — Lists work items from `wl next` in a scrollable, keyboard-navigable list. The top-level list is root-only: child work items are hidden and appear only under their parent via expand. Expanded parents **stay expanded across refreshes**: each auto/manual refresh re-fetches their children in parallel with the top-level list and swaps both in atomically, so the hierarchy never momentarily collapses or flickers (WL-0MSBVBNGH002RDP5). +- **Browse work items** — Lists work items from `wl next` in a scrollable, keyboard-navigable list. The top-level list is root-only: child work items are hidden and appear only under their parent via expand — **at any depth** (epic → feature → task and deeper): any item with children (its `childCount > 0`) can be expanded with Tab/Enter, its children fetched on demand via `wl list --parent` and shown indented at their hierarchy depth (WL-0MSQ3FH1K000MMJW). Expanded parents **stay expanded across refreshes**: each auto/manual refresh re-fetches their children in parallel with the top-level list and swaps both in atomically, so the hierarchy never momentarily collapses or flickers (WL-0MSBVBNGH002RDP5). - **Filter by stage** — Press `f` followed by a chord key (`i`=idea, `n`=intake, `p`=plan, `r`=review, `s`=sprint back to the default view), or type `/wl ` (shorthand alias or canonical stage name, e.g. `/wl intake_complete` or `/wl progress`), to filter items by stage. Stage-filtered views show every root item in the selected stage matching the stage's status rule (open items for most stages; `completed`/`in-progress`/`open` for the in_review stage) — no `browseItemCount` cap and no `wl next` selection omission (WL-0MSDT8X1V003206G, WL-0MSKCRX730052IIW) - **View details** — Press Enter on any item to see its full details (description, acceptance criteria, metadata, tags, priority, GitHub issue number, and audit status information such as audit result, review status, and last audit timestamp) - **Audit indicators** — The list view shows audit icons next to `in_review` items (✅ audited, ❌ failed, ❓ unaudited). The detail view metadata section additionally shows the review status (❌ needs review / ✅ reviewed) and the last audit timestamp. @@ -84,8 +84,8 @@ The plugin pane will then be available via the Herdr plugin system. - `PgDn` — Page down - `g` — Go to first item - `G` — Go to last item (last visible item in expanded hierarchy) - - `Enter` — View item details, or expand a parent item with children - - `Tab` — Toggle expand/collapse a parent item with children + - `Enter` — View item details, or expand a parent item with children (at any depth) + - `Tab` — Toggle expand/collapse a parent item with children (at any depth) - `Escape` — Go back (from detail or filter mode); in a child list, return to the parent level at the previous scroll position. When inside a child list the footer shows a `[esc] back` hint (with `(N levels)` when nested deeper than one level). 3. Filter by stage using chord shortcuts (or type `/wl `): diff --git a/packages/herdr/src/fetcher.ts b/packages/herdr/src/fetcher.ts index 522717834..336d87f56 100644 --- a/packages/herdr/src/fetcher.ts +++ b/packages/herdr/src/fetcher.ts @@ -581,13 +581,18 @@ export async function runWlSync(): Promise<{ success: boolean; error?: string }> /** * Fetch child work items for a given parent ID (via `wl list --parent`). - * Child items are returned with depth=1 for hierarchical display. + * + * Child items are returned with the given hierarchy `depth` for display + * (default 1 = direct children of a top-level item). Nested expansion + * passes the parent's depth + 1 so grandchildren render at depth 2, etc. + * (WL-0MSQ3FH1K000MMJW) — depth is derived from the fetch path, never + * hardcoded. */ -export async function fetchChildrenForItem(parentId: string): Promise { +export async function fetchChildrenForItem(parentId: string, depth = 1): Promise { const output = await runWl(['list', '--parent', parentId]); const payload = extractJson(output); const items = extractItems(payload); - return items.map((item) => ({ ...item, depth: 1 })); + return items.map((item) => ({ ...item, depth })); } // ── Work-item claiming ──────────────────────────────────────────────── diff --git a/packages/herdr/src/worklist-nested-expand.test.ts b/packages/herdr/src/worklist-nested-expand.test.ts new file mode 100644 index 000000000..a3d536c5b --- /dev/null +++ b/packages/herdr/src/worklist-nested-expand.test.ts @@ -0,0 +1,325 @@ +/** + * packages/herdr/src/worklist-nested-expand.test.ts — TUI-level tests for + * expanding second-level (and deeper) children in the Herdr worklist + * (WL-0MSQ3FH1K000MMJW). + * + * Before the fix, Tab/Enter only toggled TOP-LEVEL items: the select and + * toggle-expand handlers gated expansion on `depth === undefined`, and + * getFlattenedItems() never recursed past one level, so grandchildren were + * unreachable from the worklist UI. These tests drive the real + * `runWorklistTui` with a production-shaped fetcher/exec mock and assert: + * + * 1. Tab on a child (depth ≥ 1) with childCount > 0 fetches ITS children + * on demand via `wl list --parent ` and renders them inline. + * 2. The fetched grandchildren carry the correct depth (parent depth + 1). + * 3. Collapsing the child removes the grandchildren from the list. + * 4. Nested expanded state survives the auto-refresh cycle. + * + * Run: npx vitest run packages/herdr/src/worklist-nested-expand.test.ts + */ + +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import type { Mock } from 'vitest'; + +// --------------------------------------------------------------------------- +// Module mocks (must be hoisted before worklist.js is imported) +// --------------------------------------------------------------------------- +// Keep the REAL fetcher + visibility modules (so the exec seam and PollGate +// run for real) but avoid real `wl`/`herdr` process spawns by injecting a +// mock execFileAsync via setExecFileAsync() in each test. + +vi.mock('./auto-sync.js', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + // Keep the real scheduler/sync plumbing but stub runSync so no real + // `wl sync` is spawned. + runSync: vi.fn().mockResolvedValue({ success: true }), + }; +}); + +vi.mock('./notify.js', () => ({ + showToast: vi.fn(), +})); + +import { runWorklistTui } from './worklist.js'; +import { setExecFileAsync, resetExecFileAsync } from './fetcher.js'; +import type { WorkItem } from './fetcher.js'; + +// --------------------------------------------------------------------------- +// Fake stdin/stdout harness (same pattern as worklist-expand-refresh.test.ts) +// --------------------------------------------------------------------------- + +let dataHandler: ((chunk: Buffer) => void) | undefined; +let writes: string[]; + +beforeEach(() => { + vi.clearAllMocks(); + dataHandler = undefined; + writes = []; + + // Define missing stdin properties (vitest's process.stdin may not expose them) + for (const prop of ['on', 'removeListener', 'pause', 'resume', 'setRawMode'] as const) { + if (!(prop in process.stdin)) { + Object.defineProperty(process.stdin, prop, { + value: vi.fn(), + configurable: true, + writable: true, + }); + } + } + (process.stdin as any).on = vi.fn((event: string, cb: (chunk: Buffer) => void) => { + if (event === 'data') dataHandler = cb; + return process.stdin; + }); + (process.stdin as any).removeListener = vi.fn(() => process.stdin); + (process.stdin as any).pause = vi.fn(() => process.stdin); + (process.stdin as any).resume = vi.fn(() => process.stdin); + (process.stdin as any).setRawMode = vi.fn(() => process.stdin); + Object.defineProperty(process.stdin, 'isTTY', { value: false, configurable: true }); + + vi.spyOn(process.stdout, 'write').mockImplementation(((s: any) => { + writes.push(String(s)); + return true; + }) as any); + vi.spyOn(process.stdout, 'on').mockImplementation((() => process.stdout as any) as any); + vi.spyOn(process.stdout, 'removeListener').mockImplementation((() => process.stdout as any) as any); +}); + +afterEach(() => { + resetExecFileAsync(); + delete process.env.HERDR_PANE_ID; + vi.useRealTimers(); + vi.restoreAllMocks(); +}); + +/** Quit the TUI and await its promise (cleans up timers/listeners). */ +async function quit(p: Promise): Promise { + dataHandler?.(Buffer.from('q')); + await p; +} + +/** + * Build a production-shaped work item: no `children` array unless given + * (normalizeItem never populates it), childCount present, depth set for + * fetched children. + */ +function makeItem(id: string, depth: number | undefined, childCount: number, children?: WorkItem[]): WorkItem { + return { + id, + title: `Item ${id}`, + status: 'open', + ...(depth !== undefined ? { depth } : {}), + ...(childCount > 0 ? { childCount } : {}), + ...(children ? { children } : {}), + }; +} + +/** + * Exec-mock helper: visible pane (focused=true) so auto-refresh ticks fire. + * `wl list --parent P` returns the top-level parent's children (FEATURE + * with childCount 2); `wl list --parent FEATURE` returns its grandchildren + * (TASK-1, TASK-2). Records every --parent fetch so tests can assert the + * on-demand fetch target and depth. + */ +function makeNestedExecMock(): { mock: Mock; parentFetches: Array<{ parentId: string }> } { + const parentFetches: Array<{ parentId: string }> = []; + const mock = vi.fn(async (bin: string, args: string[]) => { + if (bin === 'herdr' && args[0] === 'pane' && args[1] === 'get') { + return { + stdout: JSON.stringify({ + id: 'cli:pane:get', + result: { pane: { focused: true } }, + }), + stderr: '', + }; + } + const parentIdx = args.indexOf('--parent'); + if (args.includes('list') && parentIdx >= 0) { + const parentId = args[parentIdx + 1]; + parentFetches.push({ parentId }); + if (parentId === 'P') { + return { + stdout: JSON.stringify({ + workItems: [makeItem('FEATURE', 1, 2), makeItem('FEATURE-2', 1, 0)], + }), + stderr: '', + }; + } + if (parentId === 'FEATURE') { + return { + stdout: JSON.stringify({ + workItems: [makeItem('TASK-1', 2, 0), makeItem('TASK-2', 2, 0)], + }), + stderr: '', + }; + } + return { stdout: JSON.stringify({ workItems: [] }), stderr: '' }; + } + if (args.includes('list') && args.includes('--status')) { + return { stdout: JSON.stringify({ count: 5 }), stderr: '' }; + } + return { stdout: JSON.stringify({ workItems: [] }), stderr: '' }; + }); + return { mock, parentFetches }; +} + +// --------------------------------------------------------------------------- +// Tests +// --------------------------------------------------------------------------- + +describe('nested expansion (WL-0MSQ3FH1K000MMJW)', () => { + it('Tab on a child fetches its grandchildren on demand and renders them at depth 2', async () => { + vi.useFakeTimers(); + process.env.HERDR_PANE_ID = 'w1:pCM'; + const { mock, parentFetches } = makeNestedExecMock(); + setExecFileAsync(mock as any); + + const fetcher = vi.fn().mockResolvedValue([makeItem('P', undefined, 1)]); + const p = runWorklistTui(fetcher, undefined, undefined, { + autoRefresh: true, + refreshIntervalMs: 30_000, + autoSync: false, + showHelpText: false, + }); + await vi.advanceTimersByTimeAsync(0); + + // Tab on P: fetch FEATURE (depth 1) on demand and expand P. + dataHandler?.(Buffer.from('\t')); + await vi.advanceTimersByTimeAsync(0); + expect(writes.join('')).toContain('FEATURE'); + expect(parentFetches).toEqual([{ parentId: 'P' }]); + + // Select FEATURE (flattened index 1) with ↓. + writes.length = 0; + dataHandler?.(Buffer.from('j')); + await vi.advanceTimersByTimeAsync(0); + + // Tab on FEATURE: fetch grandchildren (TASK-1, TASK-2) on demand. + dataHandler?.(Buffer.from('\t')); + await vi.advanceTimersByTimeAsync(0); + + expect(parentFetches).toEqual([{ parentId: 'P' }, { parentId: 'FEATURE' }]); + const output = writes.join(''); + expect(output).toContain('TASK-1'); + expect(output).toContain('TASK-2'); + + await quit(p); + }); + + it('collapsing the child removes the grandchildren from the list', async () => { + vi.useFakeTimers(); + process.env.HERDR_PANE_ID = 'w1:pCM'; + setExecFileAsync(makeNestedExecMock().mock as any); + + const fetcher = vi.fn().mockResolvedValue([makeItem('P', undefined, 1)]); + const p = runWorklistTui(fetcher, undefined, undefined, { + autoRefresh: true, + refreshIntervalMs: 30_000, + autoSync: false, + showHelpText: false, + }); + await vi.advanceTimersByTimeAsync(0); + + // Expand P, move to FEATURE, expand FEATURE. + dataHandler?.(Buffer.from('\t')); + await vi.advanceTimersByTimeAsync(0); + dataHandler?.(Buffer.from('j')); + await vi.advanceTimersByTimeAsync(0); + dataHandler?.(Buffer.from('\t')); + await vi.advanceTimersByTimeAsync(0); + expect(writes.join('')).toContain('TASK-1'); + + // Tab on FEATURE again → collapse → grandchildren gone. + writes.length = 0; + dataHandler?.(Buffer.from('\t')); + await vi.advanceTimersByTimeAsync(0); + const collapsedOutput = writes.join(''); + expect(collapsedOutput).not.toContain('TASK-1'); + expect(collapsedOutput).not.toContain('TASK-2'); + expect(collapsedOutput).toContain('FEATURE'); + + await quit(p); + }); + + it('nested expanded state survives the auto-refresh cycle', async () => { + vi.useFakeTimers(); + process.env.HERDR_PANE_ID = 'w1:pCM'; + const { mock } = makeNestedExecMock(); + setExecFileAsync(mock as any); + + // Startup fetch returns P; the refresh tick returns a NEW P object + // (fresh title) — production fetcher shape (children-less). + const fetcher = vi.fn() + .mockResolvedValueOnce([makeItem('P', undefined, 1)]) + .mockResolvedValueOnce([{ ...makeItem('P', undefined, 1), title: 'P (refreshed)' }]); + + const p = runWorklistTui(fetcher, undefined, undefined, { + autoRefresh: true, + refreshIntervalMs: 30_000, + autoSync: false, + showHelpText: false, + }); + await vi.advanceTimersByTimeAsync(0); + fetcher.mockClear(); + + // Expand P → FEATURE, then FEATURE → TASK-1/TASK-2. + dataHandler?.(Buffer.from('\t')); + await vi.advanceTimersByTimeAsync(0); + dataHandler?.(Buffer.from('j')); + await vi.advanceTimersByTimeAsync(0); + dataHandler?.(Buffer.from('\t')); + await vi.advanceTimersByTimeAsync(0); + expect(writes.join('')).toContain('TASK-1'); + + // Auto-refresh tick: the refreshed P must keep FEATURE expanded with + // its grandchildren visible — no re-collapse (AC-4). + writes.length = 0; + await vi.advanceTimersByTimeAsync(30_000); + + const output = writes.join(''); + expect(output).toContain('P (refreshed)'); + expect(output).toContain('FEATURE'); + expect(output).toContain('TASK-1'); + expect(output).toContain('TASK-2'); + + await quit(p); + }); + + it('Enter on a child with children toggles expansion instead of opening the detail view', async () => { + vi.useFakeTimers(); + process.env.HERDR_PANE_ID = 'w1:pCM'; + setExecFileAsync(makeNestedExecMock().mock as any); + + const fetcher = vi.fn().mockResolvedValue([makeItem('P', undefined, 1)]); + const p = runWorklistTui(fetcher, undefined, undefined, { + autoRefresh: true, + refreshIntervalMs: 30_000, + autoSync: false, + showHelpText: false, + }); + await vi.advanceTimersByTimeAsync(0); + + // Expand P with Tab (fetches FEATURE), select FEATURE, then Tab-expand + // FEATURE so its grandchildren are loaded. + dataHandler?.(Buffer.from('\t')); + await vi.advanceTimersByTimeAsync(0); + dataHandler?.(Buffer.from('j')); + await vi.advanceTimersByTimeAsync(0); + dataHandler?.(Buffer.from('\t')); + await vi.advanceTimersByTimeAsync(0); + expect(writes.join('')).toContain('TASK-1'); + + // Enter on FEATURE (children now loaded): toggles collapse — the detail + // view must NOT open (matching top-level Enter behavior). + writes.length = 0; + dataHandler?.(Buffer.from('\r')); + await vi.advanceTimersByTimeAsync(0); + const output = writes.join(''); + expect(output).not.toContain('TASK-1'); + expect(output).not.toContain('TASK-2'); + expect(output).toContain('FEATURE'); + + await quit(p); + }); +}); diff --git a/packages/herdr/src/worklist.test.ts b/packages/herdr/src/worklist.test.ts index 50f81b933..bad480853 100644 --- a/packages/herdr/src/worklist.test.ts +++ b/packages/herdr/src/worklist.test.ts @@ -491,6 +491,158 @@ describe('WorkItemListState.refreshItems — preserve selection by ID', () => { }); }); +describe('nested expansion — 3+ level hierarchies (WL-0MSQ3FH1K000MMJW)', () => { + /** + * Build a 3-level hierarchy: EPIC → FEATURE → TASK. Children carry the + * depth the fetcher assigns (fetchChildrenForItem), mirroring production + * shape where grandchildren are fetched with depth = parent depth + 1. + */ + function makeThreeLevelTree(): WorkItem[] { + const epic = makeItem('EPIC', 'in_progress'); + const feature = { + ...makeItem('FEATURE', 'in_progress'), + depth: 1, + childCount: 1, + children: [{ ...makeItem('TASK', 'in_progress'), depth: 2 }], + }; + epic.childCount = 1; + epic.children = [feature]; + return [epic]; + } + + it('recursively flattens 3+ level hierarchies with correct depths', () => { + const state = new WorkItemListState(makeThreeLevelTree(), TERM_80x24); + + // Only EPIC visible initially. + expect(state.getFlattenedItems().map((i) => [i.id, i.depth])).toEqual([ + ['EPIC', undefined], + ]); + + // Expand EPIC → FEATURE appears at depth 1; TASK stays hidden (FEATURE + // not expanded yet). + state.toggleExpand('EPIC'); + expect(state.getFlattenedItems().map((i) => [i.id, i.depth])).toEqual([ + ['EPIC', undefined], + ['FEATURE', 1], + ]); + + // Expand FEATURE → TASK appears at depth 2. + state.toggleExpand('FEATURE'); + expect(state.getFlattenedItems().map((i) => [i.id, i.depth])).toEqual([ + ['EPIC', undefined], + ['FEATURE', 1], + ['TASK', 2], + ]); + }); + + it('collapsing a child removes its grandchildren from the flattened list', () => { + const state = new WorkItemListState(makeThreeLevelTree(), TERM_80x24); + state.toggleExpand('EPIC'); + state.toggleExpand('FEATURE'); + expect(state.getFlattenedItems().map((i) => i.id)).toEqual(['EPIC', 'FEATURE', 'TASK']); + + state.toggleExpand('FEATURE'); + expect(state.getFlattenedItems().map((i) => i.id)).toEqual(['EPIC', 'FEATURE']); + }); + + it('collapsing an ancestor removes the entire nested subtree', () => { + const state = new WorkItemListState(makeThreeLevelTree(), TERM_80x24); + state.toggleExpand('EPIC'); + state.toggleExpand('FEATURE'); + + state.toggleExpand('EPIC'); + expect(state.getFlattenedItems().map((i) => i.id)).toEqual(['EPIC']); + }); + + it('Enter (select) on a child with children toggles expansion instead of opening the detail view', () => { + const state = new WorkItemListState(makeThreeLevelTree(), TERM_80x24); + state.toggleExpand('EPIC'); + // Select FEATURE (flattened index 1). + state.selectedIndex = 1; + expect(state.getFlattenedItems()[1].id).toBe('FEATURE'); + + const action = handleKeypress(state, '\r', TERM_80x24); + + // Enter toggles expansion — never opens the detail view. + expect(action).toBe('toggle-expand'); + expect(state.mode).toBe('list'); + expect(state.isExpanded('FEATURE')).toBe(true); + expect(state.getFlattenedItems().map((i) => i.id)).toEqual(['EPIC', 'FEATURE', 'TASK']); + }); + + it('Enter on a child toggles back to collapsed on the second press', () => { + const state = new WorkItemListState(makeThreeLevelTree(), TERM_80x24); + state.toggleExpand('EPIC'); + state.selectedIndex = 1; // FEATURE + handleKeypress(state, '\r', TERM_80x24); // expand + handleKeypress(state, '\r', TERM_80x24); // collapse + expect(state.isExpanded('FEATURE')).toBe(false); + expect(state.getFlattenedItems().map((i) => i.id)).toEqual(['EPIC', 'FEATURE']); + }); + + it('Tab on a child with childCount > 0 returns toggle-expand even when children not yet loaded', () => { + // FEATURE has childCount but NO children array yet (production shape: + // grandchildren are fetched on demand when first expanded). + const epic = makeItem('EPIC'); + epic.childCount = 1; + epic.children = [{ ...makeItem('FEATURE'), depth: 1, childCount: 2 }]; + const state = new WorkItemListState([epic], TERM_80x24); + state.toggleExpand('EPIC'); + state.selectedIndex = 1; // FEATURE + + const action = handleKeypress(state, '\t', TERM_80x24); + expect(action).toBe('toggle-expand'); + }); + + it('Tab on an item without children returns null (no toggle, no crash)', () => { + const epic = makeItem('EPIC'); + epic.childCount = 1; + epic.children = [{ ...makeItem('FEATURE'), depth: 1 }]; // no childCount + const state = new WorkItemListState([epic], TERM_80x24); + state.toggleExpand('EPIC'); + state.selectedIndex = 1; // FEATURE + + expect(handleKeypress(state, '\t', TERM_80x24)).toBeNull(); + }); + + it('getItemDepth reports the hierarchy position (0 = top-level)', () => { + const state = new WorkItemListState(makeThreeLevelTree(), TERM_80x24); + expect(state.getItemDepth('EPIC')).toBe(0); + expect(state.getItemDepth('FEATURE')).toBe(1); + expect(state.getItemDepth('TASK')).toBe(2); + // Unknown IDs default to 0 (safe fetch depth). + expect(state.getItemDepth('UNKNOWN')).toBe(0); + }); + + it('attachChildren attaches fetched children to the live tree object at any depth', () => { + const state = new WorkItemListState(makeThreeLevelTree(), TERM_80x24); + const freshGrandchild = { ...makeItem('TASK2'), depth: 2 }; + state.attachChildren('FEATURE', [freshGrandchild]); + + // The tree object (not a flattened copy) received the children. + const epic = state.items[0]; + const feature = epic.children![0]; + expect(feature.children).toEqual([freshGrandchild]); + // And the flattened view renders them once FEATURE is expanded. + state.toggleExpand('EPIC'); + state.toggleExpand('FEATURE'); + expect(state.getFlattenedItems().map((i) => i.id)).toEqual(['EPIC', 'FEATURE', 'TASK2']); + }); + + it('nested expanded state survives refreshItems (carry-over restores the subtree)', () => { + const state = new WorkItemListState(makeThreeLevelTree(), TERM_80x24); + state.toggleExpand('EPIC'); + state.toggleExpand('FEATURE'); + + // Refresh with a children-less EPIC (production fetcher shape): the + // carried-over subtree must keep TASK visible. + const freshEpic = { ...makeItem('EPIC'), childCount: 1 }; + state.refreshItems([freshEpic]); + + expect(state.getFlattenedItems().map((i) => i.id)).toEqual(['EPIC', 'FEATURE', 'TASK']); + }); +}); + describe('executeResolvedCommand', () => { it('returns noop when command has but no items', () => { const state = new WorkItemListState([], TERM_80x24); diff --git a/packages/herdr/src/worklist.ts b/packages/herdr/src/worklist.ts index 648aae847..fe658d91f 100644 --- a/packages/herdr/src/worklist.ts +++ b/packages/herdr/src/worklist.ts @@ -470,20 +470,80 @@ export class WorkItemListState { /** * Get the flattened item list, inserting children of expanded parents. + * + * Recurses into any depth (WL-0MSQ3FH1K000MMJW): after inserting a child + * of an expanded item, the child's own children are inserted too when the + * child is itself expanded. Depth is derived from the hierarchy position + * (top-level = no depth, each nested level = parent's depth + 1), so + * items fetched at any level render at the correct indentation. */ getFlattenedItems(): WorkItem[] { const result: WorkItem[] = []; - for (const item of this.items) { - result.push(item); + const appendItem = (item: WorkItem, depth: number): void => { + // Top-level items are pushed as-is (no depth field, matching the + // pre-hierarchy shape). Nested items keep their object reference when + // their stored depth already matches the hierarchy position (so + // on-demand child fetches mutate the live tree object), otherwise the + // depth is corrected with a shallow copy. + if (depth === 0) { + result.push(item); + } else { + result.push(item.depth === depth ? item : { ...item, depth }); + } if (item.childCount && item.children && item.children.length > 0 && this.expandedItems.has(item.id)) { for (const child of item.children) { - result.push({ ...child, depth: child.depth ?? 1 }); + appendItem(child, depth + 1); } } + }; + for (const item of this.items) { + appendItem(item, 0); } return result; } + /** + * Compute the hierarchy depth of an item in the current tree + * (0 = top-level, 1 = child, …). Returns 0 for unknown IDs — the safe + * default used when fetching their children at depth 1 (WL-0MSQ3FH1K000MMJW). + */ + getItemDepth(id: string): number { + const walk = (items: WorkItem[], depth: number): number => { + for (const item of items) { + if (item.id === id) return depth; + if (item.children && item.children.length > 0) { + const found = walk(item.children, depth + 1); + if (found >= 0) return found; + } + } + return -1; + }; + const depth = walk(this.items, 0); + return depth >= 0 ? depth : 0; + } + + /** + * Attach fetched children to the item in the live tree (walking nested + * levels), never to a flattened copy. Ensures an on-demand fetch for a + * child at any depth lands on the object the flatten recursion walks, so + * the grandchildren actually render (WL-0MSQ3FH1K000MMJW). + */ + attachChildren(id: string, children: WorkItem[]): void { + const walk = (items: WorkItem[]): boolean => { + for (const item of items) { + if (item.id === id) { + item.children = children; + return true; + } + if (item.children && item.children.length > 0 && walk(item.children)) { + return true; + } + } + return false; + }; + walk(this.items); + } + selectItem(): void { if (this.items.length === 0) return; const flat = this.getFlattenedItems(); @@ -1716,8 +1776,11 @@ export function handleKeypress( const flat = state.getFlattenedItems(); if (state.selectedIndex < flat.length) { const selected = flat[state.selectedIndex]; - // Toggle expand/collapse for items with actual children data - if (selected.children && selected.children.length > 0 && selected.depth === undefined) { + // Toggle expand/collapse for items with actual children data at + // ANY depth (WL-0MSQ3FH1K000MMJW): Enter on a child with children + // expands it like a top-level parent, instead of opening the + // detail view. + if (selected.children && selected.children.length > 0) { if (state.isExpanded(selected.id)) { // Collapsing — remove the matching navigation-stack entry so // a later Escape does not pop back into a collapsed parent. @@ -1773,8 +1836,10 @@ export function handleKeypress( const flat = state.getFlattenedItems(); if (state.selectedIndex < flat.length) { const selected = flat[state.selectedIndex]; - // Only top-level items with children can be expanded/collapsed - if (selected.depth === undefined && selected.childCount && selected.childCount > 0) { + // Any item with children — at ANY depth — can be expanded/ + // collapsed with Tab (WL-0MSQ3FH1K000MMJW). Children data is + // fetched on demand by the caller when not yet loaded. + if (selected.childCount && selected.childCount > 0) { // Track hierarchy: push parent state before expanding so Escape // can return to it; drop the entry when collapsing. if (state.isExpanded(selected.id)) { @@ -2767,7 +2832,10 @@ export async function runWorklistTui( const freshChildren = new Map(); const fetchExpandedChildren = async (parentId: string): Promise => { try { - const children = await fetchChildrenForItem(parentId); + // Depth derived from the item's hierarchy position so nested + // expanded items re-fetch their children at the correct depth + // (WL-0MSQ3FH1K000MMJW). + const children = await fetchChildrenForItem(parentId, state.getItemDepth(parentId) + 1); freshChildren.set(parentId, children); return children; } catch { @@ -2784,13 +2852,25 @@ export async function runWorklistTui( // Attach freshly fetched children to the new parent objects BEFORE // the swap so the flattened view is complete at render time. + // Walks the whole tree (top-level + nested children) because + // expandedItems may contain items at any depth: a nested parent is + // not in the top-level id→item map, so without the walk its fresh + // grandchildren would be dropped and the nested expansion would + // collapse on refresh (WL-0MSQ3FH1K000MMJW AC-4). if (freshChildren.size > 0) { - const byId = new Map(newItems.map((it) => [it.id, it])); - for (const [parentId, children] of freshChildren) { - const parent = byId.get(parentId); - if (parent) { - parent.children = children; + const attachFresh = (list: WorkItem[]): void => { + for (const item of list) { + const fresh = freshChildren.get(item.id); + if (fresh) { + item.children = fresh; + } + if (item.children && item.children.length > 0) { + attachFresh(item.children); + } } + }; + for (const topLevel of newItems) { + attachFresh([topLevel]); } } state.refreshItems(newItems); @@ -3274,8 +3354,14 @@ export async function runWorklistTui( // If children data not yet loaded, fetch them on demand if (selected.childCount && selected.childCount > 0 && (!selected.children || selected.children.length === 0)) { render(); // immediate render while fetch is pending - const children = await fetchChildrenForItem(selected.id); - selected.children = children; + // Depth derived from the selected item's hierarchy position so + // grandchildren of a nested item fetch at depth N+1, not 1 + // (WL-0MSQ3FH1K000MMJW). + const children = await fetchChildrenForItem(selected.id, (selected.depth ?? 0) + 1); + // Attach to the live tree object so the fetched grandchildren + // survive the flatten (which may hold a copy with a corrected + // depth) — see WorkItemListState.attachChildren. + state.attachChildren(selected.id, children); // Merge agent-status state into the freshly loaded children so // their rows show agent icons too (WL-0MSBQUJQX005RAT9). try { diff --git a/src/version.ts b/src/version.ts index 831d85f95..e659049e5 100644 --- a/src/version.ts +++ b/src/version.ts @@ -1,2 +1,2 @@ // Auto-generated; do not edit. -export const WORKLOG_VERSION = '1.0.9'; +export const WORKLOG_VERSION = '1.0.10'; From 5107645b09ee291242c9364e51b2cad0b75d4539 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 11:32:21 +0100 Subject: [PATCH 02/34] WL-0MSRBFFLN005W3VT: Implementation complete --- packages/herdr/README.md | 26 ++ .../downtime-redispatch-rca-2026-08-13.md | 267 ++++++++++++++++++ .../scripts/scan_duplicate_dispatches.py | 177 ++++++++++++ .../tests/test_scan_duplicate_dispatches.py | 98 +++++++ 4 files changed, 568 insertions(+) create mode 100644 packages/herdr/docs/downtime-redispatch-rca-2026-08-13.md create mode 100755 packages/herdr/scripts/scan_duplicate_dispatches.py create mode 100644 packages/herdr/scripts/tests/test_scan_duplicate_dispatches.py diff --git a/packages/herdr/README.md b/packages/herdr/README.md index ce33496a6..235a603ff 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -382,6 +382,32 @@ distinguish "no candidate (paused)" from "worker disabled" or from any of the silent failure paths above; only the three-strike and success paths leave entries. The follow-up items above close the three code-level silent paths. +**Known re-dispatch gaps** (RCA WL-0MSRBFFLN005W3VT, evidence reproduced by +`packages/herdr/scripts/scan_duplicate_dispatches.py`; see +[docs/downtime-redispatch-rca-2026-08-13.md](docs/downtime-redispatch-rca-2026-08-13.md)): + +- **Same-instant cross-pane race** — two panes can select the same candidate + before either claims it: selection precedes claim, and the pre-dispatch + claim (`wl update --status in_progress`) is idempotent, so both panes + proceed. Observed post-fix: a `kind:audit` pair 14 ms apart + (SA-0MSN4AXIQ007IZG2, 2026-08-10). The dispatched-marker exclusion is + read-then-write and written after spawn, so it cannot prevent this. Fix + design (follow-up): atomic conditional claim (`--if-status`/`--if-stage`) + — a losing pane aborts the dispatch. +- **Plan/intake tiers have no dispatched-marker exclusion** — the audit and + implement tiers exclude already-dispatched items; the plan + (`--stage intake_complete`) and intake (`--stage idea`) tiers do not, and + `wl next` keeps `completed` items under a stage filter, so a plan run whose + error/abort path resets status only (stage left at `intake_complete`) + leaves the item selectable for re-dispatch. Observed: `kind:plan` ×2 on + SA-0MSMAZP6T007NM0O and SA-0MSN04X2S006ONH0 (2026-08-12/13), `kind:plan` ×7 + in ~7 h pre-fix (SA-0MSJI53RX006E2PS). Fix design (follow-up): kind-scoped + plan/intake markers with a change-guard plus a client-side `status === + 'open'` filter. + +These are documented as design candidates in the RCA; implementation is a +follow-up work item. + The worker runs inside the plugin's single consolidated scheduler loop (one `setInterval`; no independent timers), uses unref'd timers, and is cleaned up when the pane exits. While the pane is open the list header shows the worker diff --git a/packages/herdr/docs/downtime-redispatch-rca-2026-08-13.md b/packages/herdr/docs/downtime-redispatch-rca-2026-08-13.md new file mode 100644 index 000000000..c61d649a7 --- /dev/null +++ b/packages/herdr/docs/downtime-redispatch-rca-2026-08-13.md @@ -0,0 +1,267 @@ +# RCA — Downtime dispatcher re-dispatches the same work item + +**Date:** 2026-08-13 · **Work item:** WL-0MSRBFFLN005W3VT · **Severity:** critical (operational) + +## Headline + +The herdr downtime worker (`dispatchDowntimeWork`, `packages/herdr/src/downtime-worker.ts`) +still dispatches the same work item more than once. Two distinct root causes are +proven from dispatch-log evidence, reproduced by +`packages/herdr/scripts/scan_duplicate_dispatches.py` across all five +`.worklog/downtime-dispatches.log` roots on this host: + +1. **RC-1 — same-instant cross-pane race (audit tier; affects all tiers).** Two + panes select the same candidate before either claims it, because selection + happens **before** claim and the claim is not conditional (read-then-write / + TOCTOU). Proven by a pair of audit dispatches **14 ms apart** on the same + item/root (SA-0MSN4AXIQ007IZG2, 2026-08-10T21:20:53.909Z and + .923Z) — *after* the dispatched-marker exclusion landed (2026-08-09). +2. **RC-2 — plan tier has no dispatched-marker exclusion; "stage self-guard" fails.** + The plan tier (`getNextItem('intake_complete')`) has no marker exclusion + (unlike audit/implement), and `wl next --stage intake_complete` keeps + `completed` items under a stage filter (no client-side `status === 'open'` + guard on the plan/intake tiers). A plan run that aborts/errors resets + **status only** (`StatusLifecycle.update_status(, "open")`, + `skill/plan/SKILL.md`), leaving the stage at `intake_complete` — the item is + re-selected on the next idle window. Proven by plan-tier duplicate pairs + post-fix (SA-0MSMAZP6T007NM0O, SA-0MSN04X2S006ONH0), including one item + plan-dispatched 7× in ~7 h (SA-0MSJI53RX006E2PS, pre-fix, demonstrating the + unbounded-loop severity). + +**Deliverable:** this RCA + a fix design (below). Implementation is explicitly +deferred to a follow-up work item — **WL-0MSRDEWES0059TZN** (scope guard, +blocked-by this item). + +## Evidence — reproduced duplicate scan + +Run: +```bash +python3 packages/herdr/scripts/scan_duplicate_dispatches.py # all pairs +python3 packages/herdr/scripts/scan_duplicate_dispatches.py --json # machine-readable +python3 packages/herdr/scripts/scan_duplicate_dispatches.py --since 2026-08-09T14:16:32Z # post-fix +``` + +Full scan (2026-08-13): **17 duplicate (kind, itemId) pairs** across the five +roots — 12 audit pairs and 5 plan pairs. + +### Post-fix pairs (marker exclusion landed 2026-08-09, commit `228019bd` 14:16:32Z) + +| Kind | Item | Count | First / last dispatch (UTC) | Gap | Root cause | +|---|---|---|---|---|---| +| audit | SA-0MSN4AXIQ007IZG2 | 2 | 2026-08-10T21:20:53.909Z / .923Z | **14 ms** | RC-1 (same-instant race) | +| plan | SA-0MSMAZP6T007NM0O | 2 | 2026-08-12T06:59:43.732Z / 2026-08-13T04:50:06.423Z | ~21.8 h | RC-2 (no plan marker; stage self-guard fails) | +| plan | SA-0MSN04X2S006ONH0 | 2 | 2026-08-12T11:05:30.321Z / 2026-08-13T05:19:08.331Z | ~18.2 h | RC-2 (no plan marker) | +| audit | WL-0MSHUMX5C004NC4O | 2 | 2026-08-09T15:29:41.663Z / 16:35:15.612Z | ~1.1 h | same-day, pre-reload of the Aug-9 fix | +| audit | OSL-0MSLY6OY2003BAFL | 2 | 2026-08-09T16:02:55.919Z / 17:17:01.825Z | ~1.2 h | same-day, pre-reload of the Aug-9 fix | +| audit | CG-0MSL0OP040043KKZ | 2 | 2026-08-09T16:48:42.313Z / 18:54:50.274Z | ~2.1 h | same-day, pre-reload of the Aug-9 fix | + +> **Deployment-timing note:** the three audit pairs on 2026-08-09 are **after** +> the fix commit time but on the same day the fix landed; herdr panes reload +> plugin code only when restarted, so these are consistent with panes still +> running the pre-fix code (the re-selection loop the exclusion closes). They +> are **not** evidence against the exclusion. The definitive post-fix audit +> duplicate is the 14 ms RC-1 pair on 2026-08-10 — which the marker exclusion +> cannot prevent by construction (selection precedes claim/marker write). + +### Pre-fix pairs (re-selection loop the Aug-9 fix closed, plus severity demo) + +| Kind | Item | Count | Span | Note | +|---|---|---|---|---| +| plan | SA-0MSJI53RX006E2PS | **7** | 2026-08-09T00:46Z–07:31Z (~6.8 h) | unbounded loop severity, plan tier | +| audit | WL-0MSGTLSUT002NF29 | 2 | 2026-08-09T01:51Z–10:59Z | re-selection loop, audit tier | +| audit | WL-0MSKUG2WW0058A7W | 2 | 2026-08-09T06:38Z–13:49Z | re-selection loop, audit tier | +| audit | SA-0MSL51XSF0086KM5 | 2 | 2026-08-09T14:45Z–16:48Z | re-selection loop, audit tier | +| plan | WL-0MSGG5N5Z0074TLY | 2 | 2026-08-06T23:51Z–2026-08-07T00:03Z | pre-fix | +| audit | CG-0MSJ7ZXDB002CX97 | 2 | 2026-08-09T02:58Z–15:42Z | re-selection loop | +| audit | CG-0MSJ7ZXD5005N9E5 | 2 | 2026-08-09T13:49Z–14:55Z | re-selection loop | +| audit | DS-0MSAZSAPL003YHVR | 2 | 2026-08-07T06:07Z–2026-08-08T02:49Z | pre-fix | +| audit | DS-0MSDIVUPU003Y22W | 2 | 2026-08-09T07:20Z–10:22Z | re-selection loop | +| audit | OSL-0MSKODN9T001G9U7 | 2 | 2026-08-09T00:46Z–15:05Z | re-selection loop | +| plan | OSL-0MSGCVSDH002656S | 2 | 2026-08-07T09:49Z–10:08Z | pre-fix | + +## Root causes + +### RC-1 — Same-instant cross-pane race (audit tier; affects all tiers) + +- **Evidence:** SA-0MSN4AXIQ007IZG2 (SorraAgents root) — `kind=audit` + dispatched at 2026-08-10T21:20:53.909Z **and** 21:20:53.923Z — two log + entries **14 ms apart**, same item, same root, both recorded as success. + Post-fix (marker exclusion landed 2026-08-09). +- **Mechanism (code-traced):** + - Every herdr pane runs its own `dispatchDowntimeWork`; `dispatchInFlight` + is per-process only (`downtime-worker.ts:196`). Cross-pane serialization + is explicitly delegated to the pre-dispatch claim (code comment on + `dispatchInFlight`: "the claimed item leaves `wl next`'s selection set"). + - Selection happens **before** claim in every tier + (`dispatchDowntimeWork`, `downtime-worker.ts:232-280`): + `getNextAuditCandidate`/`getNextItem` reads the dispatch log (no marker + yet) → selects → `claimItem` → `spawnAgentPane` → `recordDispatch` + (marker written **after** spawn). + - `claimWorkItem` (`packages/herdr/src/fetcher.ts:623`) runs + `wl update --status in_progress --assignee ` — idempotent: + re-claiming an already-`in_progress` item also "succeeds", so both panes + proceed. Claim failures are discarded by `claimItem` + (`packages/herdr/src/index.ts:456-462`; open follow-up + WL-0MSLWJ310000ND0X). Result: two identical panes for one item. + - **Why the existing marker cannot prevent it:** the dispatched-marker + exclusion is read-then-write (TOCTOU) on + `.worklog/downtime-dispatches.log`; both panes read the log before either + appends its marker, and the marker is written after spawn — even a correct + read could not block the second pane in time. + +### RC-2 — Plan tier has no dispatched-marker exclusion; "stage self-guard" fails + +- **Evidence (post-fix, plan tier):** + - SA-0MSN04X2S006ONH0 — `kind=plan` ×2: 2026-08-12T11:05:30Z and + 2026-08-13T05:19:08Z — no implement/audit between the two, i.e. the first + plan run left the item selectable again. + - SA-0MSMAZP6T007NM0O — `kind=plan` ×2: 2026-08-12T06:59:43Z and + 2026-08-13T04:50:06Z — the second plan dispatch occurred even though the + item had completed a full plan → implement → audit cycle. + - SA-0MSJI53RX006E2PS — `kind=plan` ×7 in ~7 h (2026-08-09T00:46Z–07:31Z) — + pre-fix, but demonstrates the unbounded loop severity of the same tier. +- **Mechanism (code-traced):** + - The plan tier (`getNextItem('intake_complete')` → + `wl next --stage intake_complete --json`, `index.ts:351-383`) has **no** + dispatched-marker exclusion, unlike the audit/implement tiers + (`getNextAuditCandidate`/`getNextImplementCandidate` consult + `readDowntimeLogEntries`; the plan/intake tiers do not). + - The plan skill's error/abort path is `StatusLifecycle.update_status(, + "open")` — **status only; stage is left at `intake_complete`** + (`skill/plan/SKILL.md` "On error/abort"). A plan pane that aborts (model + error, infra failure, killed session) resets the item to + `open`/`intake_complete` → the dispatcher re-selects it on the next idle + window. The audit runner has a similar restore-to-captured-state path + (`skill/audit/scripts/audit_runner.py` `finally` block, lines ~6458-6538) + that can likewise leave an item selectable. + - **Amplifier:** `wl next`'s stage filter does **not** remove completed + items (`filterCandidates` step 3 in + `packages/shared/src/database.ts:1931-1968`: completed items removed only + when **no** stage filter is active), and the plan/intake tiers have no + client-side `status === 'open'` guard (the implement tier has one — + `selectImplementCandidate` filters `c.status === 'open'`). So even a + `completed`/`in_review` item whose stage equals `intake_complete` is + selectable for `/skill:plan`. + +### RC-3 (contributing, audit tier) — marker fragility + +The audit-tier marker exclusion works for the re-selection loop (no post-fix +audit re-dispatch beyond RC-1), but it is per-worklog-root +(`.worklog/downtime-dispatches.log` at ``), bounded at +`DOWNTIME_LOG_MAX_ENTRIES = 100`, and written **after** spawn — a pane that +dies before `recordDispatch` leaves no marker. These are residual risks, not +the observed failure. + +## Fix design (for follow-up implementation) + +Five design points, matching the four mechanisms above. Implementation is the +follow-up work item WL-0MSRDEWES0059TZN; this section records the agreed +option choices. + +1. **Verified conditional claim (compare-and-swap) before spawn — closes RC-1.** + Make the pre-dispatch claim atomic at the DB layer: transition to + `in_progress` only if the item is still in the selectable state the tier + chose it in. Option A (recommended): `wl update --status in_progress + --if-status [--if-stage ]` — a guarded update + added to `src/commands/update.ts` + `packages/shared/src/database.ts + update()`. Option B: a DB-level guarded update exposed via a new CLI flag. + Exactly one concurrent pane wins; a loser (0 rows changed, or claim + failure) **aborts the dispatch** — no pane, no marker, no success log. + This also absorbs WL-0MSLWJ310000ND0X (claim failures must not be + silently discarded when they mean "another pane won"). +2. **Marker write before spawn + fail-closed — hardens RC-1 at the log layer.** + Move `recordDispatch`'s log append ahead of `spawnAgentPane` in every tier + (`dispatchDowntimeWork`, `downtime-worker.ts:232-280`); if the marker + cannot be written, abort the dispatch (fail-closed) rather than dispatching + an unmarked item. The work-item comment may stay post-spawn (best-effort + trail); the **log marker is the exclusion source and must precede the + pane**. +3. **Extend the dispatched-marker exclusion to the plan and intake tiers — + closes RC-2.** Add kind-scoped marker sets (`plan`/`intake` entries) to + `downtime-log.ts` (`planDispatchedItemIds`/`intakeDispatchedItemIds`, + mirroring `auditDispatchedItemIds`/`implementDispatchedItemIds`) and wire + them into `getNextItem`'s plan/intake selection paths in `index.ts`. Add a + **change-guard**: exclude while the item is still at the stage it had at + dispatch (`intake_complete` for plan, `idea` for intake); a stage + advancement or an explicit human/state change releases the item for a + legitimate retry (prevents plan-retry starvation, risk below). +4. **Client-side open-status guard on plan/intake tiers — closes the + amplifier.** Filter to `status === 'open'` in the plan/intake + parse/select paths (mirroring the implement tier's guard in + `selectImplementCandidate`, `downtime-worker.ts`), so a `completed` item + whose stage matches the filter is never dispatched for + `/skill:plan`/`/skill:intake`. Requires `parseNextItemOutput` to carry + `status` through (`DowntimeCandidate` gains a `status` field). +5. **Record dependencies/follow-ups:** WL-0MSLWJ3I70031Z8U (spawn failure + still logged as success — with marker-before-spawn this becomes an abort + path) and document the 100-entry log-roll limit as a residual risk (or + extend `DOWNTIME_LOG_MAX_ENTRIES`). + +### Option choices (explicit) + +- **Conditional claim:** Option A (`wl update --if-status`/`--if-stage` + guarded update) chosen as primary — reuses the existing CLI seam, keeps the + dispatch code DB-agnostic; a DB-level guarded update is the fallback if the + CLI flag cannot express `completed` (audit tier) + `open` (implement/plan/ + intake) uniformly. +- **Marker-before-spawn:** log append moves ahead of spawn in **all four + tiers** (audit, implement, plan, intake); fail-closed abort on write error. +- **Change-guard semantics:** marker releases when the item's stage advances + past the dispatched-at stage, or when a human/agent state change updates + the item. Purely time-based release is **not** chosen (would re-open the + re-selection loop). + +## Key files + +| File | Role | +|---|---| +| `packages/herdr/src/downtime-worker.ts` | `dispatchDowntimeWork` tier order; selection; where claim/spawn/marker ordering changes land | +| `packages/herdr/src/index.ts` | `getNextItem` (~351-383), `getNextAuditCandidate` (~378-410), `getNextImplementCandidate` (~413-455), `claimItem` (~456-462), `recordDispatch` (~465-490) | +| `packages/herdr/src/fetcher.ts` | `claimWorkItem` (~623) — where the guarded claim lands | +| `packages/herdr/src/downtime-log.ts` | marker source; extend with `plan`/`intake` kinds | +| `packages/shared/src/database.ts` | `filterCandidates` (stage-filter amplifier); `update()` (guarded update) | +| `src/commands/update.ts` | `--if-status`/`--if-stage` CLI flags | +| `skill/plan/SKILL.md` | error/abort resets status only (RC-2 mechanism) | +| `skill/audit/scripts/audit_runner.py` | finally-block status/stage restore path | +| `packages/herdr/scripts/scan_duplicate_dispatches.py` | evidence scan (this RCA) | + +## Traceability + +- **Prior fix:** WL-0MSLIY8ZR004QUSY "Downtime dispatcher triggers multiple + identical sessions" (completed) — audit-tier-only marker exclusion; the + "self-guard" assumption this RCA refutes. Commit `228019bd`. +- **Exclusion implementation:** WL-0MSLUASNF001LLAT (completed). +- **Regression tests to extend:** WL-0MSLUANM5004NNPU "Regression tests for + audit-tier re-dispatch exclusion" (completed) — extend for plan/intake. +- **Open dependencies (absorbed by design):** WL-0MSLWJ310000ND0X (claim + failure discarded → design point 1), WL-0MSLWJ3I70031Z8U (spawn failure + logged as success → design point 2). +- **Wiki:** `concepts/downtime-dispatch` updated with this RCA's findings. + +## Risks & assumptions + +- **Assumption — same-host multi-pane is the observed scope.** The RC-1 + evidence is same-host (SorraAgents root, one host). Cross-host duplicates + (two hosts sharing a worklog root) remain out of scope; the per-item + worklog comment remains the durable cross-machine trail. +- **Risk — conditional claim semantics change claim behavior for all + dispatches** (implement tier uses `claimItem` too): the CAS must accept + every status the tier's query can return (`open` for plan/intake/implement, + `completed` for audit) — design point 1's `--if-status` list. +- **Risk — plan retry starvation:** design point 3's change-guard must release + the marker when the item legitimately advances, so a genuinely replanned + item is not permanently blocked; define and test the change-guard precisely. +- **Risk — log roll:** the 100-entry bound can silently drop markers for + long-unaudited items; documented residual risk (design point 5) unless + trivially extended. + +## Validation script + +`packages/herdr/scripts/scan_duplicate_dispatches.py` reproduces the duplicate +scan (used for this RCA; also useful for verifying the follow-up fix — the +post-fix scan should show no audit pairs beyond a same-instant race and no +plan/intake pairs while items stay at their dispatched stage): + +```bash +python3 packages/herdr/scripts/scan_duplicate_dispatches.py --since +``` diff --git a/packages/herdr/scripts/scan_duplicate_dispatches.py b/packages/herdr/scripts/scan_duplicate_dispatches.py new file mode 100755 index 000000000..e3ccd0e75 --- /dev/null +++ b/packages/herdr/scripts/scan_duplicate_dispatches.py @@ -0,0 +1,177 @@ +#!/usr/bin/env python3 +"""Scan herdr downtime dispatch logs for re-dispatched work items. + +Reproduces the duplicate-dispatch evidence scan from RCA +WL-0MSRBFFLN005W3VT ("RCA: Downtime dispatcher re-dispatches same item"): +for every `.worklog/downtime-dispatches.log` under the given roots, group +entries by (kind, itemId) and report every pair dispatched more than once, +with the first/last dispatch timestamps and the gap between them. + +A duplicate pair means the downtime worker opened more than one pane for the +same work item without the item's state durably changing between dispatches +— the re-dispatch behaviour the RCA investigates. + +Usage: + python3 packages/herdr/scripts/scan-duplicate-dispatches.py [root ...] [--json] [--since ISO] + +Roots default to the five worklog roots observed on this host. `--since` +filters to entries dispatched at or after the given ISO-8601 timestamp +(useful for focusing on post-fix evidence). Output is a human-readable table +by default; `--json` emits a machine-readable report. Exit code is always 0 +(the tool reports; it does not fail on findings). +""" + +from __future__ import annotations + +import argparse +import json +import sys +from collections import defaultdict +from datetime import datetime +from pathlib import Path + +# The five worklog roots with an active downtime dispatch log on this host +# (dispatch logs are written under each project's `.worklog/` directory). +DEFAULT_ROOTS = [ + "/home/rgardler/projects/ContextHub", + "/home/rgardler/projects/SorraAgents", + "/home/rgardler/projects/Tableau-Card-Engine", + "/home/rgardler/projects/dev-scripts", + "/home/rgardler/projects/open_source_llm", +] + +LOG_REL = Path(".worklog") / "downtime-dispatches.log" + + +def read_entries(log_path: Path) -> list[dict]: + """Parse a dispatch log file into entries (fail-safe: skip bad lines). + + Returns an empty list for a missing/unreadable file, mirroring + `readDowntimeLogEntries` in packages/herdr/src/downtime-log.ts. + """ + entries: list[dict] = [] + try: + raw = log_path.read_text(encoding="utf-8") + except OSError: + return [] + for line in raw.splitlines(): + line = line.strip() + if not line: + continue + try: + parsed = json.loads(line) + except (json.JSONDecodeError, ValueError): + continue + if isinstance(parsed, dict): + entries.append(parsed) + return entries + + +def find_duplicates( + roots: list[str], since: str | None = None +) -> list[dict]: + """Scan the roots' dispatch logs for (kind, itemId) pairs dispatched >1. + + Returns a list of report dicts, one per duplicate pair: + ``{root, kind, itemId, title, count, firstDispatchedAt, lastDispatchedAt, + gapSeconds}``. Entries without an itemId/kind (e.g. persistent-error + events) are ignored. + """ + cutoff = None + if since: + cutoff = datetime.fromisoformat(since.replace("Z", "+00:00")) + + reports: list[dict] = [] + for root in roots: + log_path = Path(root) / LOG_REL + entries = read_entries(log_path) + by_pair: dict[tuple[str, str], list[dict]] = defaultdict(list) + for e in entries: + item_id = e.get("itemId") + kind = e.get("kind") + dispatched_at = e.get("dispatchedAt") + if not isinstance(item_id, str) or not item_id: + continue + if not isinstance(kind, str) or not kind: + continue + if not isinstance(dispatched_at, str) or not dispatched_at: + continue + if cutoff: + try: + ts = datetime.fromisoformat(dispatched_at.replace("Z", "+00:00")) + except ValueError: + continue + if ts < cutoff: + continue + by_pair[(kind, item_id)].append(e) + + for (kind, item_id), group in sorted(by_pair.items()): + if len(group) < 2: + continue + times = [g["dispatchedAt"] for g in group] + times_sorted = sorted(times) + gap_seconds = None + if len(times_sorted) >= 2: + t0 = datetime.fromisoformat(times_sorted[0].replace("Z", "+00:00")) + t1 = datetime.fromisoformat(times_sorted[-1].replace("Z", "+00:00")) + gap_seconds = round((t1 - t0).total_seconds()) + reports.append( + { + "root": str(Path(root).resolve()), + "kind": kind, + "itemId": item_id, + "title": group[0].get("title", ""), + "count": len(group), + "firstDispatchedAt": times_sorted[0], + "lastDispatchedAt": times_sorted[-1], + "gapSeconds": gap_seconds, + } + ) + return reports + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser( + description=( + "Scan herdr downtime dispatch logs for work items dispatched " + "more than once (RCA WL-0MSRBFFLN005W3VT evidence scan)." + ) + ) + parser.add_argument("roots", nargs="*", help="worklog roots to scan (default: the five host roots)") + parser.add_argument( + "--json", action="store_true", help="emit machine-readable JSON report" + ) + parser.add_argument( + "--since", + metavar="ISO", + help="only consider entries dispatched at/after this ISO-8601 timestamp", + ) + args = parser.parse_args(argv) + + roots = args.roots or DEFAULT_ROOTS + reports = find_duplicates(roots, since=args.since) + + if args.json: + print(json.dumps({"roots": [str(Path(r).resolve()) for r in roots], "duplicates": reports}, indent=2)) + return 0 + + if not reports: + print("No duplicate (kind, itemId) dispatch pairs found.") + return 0 + + print(f"Duplicate dispatch pairs across {len(roots)} roots:\n") + for r in reports: + gap = f"{r['gapSeconds']}s" if r["gapSeconds"] is not None else "?" + print( + f" {r['kind']:<10} {r['itemId']} x{r['count']} " + f"({r['root']})" + ) + print(f" first={r['firstDispatchedAt']} last={r['lastDispatchedAt']} gap={gap}") + if r["title"]: + print(f" title: {r['title']}") + print(f"\nTotal duplicate pairs: {len(reports)}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/packages/herdr/scripts/tests/test_scan_duplicate_dispatches.py b/packages/herdr/scripts/tests/test_scan_duplicate_dispatches.py new file mode 100644 index 000000000..6967a8c9f --- /dev/null +++ b/packages/herdr/scripts/tests/test_scan_duplicate_dispatches.py @@ -0,0 +1,98 @@ +"""Tests for the herdr duplicate-dispatch scan script. + +Validates the evidence-reproduction logic behind RCA WL-0MSRBFFLN005W3VT: +grouping `.worklog/downtime-dispatches.log` entries by (kind, itemId) and +reporting pairs dispatched more than once. Every test asserts observable +behaviour of `find_duplicates` / `read_entries` via the public API. + +Run with: + cd /path/to/ContextHub + python3 -m pytest packages/herdr/scripts/tests/test_scan_duplicate_dispatches.py -v +""" +from __future__ import annotations + +import json +import sys +from pathlib import Path + +import pytest + +_SCRIPTS_DIR = Path(__file__).resolve().parent.parent +if str(_SCRIPTS_DIR) not in sys.path: + sys.path.insert(0, str(_SCRIPTS_DIR)) + +from scan_duplicate_dispatches import find_duplicates, read_entries + + +@pytest.fixture() +def root_a(tmp_path: Path) -> Path: + """A worklog root whose dispatch log contains one duplicate pair and one + singleton (a normal dispatch), plus a malformed line to exercise the + fail-safe parse.""" + log = tmp_path / ".worklog" / "downtime-dispatches.log" + log.parent.mkdir(parents=True) + log.write_text( + "\n".join( + [ + json.dumps( + {"itemId": "WL-A", "kind": "audit", "dispatchedAt": "2026-08-10T21:20:53.909Z", "cwd": str(tmp_path), "title": "t"} + ), + json.dumps( + {"itemId": "WL-A", "kind": "audit", "dispatchedAt": "2026-08-10T21:20:53.923Z", "cwd": str(tmp_path), "title": "t"} + ), + json.dumps( + {"itemId": "WL-B", "kind": "plan", "dispatchedAt": "2026-08-12T11:05:30.321Z", "cwd": str(tmp_path), "title": "once"} + ), + "not-json{{{", + ] + ) + + "\n" + ) + return tmp_path + + +def test_read_entries_skips_malformed_lines_and_missing_file(root_a: Path) -> None: + entries = read_entries(root_a / ".worklog" / "downtime-dispatches.log") + assert len(entries) == 3 # the malformed line is skipped (fail-safe) + assert read_entries(root_a / "no-such" / "downtime-dispatches.log") == [] + + +def test_find_duplicates_reports_re_dispatched_pairs(root_a: Path) -> None: + reports = find_duplicates([str(root_a)]) + assert len(reports) == 1 + r = reports[0] + assert r["kind"] == "audit" + assert r["itemId"] == "WL-A" + assert r["count"] == 2 + assert r["firstDispatchedAt"] == "2026-08-10T21:20:53.909Z" + assert r["lastDispatchedAt"] == "2026-08-10T21:20:53.923Z" + assert r["gapSeconds"] == 0 # 14 ms rounds to 0 s + + +def test_find_duplicates_ignores_singletons(root_a: Path) -> None: + reports = find_duplicates([str(root_a)]) + assert all(r["itemId"] != "WL-B" for r in reports) # single dispatch is not a duplicate + + +def test_find_duplicates_aggregates_across_roots(tmp_path: Path) -> None: + root_b = tmp_path / "root-b" + (root_b / ".worklog").mkdir(parents=True) + (root_b / ".worklog" / "downtime-dispatches.log").write_text( + json.dumps({"itemId": "WL-C", "kind": "plan", "dispatchedAt": "2026-08-09T00:46:28.620Z", "cwd": str(root_b)}) + + "\n" + + json.dumps({"itemId": "WL-C", "kind": "plan", "dispatchedAt": "2026-08-09T01:17:00.634Z", "cwd": str(root_b)}) + + "\n" + ) + reports = find_duplicates([str(root_b)]) + assert len(reports) == 1 + assert reports[0]["itemId"] == "WL-C" + assert reports[0]["count"] == 2 + assert reports[0]["root"] == str(root_b.resolve()) + + +def test_find_duplicates_since_filters_to_recent_entries(root_a: Path) -> None: + # Both WL-A entries are 2026-08-10; a since cutoff after them excludes the pair. + reports = find_duplicates([str(root_a)], since="2026-08-11T00:00:00Z") + assert reports == [] + reports = find_duplicates([str(root_a)], since="2026-08-10T21:20:53.000Z") + assert len(reports) == 1 From 76ac767e95bd2855f18469284ec1437cf8956a7a Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 13:10:44 +0100 Subject: [PATCH 03/34] WL-0MSRDEWES0059TZN: Downtime dispatcher CAS claim + plan/intake marker exclusion Implements the RCA fix design (WL-0MSRBFFLN005W3VT) in the herdr downtime worker and the wl CLI: - CAS claim: `wl update --status in_progress --if-status [--if-stage ]` runs the check+write in one `BEGIN IMMEDIATE` transaction (updateIfMatches); the downtime pre-dispatch claim uses it, so exactly one concurrent pane wins and a loser aborts with no pane, no marker, no success record (closes the same-instant race, RC-1). - Marker-before-spawn: the .worklog/downtime-dispatches.log marker is written before the pane spawns; a marker write failure aborts the dispatch (fail-closed), and a handled spawn error yields a non-success outcome (spawn-failed) instead of a false success (absorbs WL-0MSLWJ310000ND0X and WL-0MSLWJ3I70031Z8U). - Plan/intake dispatched-marker exclusion with change-guard: kind-scoped plan/intake markers record the item's stage at dispatch; a candidate is excluded while still at its dispatched-at stage and released on stage advancement (closes RC-2). - Client-side open-status guard on plan/intake tiers (mirrors the implement tier) so completed/in_review items are never dispatched. Also: claim failures are never silently discarded (outcome reason 'claim-failed'/'wl-error' + stderr), and getNextItem now fetches a batch (-n 10) so marker-excluded candidates do not starve selection. --- packages/herdr/README.md | 99 +++--- packages/herdr/src/claim.test.ts | 31 ++ packages/herdr/src/downtime-log.test.ts | 45 +++ packages/herdr/src/downtime-log.ts | 70 +++- packages/herdr/src/downtime-worker.test.ts | 360 ++++++++++++++++++--- packages/herdr/src/downtime-worker.ts | 346 ++++++++++++++++---- packages/herdr/src/fetcher.ts | 43 ++- packages/herdr/src/index.test.ts | 149 +++++++-- packages/herdr/src/index.ts | 85 +++-- packages/shared/src/database.test.ts | 68 ++++ packages/shared/src/database.ts | 51 +++ packages/shared/src/persistent-store.ts | 16 + packages/shared/src/types.ts | 13 + src/cli-types.ts | 4 + src/commands/update.ts | 65 +++- tests/cli/update-if-status.test.ts | 99 ++++++ 16 files changed, 1332 insertions(+), 212 deletions(-) create mode 100644 tests/cli/update-if-status.test.ts diff --git a/packages/herdr/README.md b/packages/herdr/README.md index 235a603ff..f8424fec2 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -342,9 +342,14 @@ true`), then stop — it never blocks indefinitely. **No lock file (cross-pane decision Q5)** — concurrent panes are serialized by the idle→busy cadence (a dispatch consumes the local slot, so the proxy reports busy and the worker requires a fresh full idle period) and by the -pre-dispatch claim; there is deliberately **no cross-pane lock file**. A -small residual risk of two panes dispatching at the same idle boundary is -accepted — the claim prevents duplicate work on the same item. +pre-dispatch claim, which is a **compare-and-swap** (`wl update +--status in_progress --if-status [--if-stage ]`, RCA +WL-0MSRBFFLN005W3VT design point 1): the transition only applies while the +item is still in the exact state the tier selected it in, so exactly one +concurrent pane wins and a losing pane aborts its dispatch (no pane, no +marker, no success record). There is deliberately **no cross-pane lock +file** — the CAS claim is the serialization point, and it is atomic at the +SQLite layer (`BEGIN IMMEDIATE`). **Audit trail** — every successful dispatch records two traces: (1) a comment on the dispatched item (`wl comment add`, author @@ -352,61 +357,67 @@ comment on the dispatched item (`wl comment add`, author UTC timestamp — this survives `wl sync` and is the durable trail; and (2) a bounded JSONL entry in `.worklog/downtime-dispatches.log` under the resolved worklog root (rolling — only the most recent 100 entries are -kept). The `kind:audit` entries in this log double as the dispatched-marker -exclusion source for the audit tier (WL-0MSLIY8ZR004QUSY): an item the -worker already dispatched for `/skill:audit` is excluded from later audit -tier selection while it still lacks a fresh audit (plan/intake markers are -scoped to their own tiers and never suppress audit selection). A -three-strike CLI-error pause additionally writes a JSONL entry to -the same rolling log (with the `at` timestamp and an error message) so the -persistent failure is auditable even though nothing was dispatched. The -`.worklog` log file is gitignored and local-only; all writes are -fail-closed, so a comment or log failure never blocks or fails a dispatch. +kept). The marker is written **before** the pane spawns (fail-closed: an +unmarked item is never dispatched). The `kind:audit` entries double as the +dispatched-marker exclusion source for the audit tier (WL-0MSLIY8ZR004QUSY); +`kind:implement` entries for the implement tier; and `kind:plan` / +`kind:intake` entries (which also record the item's `stage` at dispatch) +for the plan/intake change-guard — an item already dispatched for its tier +is excluded while it is still at its dispatched-at stage, and a stage +advancement releases it (RCA WL-0MSRBFFLN005W3VT design point 3). Plan / +intake markers are scoped to their own tiers and never suppress audit +selection. A three-strike CLI-error pause additionally writes a JSONL entry +to the same rolling log (with the `at` timestamp and an error message) so +the persistent failure is auditable even though nothing was dispatched. The +`.worklog` log file is gitignored and local-only. **Failure-path logging** — a complete account of what each dispatch outcome leaves behind (documented for WL-0MSKUG2WW0058A7W, audit gap AC2): | Outcome | Trace in `.worklog/downtime-dispatches.log` | Notes | |---|---|---| -| Successful dispatch | comment on the item + JSONL entry (`kind`, `itemId`, `dispatchedAt`) | the only fully-visible outcome | +| Successful dispatch | comment on the item + JSONL entry (`kind`, `itemId`, `dispatchedAt`, `stage` for plan/intake) | the only fully-visible success outcome | | Genuine empty backlog (no-candidate) | **none** — intentionally silent | full cooldown pause (default 60 min); worker stops polling | | 1–2 transient wl CLI errors (strikes) | **none** — silent | one strike per `wl-error` outcome; retries on the next idle window | | 3rd consecutive wl CLI error | `recordError` JSONL entry | three-strike pause; the only failure path that logs | -| Audit-tier wl/parse failure | **none** — silent, and **no strike** | `getNextAuditCandidate` collapses the failure to `null` like an empty tier (index.ts:389-391); worker falls through to the plan tier and looks healthy — known silent path, follow-up WL-0MSLWJ2KP0002SV0 | -| Claim failure (`wl update --status in_progress`) | **none** — result discarded | dispatch proceeds and is still recorded as a success; the claim is the cross-pane serialization guard, so a failed claim leaves the item selectable by another pane — known silent path, follow-up WL-0MSLWJ310000ND0X | -| Pane spawn failure (`send-to-pi.sh`) | **none** — invisible | spawn is detached/`stdio: ignore`/`unref`d with no `error`/`exit` handler; a failed spawn (or a script that exits non-zero) is never observed and the dispatch is still logged as a success — known silent path, follow-up WL-0MSLWJ3I70031Z8U | -| `recordDispatch` / `recordError` write failure | **none** | fail-closed by design: logging must never crash or block the worker | - -Consequence: the log's *absence* of an entry is ambiguous — it cannot -distinguish "no candidate (paused)" from "worker disabled" or from any of -the silent failure paths above; only the three-strike and success paths leave -entries. The follow-up items above close the three code-level silent paths. +| Audit-tier wl/parse failure | **none** — silent, and **no strike** | `getNextAuditCandidate` collapses the failure to `null` like an empty tier; worker falls through to the plan tier and looks healthy — known silent path, follow-up WL-0MSLWJ2KP0002SV0 | +| Lost CAS claim race (`--if-status`/`--if-stage` stale) | **none** — and **no marker, no pane, no success record** | the dispatch ABORTS with reason `claim-failed` (neutral — another pane won); the failure is observable via the outcome and a stderr line, never silently discarded (WL-0MSLWJ310000ND0X absorbed) | +| Claim wl CLI failure (non-stale) | **none** — counts as a `wl-error` strike | dispatch aborts; three consecutive such failures pause the worker | +| Marker write failure | **none** — the item stays claimed (`in_progress`) | dispatch ABORTS **before** the pane spawns with reason `marker-write-failed` (fail-closed: an unmarked item is never dispatched; the claim still removes it from `wl next`, so no other pane selects it) | +| Pane spawn failure (`send-to-pi.sh`) | marker already written (pre-spawn) | a spawn `error` is handled (no unhandled-exception crash) and the outcome is **not** success (`spawn-failed`); the marker stands so the item is not re-dispatched (WL-0MSLWJ3I70031Z8U absorbed) | +| `recordError` write failure | **none** | fail-closed by design: logging must never crash or block the worker | + +Consequence: the log's *absence* of an entry is still ambiguous (it cannot +distinguish "no candidate (paused)" from "worker disabled" or from a +lost claim race), but the code-level silent failure paths are closed: a +failed claim or spawn can no longer produce a false success record. **Known re-dispatch gaps** (RCA WL-0MSRBFFLN005W3VT, evidence reproduced by `packages/herdr/scripts/scan_duplicate_dispatches.py`; see [docs/downtime-redispatch-rca-2026-08-13.md](docs/downtime-redispatch-rca-2026-08-13.md)): -- **Same-instant cross-pane race** — two panes can select the same candidate - before either claims it: selection precedes claim, and the pre-dispatch - claim (`wl update --status in_progress`) is idempotent, so both panes - proceed. Observed post-fix: a `kind:audit` pair 14 ms apart - (SA-0MSN4AXIQ007IZG2, 2026-08-10). The dispatched-marker exclusion is - read-then-write and written after spawn, so it cannot prevent this. Fix - design (follow-up): atomic conditional claim (`--if-status`/`--if-stage`) - — a losing pane aborts the dispatch. -- **Plan/intake tiers have no dispatched-marker exclusion** — the audit and - implement tiers exclude already-dispatched items; the plan - (`--stage intake_complete`) and intake (`--stage idea`) tiers do not, and - `wl next` keeps `completed` items under a stage filter, so a plan run whose - error/abort path resets status only (stage left at `intake_complete`) - leaves the item selectable for re-dispatch. Observed: `kind:plan` ×2 on - SA-0MSMAZP6T007NM0O and SA-0MSN04X2S006ONH0 (2026-08-12/13), `kind:plan` ×7 - in ~7 h pre-fix (SA-0MSJI53RX006E2PS). Fix design (follow-up): kind-scoped - plan/intake markers with a change-guard plus a client-side `status === - 'open'` filter. - -These are documented as design candidates in the RCA; implementation is a -follow-up work item. +These were closed by WL-0MSRDEWES0059TZN (implement RCA fix design): + +- **Same-instant cross-pane race (closed)** — the pre-dispatch claim is now a + compare-and-swap (`--if-status`/`--if-stage`) executed atomically at the + SQLite layer, so exactly one pane wins; a losing pane aborts with no pane, + no marker, no success record. (Formerly: selection preceded an idempotent + claim, so two panes could both proceed — observed as a `kind:audit` pair + 14 ms apart, SA-0MSN4AXIQ007IZG2, 2026-08-10.) +- **Plan/intake tiers had no dispatched-marker exclusion (closed)** — the + plan (`--stage intake_complete`) and intake (`--stage idea`) tiers now + exclude items already dispatched for their tier while the item is still at + its dispatched-at stage (kind-scoped `plan`/`intake` markers carrying the + stage at dispatch; a stage advancement releases the item), and both tiers + filter client-side to `status === 'open'` so a `completed`/`in_review` + item whose stage matches is never dispatched. (Formerly: a plan run whose + error/abort path reset status only — stage left at `intake_complete` — + left the item selectable; observed `kind:plan` ×2 on + SA-0MSMAZP6T007NM0O and SA-0MSN04X2S006ONH0, `kind:plan` ×7 in ~7 h + pre-fix, SA-0MSJI53RX006E2PS.) + +The remaining residual risk is the 100-entry log roll: a very long-unaudited +item can have its marker rolled out of the log, allowing one re-dispatch. The worker runs inside the plugin's single consolidated scheduler loop (one `setInterval`; no independent timers), uses unref'd timers, and is cleaned up diff --git a/packages/herdr/src/claim.test.ts b/packages/herdr/src/claim.test.ts index 89acce423..0d1422c11 100644 --- a/packages/herdr/src/claim.test.ts +++ b/packages/herdr/src/claim.test.ts @@ -91,6 +91,35 @@ describe('claimWorkItem (fetcher)', () => { expect(callArgs).toContain('--json'); }); + it('passes --if-status/--if-stage when an expected state is given (CAS claim)', async () => { + const mockFn = vi.fn().mockResolvedValue({ + stdout: '{"success":true}', + stderr: '', + }); + setExecFileAsync(mockFn as any); + + const result = await claimWorkItem('WL-ABC', 'Map', { status: 'open', stage: 'idea' }); + + expect(result.success).toBe(true); + const callArgs = mockFn.mock.calls[0][1] as string[]; + expect(callArgs).toContain('--if-status'); + expect(callArgs[callArgs.indexOf('--if-status') + 1]).toBe('open'); + expect(callArgs).toContain('--if-stage'); + expect(callArgs[callArgs.indexOf('--if-stage') + 1]).toBe('idea'); + }); + + it('marks a lost CAS race as stale (another pane won)', async () => { + const mockFn = vi + .fn() + .mockRejectedValue(new Error('{"success":false,"error":"stale","message":"Conditional update skipped"}')); + setExecFileAsync(mockFn as any); + + const result = await claimWorkItem('WL-ABC', 'Map', { status: 'completed', stage: 'in_review' }); + + expect(result.success).toBe(false); + expect(result.stale).toBe(true); + }); + it('includes --worklog-dir when the fetcher is configured', async () => { const mockFn = vi.fn().mockResolvedValue({ stdout: '{"success":true}', @@ -116,6 +145,8 @@ describe('claimWorkItem (fetcher)', () => { expect(result.success).toBe(false); expect(result.error).toBeDefined(); + // A non-stale failure is NOT a lost race. + expect(result.stale).toBeFalsy(); }); }); diff --git a/packages/herdr/src/downtime-log.test.ts b/packages/herdr/src/downtime-log.test.ts index 0c4bfef30..6b7e56bae 100644 --- a/packages/herdr/src/downtime-log.test.ts +++ b/packages/herdr/src/downtime-log.test.ts @@ -15,6 +15,9 @@ import { appendDowntimeLogEntry, auditDispatchedItemIds, implementDispatchedItemIds, + planDispatchedItemStages, + intakeDispatchedItemStages, + dispatchedItemStages, readDowntimeLogEntries, DOWNTIME_LOG_FILE, DOWNTIME_LOG_MAX_ENTRIES, @@ -157,3 +160,45 @@ describe('implementDispatchedItemIds (implement-tier-only scope guard)', () => { expect([...implementDispatchedItemIds([])]).toEqual([]); }); }); + +describe('plan/intake dispatched-item stages (change-guard maps)', () => { + it('planDispatchedItemStages maps plan markers to their dispatched-at stage', () => { + const stages = planDispatchedItemStages([ + { itemId: 'WL-A', kind: 'plan', stage: 'intake_complete' }, + { itemId: 'WL-B', kind: 'intake', stage: 'idea' }, + { itemId: 'WL-C', kind: 'plan' }, // legacy entry without stage → '' + { kind: 'plan' }, // no itemId → ignored + ]); + expect(stages.get('WL-A')).toBe('intake_complete'); + expect(stages.get('WL-C')).toBe(''); + expect(stages.has('WL-B')).toBe(false); // intake markers are scoped out + }); + + it('intakeDispatchedItemStages maps intake markers to their dispatched-at stage', () => { + const stages = intakeDispatchedItemStages([ + { itemId: 'WL-A', kind: 'intake', stage: 'idea' }, + { itemId: 'WL-B', kind: 'plan', stage: 'intake_complete' }, + ]); + expect(stages.get('WL-A')).toBe('idea'); + expect(stages.has('WL-B')).toBe(false); + }); + + it('dispatchedItemStages is kind-scoped and tolerant of malformed entries', () => { + const stages = dispatchedItemStages( + [ + { itemId: 'WL-A', kind: 'plan', stage: 'intake_complete' }, + { itemId: 'WL-B', kind: 'plan', stage: 42 as unknown as string }, // non-string stage → '' + { itemId: 'WL-C', kind: 'plan' }, + ], + 'plan', + ); + expect(stages.get('WL-A')).toBe('intake_complete'); + expect(stages.get('WL-B')).toBe(''); + expect(stages.get('WL-C')).toBe(''); + }); + + it('returns an empty map for empty input', () => { + expect(planDispatchedItemStages([]).size).toBe(0); + expect(intakeDispatchedItemStages([]).size).toBe(0); + }); +}); diff --git a/packages/herdr/src/downtime-log.ts b/packages/herdr/src/downtime-log.ts index b4faa1f34..eee5ff005 100644 --- a/packages/herdr/src/downtime-log.ts +++ b/packages/herdr/src/downtime-log.ts @@ -38,6 +38,14 @@ export interface DowntimeLogEntry { dispatchedAt?: string; cwd?: string; title?: string; + /** + * Worklog stage of the item AT dispatch (set on plan/intake markers, + * RCA WL-0MSRBFFLN005W3VT design point 3). Powers the change-guard: a + * candidate is excluded while it is still at its dispatched-at stage; a + * stage advancement releases it. Absent on legacy entries (backward + * compatible — a missing stage never suppresses selection). + */ + stage?: string; at?: string; message?: string; } @@ -77,21 +85,31 @@ export async function readDowntimeLogEntries(cwd: string): Promise { +function dispatchedItemIds(entries: DowntimeLogEntry[], kind: string): Set { const ids = new Set(); for (const e of entries) { - if (e.kind === 'audit' && typeof e.itemId === 'string' && e.itemId.length > 0) { + if (e.kind === kind && typeof e.itemId === 'string' && e.itemId.length > 0) { ids.add(e.itemId); } } return ids; } +/** + * Build the set of itemIds the downtime worker has already dispatched for + * `/skill:audit` (`kind === 'audit'` entries only). Plan/intake markers are + * scoped to their own tiers and must NOT suppress audit-tier selection + * (audit-tier-only scope guard, WL-0MSLIY8ZR004QUSY). Entries without an + * itemId (e.g. persistent-error events) are ignored. + */ +export function auditDispatchedItemIds(entries: DowntimeLogEntry[]): Set { + return dispatchedItemIds(entries, 'audit'); +} + /** * Build the set of itemIds the downtime worker has already dispatched for * `/skill:implement` (`kind === 'implement'` entries only). Audit/plan/ @@ -101,13 +119,45 @@ export function auditDispatchedItemIds(entries: DowntimeLogEntry[]): Set * events) are ignored. */ export function implementDispatchedItemIds(entries: DowntimeLogEntry[]): Set { - const ids = new Set(); + return dispatchedItemIds(entries, 'implement'); +} + +/** + * Build the kind-scoped id → dispatched-at-stage map used by the plan/intake + * change-guard (RCA WL-0MSRBFFLN005W3VT design point 3): entries of the + * given kind map itemId to the worklog stage the item had when it was + * dispatched. Selection excludes a candidate while it is still at its + * dispatched-at stage; a stage advancement (or any stage differing from the + * recorded one) releases it. Entries without a recorded stage map to '' — a + * missing stage never suppresses selection (legacy pre-fix entries stay + * valid and do not freeze an item). + */ +export function dispatchedItemStages(entries: DowntimeLogEntry[], kind: string): Map { + const stages = new Map(); for (const e of entries) { - if (e.kind === 'implement' && typeof e.itemId === 'string' && e.itemId.length > 0) { - ids.add(e.itemId); + if (e.kind === kind && typeof e.itemId === 'string' && e.itemId.length > 0) { + stages.set(e.itemId, typeof e.stage === 'string' ? e.stage : ''); } } - return ids; + return stages; +} + +/** + * Plan-tier marker map (`kind === 'plan'` entries only): itemId → stage at + * dispatch. A plan candidate (`intake_complete`) is excluded while a plan + * marker records the same stage; advancing the item releases it. + */ +export function planDispatchedItemStages(entries: DowntimeLogEntry[]): Map { + return dispatchedItemStages(entries, 'plan'); +} + +/** + * Intake-tier marker map (`kind === 'intake'` entries only): itemId → stage + * at dispatch. An intake candidate (`idea`) is excluded while an intake + * marker records the same stage; advancing the item releases it. + */ +export function intakeDispatchedItemStages(entries: DowntimeLogEntry[]): Map { + return dispatchedItemStages(entries, 'intake'); } /** diff --git a/packages/herdr/src/downtime-worker.test.ts b/packages/herdr/src/downtime-worker.test.ts index bbafeaadc..e0b7f3a8d 100644 --- a/packages/herdr/src/downtime-worker.test.ts +++ b/packages/herdr/src/downtime-worker.test.ts @@ -36,8 +36,10 @@ import { buildDowntimeSpawnOptions, BLOCKED_QUESTIONS_INSTRUCTION, parseNextItemOutput, + parseNextCandidatesOutput, parseImplementCandidatesOutput, selectImplementCandidate, + selectNextCandidate, parseAuditCandidatesOutput, selectAuditCandidate, toDowntimeCandidate, @@ -57,6 +59,7 @@ import { type DowntimeCandidate, type DowntimeWorkerDeps, type DowntimeSpawn, + type DowntimeStage, type AuditCandidate, type ImplementCandidate, } from './downtime-worker.js'; @@ -77,9 +80,9 @@ function makeDeps(overrides: Partial = {}): DowntimeWorkerDe getNextItem: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextAuditCandidate: vi.fn().mockResolvedValue(null), getNextImplementCandidate: vi.fn().mockResolvedValue(null), - claimItem: vi.fn().mockResolvedValue(undefined), - spawnAgentPane: vi.fn().mockResolvedValue(undefined), - recordDispatch: vi.fn().mockResolvedValue(undefined), + claimItem: vi.fn().mockResolvedValue({ ok: true }), + spawnAgentPane: vi.fn().mockResolvedValue(true), + recordDispatch: vi.fn().mockResolvedValue(true), recordError: vi.fn().mockResolvedValue(undefined), // Code-freeze gate (WL-0MSQ0RPQP00636JY): not frozen by default, so // existing dispatch tests exercise the unchanged audit/implement tiers. @@ -216,13 +219,13 @@ describe('dispatch selection', () => { const deps = makeDeps({ getNextItem: vi.fn().mockResolvedValue({ ok: true, - candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete' }, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open' }, }), }); const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); - expect(deps.getNextItem).toHaveBeenCalledWith('intake_complete'); + expect(deps.getNextItem).toHaveBeenCalledWith('intake_complete', '/repo'); expect(deps.getNextItem).toHaveBeenCalledTimes(1); expect(outcome.dispatched).toBe(true); expect(outcome.kind).toBe('plan'); @@ -233,17 +236,19 @@ describe('dispatch selection', () => { ); }); - it('claims the item BEFORE the pane spawns', async () => { + it('claims the item BEFORE the pane spawns with the tier CAS guard', async () => { const deps = makeDeps({ getNextItem: vi.fn().mockResolvedValue({ ok: true, - candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete' }, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open' }, }), }); await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); - expect(deps.claimItem).toHaveBeenCalledWith('WL-ABC'); + // The claim carries the expected state the tier selected the item in + // (RCA WL-0MSRBFFLN005W3VT design point 1 — compare-and-swap). + expect(deps.claimItem).toHaveBeenCalledWith('WL-ABC', { status: 'open', stage: 'intake_complete' }); const claimOrder = (deps.claimItem as ReturnType).mock.invocationCallOrder[0]; const spawnOrder = (deps.spawnAgentPane as ReturnType).mock.invocationCallOrder[0]; expect(claimOrder).toBeLessThan(spawnOrder); @@ -254,16 +259,16 @@ describe('dispatch selection', () => { getNextItem: vi .fn() .mockResolvedValueOnce({ ok: true, candidate: null }) - .mockResolvedValueOnce({ ok: true, candidate: { id: 'WL-DEF', title: 'An idea', stage: 'idea' } }), + .mockResolvedValueOnce({ ok: true, candidate: { id: 'WL-DEF', title: 'An idea', stage: 'idea', status: 'open' } }), }); const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); - expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete'); - expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete', '/repo'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea', '/repo'); expect(outcome.kind).toBe('intake'); expect(outcome.candidate?.id).toBe('WL-DEF'); - expect(deps.claimItem).toHaveBeenCalledWith('WL-DEF'); + expect(deps.claimItem).toHaveBeenCalledWith('WL-DEF', { status: 'open', stage: 'idea' }); expect(deps.spawnAgentPane).toHaveBeenCalledWith( expect.stringContaining('/skill:intake WL-DEF'), expect.anything(), @@ -275,8 +280,8 @@ describe('dispatch selection', () => { const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); - expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete'); - expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete', '/repo'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea', '/repo'); expect(outcome.dispatched).toBe(false); expect(outcome.reason).toBe('no-candidate'); expect(deps.claimItem).not.toHaveBeenCalled(); @@ -291,8 +296,8 @@ describe('dispatch selection', () => { // Tier 2 error does NOT short-circuit: tier 3 is still attempted. expect(deps.getNextItem).toHaveBeenCalledTimes(2); - expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete'); - expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete', '/repo'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea', '/repo'); expect(outcome.dispatched).toBe(false); expect(outcome.reason).toBe('wl-error'); expect(deps.claimItem).not.toHaveBeenCalled(); @@ -304,7 +309,7 @@ describe('dispatch selection', () => { getNextItem: vi .fn() .mockResolvedValueOnce({ ok: false }) - .mockResolvedValueOnce({ ok: true, candidate: { id: 'WL-IDE', title: 'An idea', stage: 'idea' } }), + .mockResolvedValueOnce({ ok: true, candidate: { id: 'WL-IDE', title: 'An idea', stage: 'idea', status: 'open' } }), }); const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); @@ -312,9 +317,9 @@ describe('dispatch selection', () => { expect(outcome.dispatched).toBe(true); expect(outcome.kind).toBe('intake'); expect(outcome.candidate?.id).toBe('WL-IDE'); - expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete'); - expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea'); - expect(deps.claimItem).toHaveBeenCalledWith('WL-IDE'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete', '/repo'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea', '/repo'); + expect(deps.claimItem).toHaveBeenCalledWith('WL-IDE', { status: 'open', stage: 'idea' }); expect(deps.spawnAgentPane).toHaveBeenCalledWith( expect.stringContaining('/skill:intake WL-IDE'), expect.anything(), @@ -349,8 +354,8 @@ describe('dispatch selection', () => { const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); - expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete'); - expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete', '/repo'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea', '/repo'); expect(outcome.dispatched).toBe(false); expect(outcome.reason).toBe('wl-error'); expect(deps.spawnAgentPane).not.toHaveBeenCalled(); @@ -377,7 +382,7 @@ describe('dispatch audit tier', () => { expect(outcome.kind).toBe('audit'); expect(outcome.candidate?.id).toBe('WL-AUD'); expect(deps.getNextItem).not.toHaveBeenCalled(); - expect(deps.claimItem).toHaveBeenCalledWith('WL-AUD'); + expect(deps.claimItem).toHaveBeenCalledWith('WL-AUD', { status: 'completed', stage: 'in_review' }); expect(deps.spawnAgentPane).toHaveBeenCalledWith( expect.stringContaining('/skill:audit WL-AUD'), { model: 'plan', cwd: '/repo' }, @@ -411,8 +416,8 @@ describe('dispatch audit tier', () => { expect(outcome.kind).toBe('intake'); expect(outcome.candidate?.id).toBe('WL-IDE'); expect(deps.getNextAuditCandidate).toHaveBeenCalledTimes(1); - expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete'); - expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete', '/repo'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea', '/repo'); }); it('fails closed to no-candidate when getNextAuditCandidate returns null and no plan/intake candidate exists', async () => { @@ -452,7 +457,7 @@ describe('dispatch implement tier', () => { expect(deps.getNextAuditCandidate).toHaveBeenCalledTimes(1); expect(deps.getNextImplementCandidate).toHaveBeenCalledTimes(1); expect(deps.getNextImplementCandidate).toHaveBeenCalledWith('/repo'); - expect(deps.claimItem).toHaveBeenCalledWith('WL-IMP'); + expect(deps.claimItem).toHaveBeenCalledWith('WL-IMP', { status: 'open', stage: 'plan_complete' }); expect(deps.spawnAgentPane).toHaveBeenCalledWith( expect.stringContaining('/skill:implement WL-IMP'), { model: 'plan', cwd: '/repo' }, @@ -518,7 +523,7 @@ describe('dispatch implement tier', () => { expect(outcome.kind).toBe('plan'); expect(deps.getNextImplementCandidate).toHaveBeenCalledTimes(1); expect(deps.getNextItem).toHaveBeenCalledTimes(1); - expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete', '/repo'); }); it('falls back to intake (idea) when no implement or plan candidate exists', async () => { @@ -534,8 +539,8 @@ describe('dispatch implement tier', () => { const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); expect(outcome.kind).toBe('intake'); - expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete'); - expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(1, 'intake_complete', '/repo'); + expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea', '/repo'); }); it('a wl next CLI error at the implement tier does NOT short-circuit the plan/intake fallback', async () => { @@ -599,10 +604,10 @@ describe('dispatch code-freeze gate', () => { expect(deps.getNextAuditCandidate).not.toHaveBeenCalled(); expect(deps.getNextImplementCandidate).not.toHaveBeenCalled(); // Plan/intake tiers still dispatch. - expect(deps.getNextItem).toHaveBeenCalledWith('intake_complete'); + expect(deps.getNextItem).toHaveBeenCalledWith('intake_complete', '/repo'); expect(outcome.dispatched).toBe(true); expect(outcome.kind).toBe('plan'); - expect(deps.claimItem).toHaveBeenCalledWith('WL-PLAN'); + expect(deps.claimItem).toHaveBeenCalledWith('WL-PLAN', { status: 'open', stage: 'intake_complete' }); expect(deps.spawnAgentPane).toHaveBeenCalledWith( expect.stringContaining('/skill:plan WL-PLAN'), expect.anything(), @@ -1185,18 +1190,272 @@ describe('dispatch audit trail', () => { expect(deps.recordDispatch).not.toHaveBeenCalled(); }); - it('keeps the dispatch successful when recordDispatch fails (audit never blocks work)', async () => { + it('aborts the dispatch (marker-write-failed) when recordDispatch fails — fail-closed', async () => { const deps = makeDeps({ getNextItem: vi.fn().mockResolvedValue({ ok: true, candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete' }, }), + // A rejecting marker write (or a stub that throws) must abort BEFORE + // the pane spawns: an unmarked item is never dispatched (RCA + // WL-0MSRBFFLN005W3VT design point 2 — marker-before-spawn fail-closed). recordDispatch: vi.fn().mockRejectedValue(new Error('audit boom')), }); const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); - expect(outcome.dispatched).toBe(true); + expect(outcome.dispatched).toBe(false); + expect(outcome.reason).toBe('marker-write-failed'); + expect(deps.spawnAgentPane).not.toHaveBeenCalled(); + }); + + it('aborts the dispatch (marker-write-failed) when recordDispatch resolves false', async () => { + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete' }, + }), + recordDispatch: vi.fn().mockResolvedValue(false), + }); + + const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + expect(outcome.dispatched).toBe(false); + expect(outcome.reason).toBe('marker-write-failed'); + expect(deps.spawnAgentPane).not.toHaveBeenCalled(); + }); + + it('writes the marker BEFORE the pane spawns (marker-before-spawn ordering)', async () => { + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete' }, + }), + }); + + await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + const markerOrder = (deps.recordDispatch as ReturnType).mock.invocationCallOrder[0]; + const spawnOrder = (deps.spawnAgentPane as ReturnType).mock.invocationCallOrder[0]; + expect(markerOrder).toBeLessThan(spawnOrder); + }); + + it('records the dispatched-at stage on plan/intake markers (change-guard input)', async () => { + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete' }, + }), + }); + + await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + const event = (deps.recordDispatch as ReturnType).mock.calls[0][0]; + expect(event.stage).toBe('intake_complete'); + }); +}); + +// ── CAS claim race + fail-closed paths (RCA WL-0MSRBFFLN005W3VT) ─────── + +describe('dispatch CAS claim race', () => { + it('a losing pane (stale claim) aborts with no pane, no marker, no success record', async () => { + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open' }, + }), + // Another pane claimed the item first: the CAS guard fails stale. + claimItem: vi.fn().mockResolvedValue({ ok: false, reason: 'stale' }), + }); + + const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + expect(outcome.dispatched).toBe(false); + expect(outcome.reason).toBe('claim-failed'); + // No pane, no marker, no success record — the loser aborts entirely. + expect(deps.spawnAgentPane).not.toHaveBeenCalled(); + expect(deps.recordDispatch).not.toHaveBeenCalled(); + }); + + it('a claim wl failure is a wl-error (a strike, never silently discarded)', async () => { + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open' }, + }), + claimItem: vi.fn().mockResolvedValue({ ok: false, reason: 'error' }), + }); + + const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + expect(outcome.dispatched).toBe(false); + expect(outcome.reason).toBe('wl-error'); + expect(deps.spawnAgentPane).not.toHaveBeenCalled(); + expect(deps.recordDispatch).not.toHaveBeenCalled(); + }); + + it('a stale audit-tier claim also aborts (no fall-through to other tiers)', async () => { + const deps = makeDeps({ + getNextAuditCandidate: vi.fn().mockResolvedValue({ + id: 'WL-AUD', + title: 'Audit me', + stage: 'audit', + }), + claimItem: vi.fn().mockResolvedValue({ ok: false, reason: 'stale' }), + }); + + const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + expect(outcome.dispatched).toBe(false); + expect(outcome.reason).toBe('claim-failed'); + // The loser does NOT fall through to plan/intake — exactly one dispatch + // per idle window across panes. + expect(deps.getNextItem).not.toHaveBeenCalled(); + expect(deps.spawnAgentPane).not.toHaveBeenCalled(); + }); + + it('a spawn error makes the outcome not-success (spawn-failed) with the marker already written', async () => { + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open' }, + }), + spawnAgentPane: vi.fn().mockResolvedValue(false), + }); + + const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + expect(outcome.dispatched).toBe(false); + expect(outcome.reason).toBe('spawn-failed'); + // The marker stands (fail-closed — no re-dispatch of the same item), but + // the outcome is NOT a success (WL-0MSLWJ3I70031Z8U absorbed). + expect(deps.recordDispatch).toHaveBeenCalledTimes(1); + }); + + it('claim → marker → spawn ordering is fixed (claim first, marker before spawn)', async () => { + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open' }, + }), + }); + + await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + const claimOrder = (deps.claimItem as ReturnType).mock.invocationCallOrder[0]; + const markerOrder = (deps.recordDispatch as ReturnType).mock.invocationCallOrder[0]; + const spawnOrder = (deps.spawnAgentPane as ReturnType).mock.invocationCallOrder[0]; + expect(claimOrder).toBeLessThan(markerOrder); + expect(markerOrder).toBeLessThan(spawnOrder); + }); +}); + +// ── Plan/intake selection (RCA WL-0MSRBFFLN005W3VT RC-2 + amplifier) ── + +describe('plan/intake selection (selectNextCandidate)', () => { + const openCandidate = (id: string, sortIndex = 0, stage: DowntimeStage = 'intake_complete') => ({ + id, + title: id, + stage, + status: 'open', + sortIndex, + }); + + it('selects the first open candidate in wl next priority order (ascending sortIndex)', () => { + const selected = selectNextCandidate([ + openCandidate('WL-2', 20), + openCandidate('WL-1', 10), + ]); + expect(selected?.id).toBe('WL-1'); + }); + + it('excludes a candidate still at its dispatched-at stage (change-guard)', () => { + const selected = selectNextCandidate( + [openCandidate('WL-ONCE', 5), openCandidate('WL-NEXT', 20)], + new Map([['WL-ONCE', 'intake_complete']]), + ); + // WL-ONCE was dispatched for /skill:plan at intake_complete and is still + // at intake_complete → excluded; the next candidate is selected. + expect(selected?.id).toBe('WL-NEXT'); + }); + + it('releases a candidate whose stage advanced past its dispatched-at stage', () => { + // The marker recorded intake_complete but the candidate is now at idea + // (a different stage) → not suppressed. + const selected = selectNextCandidate( + [openCandidate('WL-FREE', 5, 'idea')], + new Map([['WL-FREE', 'intake_complete']]), + ); + expect(selected?.id).toBe('WL-FREE'); + }); + + it('a legacy marker without a recorded stage never suppresses selection', () => { + const selected = selectNextCandidate( + [openCandidate('WL-LEGACY', 5)], + new Map([['WL-LEGACY', '']]), + ); + expect(selected?.id).toBe('WL-LEGACY'); + }); + + it('excludes completed candidates (client-side open-status guard, amplifier fix)', () => { + const selected = selectNextCandidate([ + { ...openCandidate('WL-DONE', 5), status: 'completed' }, + openCandidate('WL-OPEN', 10), + ]); + expect(selected?.id).toBe('WL-OPEN'); + }); + + it('a candidate without a verifiable status is never selected (fail-closed)', () => { + const selected = selectNextCandidate([ + { id: 'WL-NO-STATUS', title: 'x', stage: 'intake_complete' }, + ]); + expect(selected).toBeNull(); + }); + + it('returns null on an empty list', () => { + expect(selectNextCandidate([])).toBeNull(); + }); +}); + +describe('parseNextCandidatesOutput', () => { + it('parses the batch shape (wl next -n N) preserving status and sortIndex', () => { + const candidates = parseNextCandidatesOutput( + JSON.stringify({ + success: true, + workItems: [ + { workItem: { id: 'WL-1', title: 'One', status: 'open', sortIndex: 7 } }, + { workItem: { id: 'WL-2', title: 'Two', status: 'completed', sortIndex: 3 } }, + ], + }), + 'idea', + ); + expect(candidates).toEqual([ + { id: 'WL-1', title: 'One', stage: 'idea', status: 'open', sortIndex: 7 }, + { id: 'WL-2', title: 'Two', stage: 'idea', status: 'completed', sortIndex: 3 }, + ]); + }); + + it('parses the legacy single-item shape', () => { + const candidates = parseNextCandidatesOutput( + JSON.stringify({ success: true, workItem: { id: 'WL-1', title: 'One', status: 'open' } }), + 'intake_complete', + ); + expect(candidates).toEqual([ + { id: 'WL-1', title: 'One', stage: 'intake_complete', status: 'open', sortIndex: undefined }, + ]); + }); + + it('treats workItem:null as an empty backlog (not a parse failure)', () => { + const candidates = parseNextCandidatesOutput( + JSON.stringify({ success: false, workItem: null, reason: 'none' }), + 'idea', + ); + expect(candidates).toEqual([]); + }); + + it('returns null on malformed JSON (fail-closed)', () => { + expect(parseNextCandidatesOutput('not json', 'idea')).toBeNull(); + expect(parseNextCandidatesOutput(JSON.stringify({}), 'idea')).toBeNull(); }); }); @@ -1220,8 +1479,8 @@ describe('buildDowntimeDispatchComment', () => { describe('single-flight dispatch guard', () => { it('does not dispatch a second time while one dispatch is in flight', async () => { let release!: () => void; - const gate = new Promise((resolve) => { - release = resolve; + const gate = new Promise((resolve) => { + release = () => resolve(true); }); const deps = makeDeps({ getNextItem: vi.fn().mockResolvedValue({ @@ -1604,16 +1863,32 @@ describe('downtime pane spawn (send-to-pi.sh)', () => { expect(args).not.toContain('Downtime plan'); }); - it('spawns via the injectable spawn and unrefs the child', () => { - const spawnFn = vi.fn(() => ({ unref: vi.fn() })) as unknown as DowntimeSpawn; + it('spawns via the injectable spawn and unrefs the child', async () => { + const spawnFn = vi.fn(() => ({ unref: vi.fn(), once: vi.fn() })) as unknown as DowntimeSpawn; - spawnDowntimePane('/path/to/send-to-pi.sh', ['--no-focus', 'prompt'], { cwd: '/repo' }, spawnFn); + const spawned = await spawnDowntimePane('/path/to/send-to-pi.sh', ['--no-focus', 'prompt'], { cwd: '/repo' }, spawnFn); expect(spawnFn).toHaveBeenCalledWith('/path/to/send-to-pi.sh', ['--no-focus', 'prompt'], { cwd: '/repo', }); const handle = (spawnFn as ReturnType).mock.results[0].value; expect(handle.unref).toHaveBeenCalled(); + expect(spawned).toBe(true); + }); + + it('resolves false on a handled spawn error (no unhandled-exception crash)', async () => { + const handle = { + unref: vi.fn(), + once: vi.fn((_event: string, listener: (err: Error) => void) => { + listener(new Error('ENOENT: send-to-pi.sh')); + }), + }; + const spawnFn = vi.fn(() => handle) as unknown as DowntimeSpawn; + + const spawned = await spawnDowntimePane('/path/to/missing.sh', [], { cwd: '/repo' }, spawnFn); + + expect(spawned).toBe(false); + expect(handle.once).toHaveBeenCalledWith('error', expect.any(Function)); }); it('buildDowntimeSpawnOptions uses detached/ignore options with the resolved cwd', () => { @@ -1789,8 +2064,8 @@ describe('downtime worker orchestrator (createDowntimeWorker)', () => { it('does not start a second dispatch while one is in flight (worker single-flight)', async () => { let release!: () => void; - const gate = new Promise((resolve) => { - release = resolve; + const gate = new Promise((resolve) => { + release = () => resolve(true); }); const { worker, deps } = makeWorker({ deps: { spawnAgentPane: vi.fn().mockImplementation(() => gate) }, @@ -1943,8 +2218,8 @@ describe('downtime no-candidate cooldown (createDowntimeWorker)', () => { // in-flight dispatch; the worker's own dispatch call then returns // `dispatch-in-flight`, which must NOT trigger the cooldown. let release!: () => void; - const gate = new Promise((resolve) => { - release = resolve; + const gate = new Promise((resolve) => { + release = () => resolve(true); }); const outerDeps = makeDeps({ getNextItem: vi.fn().mockResolvedValue({ @@ -1961,7 +2236,6 @@ describe('downtime no-candidate cooldown (createDowntimeWorker)', () => { await worker.tick(); vi.setSystemTime(start + 240_000); const result = await worker.tick(); - expect(result.dispatched).toBe(false); expect(worker.paused).toBe(false); // in-flight guard is not an empty backlog diff --git a/packages/herdr/src/downtime-worker.ts b/packages/herdr/src/downtime-worker.ts index 29d548af9..f9082722c 100644 --- a/packages/herdr/src/downtime-worker.ts +++ b/packages/herdr/src/downtime-worker.ts @@ -407,6 +407,17 @@ export interface DowntimeCandidate { id: string; title: string; stage: DowntimeStage; + /** + * Worklog status of the candidate (`open` for selectable plan/intake/ + * implement items; `completed` for audit candidates). The plan/intake tiers + * filter client-side to `status === 'open'` (RCA WL-0MSRBFFLN005W3VT + * amplifier fix): `wl next --stage X` keeps completed items under a stage + * filter, so without this guard a completed/in_review item whose stage + * matches could be dispatched for /skill:plan or /skill:intake. + */ + status?: string; + /** wl next priority order preserved for deterministic selection. */ + sortIndex?: number; } /** @@ -452,10 +463,41 @@ export type DowntimeNextResult = | { ok: true; candidate: DowntimeCandidate | null } | { ok: false }; +/** + * Expected state for the pre-dispatch claim (compare-and-swap, RCA + * WL-0MSRBFFLN005W3VT design point 1): the claim transition to + * `in_progress` only applies while the item is still in the state the tier + * selected it in. `status` matches the stored hyphenated status + * (`open`/`completed`/...); `stage` matches the worklog stage. + */ +export interface DowntimeClaimExpected { + status?: string; + stage?: string; +} + +/** + * Outcome of a pre-dispatch claim. `ok:true` means THIS pane won the + * claim; `ok:false` with reason `stale` means another pane claimed the item + * first (or its state changed since selection) — the dispatch MUST abort; + * `ok:false` with reason `error` is a wl CLI failure (counted as a + * wl-error strike). + */ +export type DowntimeClaimResult = + | { ok: true } + | { ok: false; reason: 'stale' | 'error' }; + /** External boundaries injected so the dispatch logic is testable. */ export interface DowntimeWorkerDeps { - /** Runs `wl next --stage --json` and reports the first candidate (or a wl failure). */ - getNextItem(stage: DowntimeStage): Promise; + /** + * Runs `wl next --stage -n 10 --json` and reports the first + * selectable candidate (or a wl failure). `cwd` is the worklog root whose + * `.worklog/downtime-dispatches.log` is consulted for the plan/intake + * dispatched-marker change-guard (RCA WL-0MSRBFFLN005W3VT design point 3): + * a candidate already dispatched for its tier while still at its + * dispatched-at stage is excluded. A batch is fetched so excluding the + * top candidate does not starve selection of the next one. + */ + getNextItem(stage: DowntimeStage, cwd: string): Promise; /** * Look up the next completed/in_review item WITHOUT a valid audit (the * audit dispatch tier, which runs before the plan/intake tiers). Fail-closed: @@ -487,15 +529,30 @@ export interface DowntimeWorkerDeps { * trusted. */ readCodeFreezeStatus(cwd: string): CodeFreezeStatus; - /** Claims the item (`wl update --status in_progress`) before dispatch. */ - claimItem(itemId: string): Promise; - /** Opens a visible pi agent pane running the prompt (via send-to-pi.sh). */ - spawnAgentPane(prompt: string, opts: { model: string; cwd: string }): Promise; + /** + * Claim the item (CAS: `wl update --status in_progress --if-status + * [--if-stage ]`) BEFORE anything else. + * Exactly one concurrent pane wins; a loser resolves `{ok:false, + * reason:'stale'}` and the dispatcher aborts (no pane, no marker, no + * success record). A `{ok:false, reason:'error'}` is a wl CLI failure + * (a strike, never silently discarded — WL-0MSLWJ310000ND0X absorbed). + */ + claimItem(itemId: string, expected: DowntimeClaimExpected): Promise; + /** + * Open a visible pi agent pane running the prompt (via send-to-pi.sh). + * Resolves true when the spawn started; false on a handled spawn `error` + * event (no unhandled-exception crash, WL-0MSLWJ3I70031Z8U absorbed). + */ + spawnAgentPane(prompt: string, opts: { model: string; cwd: string }): Promise; /** * Audit trail for a successful dispatch: comment on the item + rolling - * log entry under `.worklog`. Must never throw (fail-closed). + * log entry under `.worklog`. Resolves TRUE only when the rolling-log + * MARKER was written (the dispatched-marker source); a comment failure is + * tolerated (the comment is a durable cross-machine trail, not the + * marker). A false result makes the dispatcher ABORT before spawning — an + * unmarked item is never dispatched (fail-closed, RCA design point 2). */ - recordDispatch(event: DowntimeDispatchEvent): Promise; + recordDispatch(event: DowntimeDispatchEvent): Promise; /** * Record a persistent CLI-error event (three consecutive wl failures). * Must never throw (fail-closed): logging must not crash the worker. @@ -512,6 +569,13 @@ export interface DowntimeDispatchEvent { /** Worklog root (the rolling log lives at `/.worklog`). */ cwd: string; title?: string; + /** + * Worklog stage of the item at dispatch (plan/intake change-guard, RCA + * WL-0MSRBFFLN005W3VT design point 3): a candidate is excluded while it is + * still at its dispatched-at stage; a stage advancement releases it. + * Backward compatible — absent on legacy entries. + */ + stage?: string; } /** @@ -531,6 +595,13 @@ export interface DowntimeDispatchOutcome { dispatched: boolean; candidate?: DowntimeCandidate; kind?: DowntimeSkillKind; + /** + * Non-dispatch reasons: 'dispatch-in-flight' | 'no-candidate' | + * 'wl-error' | 'code-freeze' | 'claim-failed' (lost the CAS race — + * another pane won; neutral) | 'marker-write-failed' (fail-closed abort + * BEFORE spawn) | 'spawn-failed' (handled spawn error; outcome is not + * success). + */ reason?: string; } @@ -538,10 +609,85 @@ export interface DowntimeDispatchOutcome { * Per-process single-flight guard: at most one dispatch can be in flight at * a time (concurrent calls are refused, not queued). Cross-pane * serialization is handled by the pre-dispatch claim (Q5 — no lock file): - * the claimed item leaves `wl next`'s selection set for other panes. + * the CAS claim atomically moves the item out of `wl next`'s selection set + * for other panes. */ let dispatchInFlight = false; +/** + * Expected claim state per tier (RCA WL-0MSRBFFLN005W3VT design point 1): + * the pre-dispatch claim only applies while the item is still in the exact + * state the tier selected it in. The audit tier selects `completed` / + * `in_review` items; every other tier selects `open` items at their stage. + */ +const TIER_EXPECTED: Record = { + audit: { status: 'completed', stage: 'in_review' }, + implement: { status: 'open', stage: 'plan_complete' }, + plan: { status: 'open', stage: 'intake_complete' }, + intake: { status: 'open', stage: 'idea' }, +}; + +/** + * Dispatch one already-selected candidate through the fixed pipeline: + * CAS claim → marker write (before spawn) → spawn. + * + * - Claim (compare-and-swap): exactly one concurrent pane wins; a loser + * (or a wl claim failure) ABORTS the dispatch — no pane, no marker, no + * success record. A lost race resolves reason 'claim-failed' (neutral, + * the winner's pane will busy the proxy); a wl claim failure resolves + * 'wl-error' (counts toward the three-strike rule). Neither is ever + * silently discarded (WL-0MSLWJ310000ND0X absorbed). + * - Marker write BEFORE the pane spawns: if the marker cannot be written + * the dispatch aborts BEFORE spawning (fail-closed — an unmarked item is + * never dispatched). The item stays claimed (in_progress), so no other + * pane can select it (RCA design point 2). + * - Spawn: a handled spawn `error` resolves reason 'spawn-failed' — the + * outcome is NOT success (WL-0MSLWJ3I70031Z8U absorbed); the marker + * stands so the item is not re-dispatched. + */ +async function dispatchClaimedTier( + deps: DowntimeWorkerDeps, + kind: DowntimeSkillKind, + candidate: DowntimeCandidate, + opts: { model: string; cwd: string }, +): Promise { + const expected = TIER_EXPECTED[kind]; + const claim = await deps.claimItem(candidate.id, expected); + if (!claim.ok) { + return claim.reason === 'stale' + ? { dispatched: false, reason: 'claim-failed' } + : { dispatched: false, reason: 'wl-error' }; + } + + let marked = false; + try { + marked = await deps.recordDispatch({ + itemId: candidate.id, + kind, + dispatchedAt: new Date().toISOString(), + cwd: opts.cwd, + title: candidate.title, + // The worklog stage at dispatch powers the plan/intake change-guard + // (exclude while the item is still at this stage; release on advancement). + stage: expected.stage, + }); + } catch { + // Fail-closed: a throwing recordDispatch (stub or regression) is a marker + // write failure — abort before spawn. + marked = false; + } + if (!marked) { + return { dispatched: false, reason: 'marker-write-failed' }; + } + + const spawned = await deps.spawnAgentPane(buildDowntimePrompt(kind, candidate), opts); + if (!spawned) { + return { dispatched: false, reason: 'spawn-failed' }; + } + + return { dispatched: true, candidate, kind }; +} + /** * dispatch one downtime work item. Selection priority (audit tier first, * WL-0MSI8H3HP000K0RG): a completed/in_review item WITHOUT a valid audit AND @@ -569,8 +715,10 @@ let dispatchInFlight = false; * tier is still attempted, so a tier-3 candidate can still dispatch. A * `wl-error` outcome (any CLI failure) is never a candidate and never a * `no-candidate`; the caller's three-strike rule decides when consecutive - * errors pause the worker. The item is claimed BEFORE the pane spawns so it - * appears in_progress immediately and a second pane cannot select it. The + * errors pause the worker. Every tier runs its candidate through + * `dispatchClaimedTier` (CAS claim → marker → spawn), so the item is + * claimed BEFORE the pane spawns and a losing pane aborts with no pane, no + * marker, no success record (the same-instant race, RC-1, is closed). The * caller must only invoke this once the idle tracker reports the threshold * met (AC1); a dispatch consumes the local slot, so the proxy reports busy * and the tracker requires a fresh full idle period before the next dispatch. @@ -595,10 +743,7 @@ export async function dispatchDowntimeWork( if (!frozen) { const auditCandidate = await deps.getNextAuditCandidate(opts.cwd); if (auditCandidate !== null) { - await deps.claimItem(auditCandidate.id); - await deps.spawnAgentPane(buildDowntimePrompt('audit', auditCandidate), opts); - await recordDispatchAudit(deps, 'audit', auditCandidate, opts.cwd); - return { dispatched: true, candidate: auditCandidate, kind: 'audit' }; + return await dispatchClaimedTier(deps, 'audit', auditCandidate, opts); } // Implement tier (WL-0MSMAYPQP001FLR6): after the audit gate, dispatch // /skill:implement for the highest-priority open plan_complete item with @@ -608,23 +753,17 @@ export async function dispatchDowntimeWork( // error at the implement tier does NOT short-circuit the fallback). const implementCandidate = await deps.getNextImplementCandidate(opts.cwd); if (implementCandidate !== null) { - await deps.claimItem(implementCandidate.id); - await deps.spawnAgentPane(buildDowntimePrompt('implement', implementCandidate), opts); - await recordDispatchAudit(deps, 'implement', implementCandidate, opts.cwd); - return { dispatched: true, candidate: implementCandidate, kind: 'implement' }; + return await dispatchClaimedTier(deps, 'implement', implementCandidate, opts); } } // Tier 2 (intake_complete → /skill:plan). A CLI error here does NOT // short-circuit: tier 3 is still attempted so a tier-3 candidate can // still dispatch (operator refinement). let tier2Error = false; - const intakeComplete = await deps.getNextItem('intake_complete'); + const intakeComplete = await deps.getNextItem('intake_complete', opts.cwd); if (intakeComplete.ok) { if (intakeComplete.candidate !== null) { - await deps.claimItem(intakeComplete.candidate.id); - await deps.spawnAgentPane(buildDowntimePrompt('plan', intakeComplete.candidate), opts); - await recordDispatchAudit(deps, 'plan', intakeComplete.candidate, opts.cwd); - return { dispatched: true, candidate: intakeComplete.candidate, kind: 'plan' }; + return await dispatchClaimedTier(deps, 'plan', intakeComplete.candidate, opts); } } else { tier2Error = true; @@ -632,13 +771,10 @@ export async function dispatchDowntimeWork( // Tier 3 (idea → /skill:intake) is ALWAYS attempted when tier 2 produced // no candidate — including when tier 2 errored. - const idea = await deps.getNextItem('idea'); + const idea = await deps.getNextItem('idea', opts.cwd); if (idea.ok) { if (idea.candidate !== null) { - await deps.claimItem(idea.candidate.id); - await deps.spawnAgentPane(buildDowntimePrompt('intake', idea.candidate), opts); - await recordDispatchAudit(deps, 'intake', idea.candidate, opts.cwd); - return { dispatched: true, candidate: idea.candidate, kind: 'intake' }; + return await dispatchClaimedTier(deps, 'intake', idea.candidate, opts); } if (tier2Error) { // Tier 2 errored and tier 3 answered empty: the backlog is NOT @@ -667,30 +803,6 @@ export async function dispatchDowntimeWork( } } -/** - * Record the audit event for a successful dispatch. Wrapped so a failing - * audit implementation (or a stub that rejects) can never fail or block the - * dispatch itself — the audit trail is best-effort (AC4). - */ -async function recordDispatchAudit( - deps: DowntimeWorkerDeps, - kind: DowntimeSkillKind, - candidate: DowntimeCandidate, - cwd: string, -): Promise { - try { - await deps.recordDispatch({ - itemId: candidate.id, - kind, - dispatchedAt: new Date().toISOString(), - cwd, - title: candidate.title, - }); - } catch { - // fail-closed: audit logging must never crash or block the worker - } -} - /** * Argument vector for spawning `send-to-pi.sh`: visible pane named * `Downtime `, `--no-focus` (visible but never steals focus), @@ -713,9 +825,14 @@ export function buildDowntimePaneArgs( ]; } -/** Minimal spawn handle: the caller unrefs so the parent can exit first. */ +/** + * Minimal spawn handle: the caller unrefs so the parent can exit first, and + * registers an `error` listener so a spawn failure is observable instead of + * crashing the plugin process (WL-0MSLWJ3I70031Z8U absorbed). + */ export interface DowntimeSpawnHandle { unref(): void; + once(event: 'error', listener: (err: Error) => void): void; } /** Injectable spawn boundary (matches the repo's injectable-seam pattern). */ @@ -747,19 +864,39 @@ export function buildDowntimeSpawnOptions(cwd: string): { export const defaultDowntimeSpawn: DowntimeSpawn = (scriptPath, args, opts) => spawn(scriptPath, args, buildDowntimeSpawnOptions(opts.cwd)); +/** + * How long to wait for a spawn-level `error` event before assuming the pane + * started (an ENOENT/EACCES error fires on the next event-loop tick). The + * probe keeps the failure observable without delaying the poll loop. + */ +export const DOWNTIME_SPAWN_PROBE_MS = 500; + /** * Spawn `send-to-pi.sh` detached with stdio ignored, then unref so the * parent (plugin) process can exit independently — same pattern as the - * TUI's existing agent dispatches. + * TUI's existing agent dispatches. An `error` listener is ALWAYS attached + * (a spawn failure must not crash the plugin with an unhandled 'error' + * event, WL-0MSLWJ3I70031Z8U); a spawn error resolves `false` so the + * dispatch outcome is not a false success. */ -export function spawnDowntimePane( +export async function spawnDowntimePane( scriptPath: string, args: string[], opts: { cwd: string }, spawnFn: DowntimeSpawn = defaultDowntimeSpawn, -): void { +): Promise { const child = spawnFn(scriptPath, args, opts); child.unref(); + return new Promise((resolve) => { + const timer = setTimeout(() => resolve(true), DOWNTIME_SPAWN_PROBE_MS); + child.once('error', (err: Error) => { + clearTimeout(timer); + process.stderr.write( + `[worklog-plugin] Downtime pane spawn failed: ${err.message}\n`, + ); + resolve(false); + }); + }); } // ── Worker orchestrator (implemented — F3) ──────────────────────────── @@ -984,10 +1121,72 @@ export function buildDowntimeDispatchComment( // ── Wiring helpers (implemented — F4) ───────────────────────────────── +/** + * Parse the stdout of `wl next --stage -n N --json` into typed + * plan/intake candidates. Accepts both the batch shape the CLI emits for + * `-n N` (`{ workItems: [{ workItem: {...} }] }`) and the legacy single-item + * shape (`{ workItem: {...} }`) for backward compatibility. Each candidate + * carries the query `stage`, the worklog `status` (for the client-side + * `open` guard) and `sortIndex` (wl next priority order). Malformed JSON or + * output without any parseable entry yields null (fail-closed); an empty + * result list yields `[]` (a genuine empty backlog). + */ +export function parseNextCandidatesOutput( + stdout: string, + stage: DowntimeStage, +): DowntimeCandidate[] | null { + let parsed: unknown; + try { + parsed = JSON.parse(stdout); + } catch { + return null; + } + + let rawItems: unknown[] | null = null; + if (parsed !== null && typeof parsed === 'object') { + const o = parsed as Record; + if (Array.isArray(o.workItems)) { + rawItems = o.workItems; + } else if (typeof o.workItem === 'object' && o.workItem !== null) { + rawItems = [o]; // legacy single-item shape with a candidate + } else if ('workItem' in o) { + rawItems = []; // legacy single-item shape answered no item → empty backlog + } + } + if (rawItems === null) return null; + + const candidates: DowntimeCandidate[] = []; + for (const entry of rawItems) { + if (typeof entry !== 'object' || entry === null) continue; + const nested = + (entry as { workItem?: unknown }).workItem !== undefined + ? (entry as { workItem: Record }).workItem + : (entry as Record); + if (typeof nested !== 'object' || nested === null) continue; + if (typeof nested.id !== 'string' || nested.id.length === 0) continue; + candidates.push({ + id: nested.id, + title: typeof nested.title === 'string' ? nested.title : '', + stage, + status: typeof nested.status === 'string' ? nested.status : undefined, + sortIndex: + typeof nested.sortIndex === 'number' && Number.isFinite(nested.sortIndex) + ? nested.sortIndex + : undefined, + }); + } + return candidates; +} + /** * Parse the stdout of `wl next --stage --json` into the first * candidate. Returns null (no dispatch) for empty output, a null * `workItem`, missing ids, or malformed JSON. + * + * Superseded by `parseNextCandidatesOutput` for production selection (the + * downtime worker fetches a batch so marker-excluded candidates do not + * starve selection); kept exported for callers/tests that use the + * single-item shape. */ export function parseNextItemOutput(stdout: string, stage: DowntimeStage): DowntimeCandidate | null { try { @@ -1233,6 +1432,39 @@ export function toImplementCandidate(candidate: ImplementCandidate): DowntimeCan return { id: candidate.id, title: candidate.title, stage: 'implement' }; } +// ── Plan/intake selection (RCA WL-0MSRBFFLN005W3VT RC-2 + amplifier) ── + +/** + * Select the next plan/intake candidate from parsed `wl next` batch output: + * the first candidate that is `status === 'open'` (client-side guard — `wl + * next --stage X` keeps completed items under a stage filter, so without it + * a completed/in_review item whose stage matches could be dispatched), + * excluding candidates in the dispatched-marker change-guard map (id → + * stage at dispatch): a candidate is excluded while it is still at its + * dispatched-at stage; a stage advancement (or any stage differing from the + * recorded one) releases it. Sorted ascending by `sortIndex` (wl next + * priority order preserved). Returns null when no candidate qualifies. + * + * Missing `status` never qualifies (fail-closed: an item whose state cannot + * be verified is not dispatched) — `parseNextCandidatesOutput` always + * carries the status from the enriched workItem. + */ +export function selectNextCandidate( + candidates: DowntimeCandidate[], + dispatchedStages?: ReadonlyMap, +): DowntimeCandidate | null { + const target = candidates + .filter((c) => c.status === 'open') + .filter((c) => { + const dispatchedAt = dispatchedStages?.get(c.id); + // Exclude while still at the dispatched-at stage; a missing recorded + // stage (legacy entry) never suppresses selection. + return dispatchedAt === undefined || dispatchedAt !== c.stage; + }) + .sort((a, b) => (a.sortIndex ?? 0) - (b.sortIndex ?? 0)); + return target[0] ?? null; +} + // ── Settings clamps (implemented — wired into settings.ts by F2) ────── /** diff --git a/packages/herdr/src/fetcher.ts b/packages/herdr/src/fetcher.ts index 336d87f56..9a0e16de6 100644 --- a/packages/herdr/src/fetcher.ts +++ b/packages/herdr/src/fetcher.ts @@ -602,6 +602,12 @@ export async function fetchChildrenForItem(parentId: string, depth = 1): Promise */ export interface ClaimResult { success: boolean; + /** + * True when the claim failed because the item no longer matched the + * `--if-status`/`--if-stage` guard (another pane won the CAS race). The + * wl CLI answered correctly — this is NOT a CLI error. + */ + stale?: boolean; error?: string; } @@ -617,18 +623,41 @@ const CLAIM_TIMEOUT_MS = 3000; * documented in AGENTS.md. Targets the configured worklog database via * `--worklog-dir` when set. * + * With `expected` (status/stage), the claim is a compare-and-swap (RCA + * WL-0MSRBFFLN005W3VT design point 1): `--if-status`/`--if-stage` are + * passed to `wl update`, so the transition only applies while the item is + * still in the state the tier selected it in. Exactly one concurrent pane + * wins; a loser gets `{success:false, stale:true}` and the caller aborts + * the dispatch. A stale result is detected from the wl CLI's `"error": + * "stale"` JSON payload on stderr. + * * Never throws — failures are returned so callers can log them without * blocking the agent pane from opening. */ -export async function claimWorkItem(id: string, assignee: string): Promise { +export async function claimWorkItem( + id: string, + assignee: string, + expected?: { status?: string; stage?: string }, +): Promise { try { - await runWl( - ['update', id, '--status', 'in_progress', '--assignee', assignee], - true, - CLAIM_TIMEOUT_MS, - ); + const args = ['update', id, '--status', 'in_progress', '--assignee', assignee]; + if (expected?.status) { + args.push('--if-status', expected.status); + } + if (expected?.stage) { + args.push('--if-stage', expected.stage); + } + await runWl(args, true, CLAIM_TIMEOUT_MS); return { success: true }; } catch (err: any) { - return { success: false, error: err.message ?? String(err) }; + const message = err.message ?? String(err); + // Lost CAS race: the wl CLI exits non-zero with `{"success":false, + // "error":"stale", ...}` on stderr (runWl surfaces stderr first). + const stale = + message.includes('"error":"stale"') || + message.includes('"error": "stale"') || + message.includes("'stale'") || + message.includes('Conditional update skipped'); + return { success: false, stale, error: message }; } } diff --git a/packages/herdr/src/index.test.ts b/packages/herdr/src/index.test.ts index 305f288a8..2a20bf337 100644 --- a/packages/herdr/src/index.test.ts +++ b/packages/herdr/src/index.test.ts @@ -16,7 +16,7 @@ import { parsePaneIdFile, CAPTURE_TIMEOUT_MS, } from './index.js'; -import { DOWNTIME_LOG_FILE, readDowntimeLogEntries } from './downtime-log.js'; +import { appendDowntimeLogEntry, DOWNTIME_LOG_FILE, readDowntimeLogEntries } from './downtime-log.js'; import { DOWNTIME_WL_TIMEOUT_MS, dispatchDowntimeWork } from './downtime-worker.js'; import { fetchItemsByStage, @@ -516,37 +516,91 @@ describe('createDowntimeDeps', () => { resetWorklogDir(); }); - it('getNextItem runs wl next --stage and parses the first workItem', async () => { + it('getNextItem runs wl next --stage -n 10 and parses the first workItem', async () => { const mockExec = vi.fn().mockResolvedValue({ - stdout: JSON.stringify({ success: true, workItem: { id: 'WL-ABC', title: 'Some task' } }), + stdout: JSON.stringify({ success: true, workItem: { id: 'WL-ABC', title: 'Some task', status: 'open' } }), stderr: '', }); setExecFileAsync(mockExec as never); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - const result = await deps.getNextItem('intake_complete'); + const result = await deps.getNextItem('intake_complete', '/repo'); expect(mockExec).toHaveBeenCalledWith( 'wl', - ['next', '--stage', 'intake_complete', '--json'], + ['next', '--stage', 'intake_complete', '-n', '10', '--json'], expect.anything(), ); expect(result).toEqual({ ok: true, - candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete' }, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open', sortIndex: undefined }, + }); + }); + + it('getNextItem parses the batch shape (wl next -n N) and preserves priority order', async () => { + const mockExec = vi.fn().mockResolvedValue({ + stdout: JSON.stringify({ + success: true, + workItems: [ + { workItem: { id: 'WL-1', title: 'Second', status: 'open', sortIndex: 20 } }, + { workItem: { id: 'WL-2', title: 'First', status: 'open', sortIndex: 10 } }, + ], + }), + stderr: '', + }); + setExecFileAsync(mockExec as never); + + const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); + const result = await deps.getNextItem('idea', '/repo'); + + // selectNextCandidate: open-status guard + sortIndex ascending (wl next + // priority order preserved). + expect(result).toEqual({ + ok: true, + candidate: { id: 'WL-2', title: 'First', stage: 'idea', status: 'open', sortIndex: 10 }, + }); + }); + + it('getNextItem excludes a plan-dispatched candidate still at its dispatched stage (change-guard)', async () => { + const mockExec = vi.fn().mockResolvedValue({ + stdout: JSON.stringify({ + success: true, + workItems: [ + { workItem: { id: 'WL-ONCE', title: 'Marked', status: 'open', sortIndex: 5 } }, + { workItem: { id: 'WL-NEXT', title: 'Next', status: 'open', sortIndex: 20 } }, + ], + }), + stderr: '', + }); + setExecFileAsync(mockExec as never); + const cwd = makeTempDir(); + // Pre-write a plan marker for WL-ONCE at stage intake_complete (the same + // stage the plan tier selects at). + await appendDowntimeLogEntry( + cwd, + JSON.stringify({ itemId: 'WL-ONCE', kind: 'plan', stage: 'intake_complete', dispatchedAt: '2026-01-01T00:00:00.000Z' }), + ); + + const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); + const result = await deps.getNextItem('intake_complete', cwd); + + // The marked candidate is excluded; the next one is selected. + expect(result).toEqual({ + ok: true, + candidate: { id: 'WL-NEXT', title: 'Next', stage: 'intake_complete', status: 'open', sortIndex: 20 }, }); }); it('getNextItem passes --worklog-dir when the tab resolved a worklog root', async () => { const mockExec = vi.fn().mockResolvedValue({ - stdout: JSON.stringify({ success: true, workItem: { id: 'SA-ABC', title: 'Sorra task' } }), + stdout: JSON.stringify({ success: true, workItem: { id: 'SA-ABC', title: 'Sorra task', status: 'open' } }), stderr: '', }); setExecFileAsync(mockExec as never); setWorklogDir('/home/user/projects/SorraAgents/.worklog'); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - const result = await deps.getNextItem('intake_complete'); + const result = await deps.getNextItem('intake_complete', '/repo'); // The global option must appear BEFORE the subcommand, exactly as the // worklist's runWl() prepends it (WL-0MSI7DQL10016QYX). @@ -558,11 +612,13 @@ describe('createDowntimeDeps', () => { 'next', '--stage', 'intake_complete', + '-n', + '10', '--json', ], expect.anything(), ); - expect(result).toEqual({ ok: true, candidate: { id: 'SA-ABC', title: 'Sorra task', stage: 'intake_complete' } }); + expect(result).toEqual({ ok: true, candidate: { id: 'SA-ABC', title: 'Sorra task', stage: 'intake_complete', status: 'open', sortIndex: undefined } }); }); it('getNextItem reports ok:true with no candidate when wl reports no item', async () => { @@ -573,24 +629,24 @@ describe('createDowntimeDeps', () => { setExecFileAsync(mockExec as never); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - expect(await deps.getNextItem('idea')).toEqual({ ok: true, candidate: null }); + expect(await deps.getNextItem('idea', '/repo')).toEqual({ ok: true, candidate: null }); }); it('getNextItem fails closed ({ok:false}) when wl errors', async () => { setExecFileAsync(vi.fn().mockRejectedValue(new Error('wl boom')) as never); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - expect(await deps.getNextItem('idea')).toEqual({ ok: false }); + expect(await deps.getNextItem('idea', '/repo')).toEqual({ ok: false }); }); it('getNextItem passes a bounded timeout so a hung wl fails closed', async () => { const mockExec = vi.fn().mockResolvedValue({ - stdout: JSON.stringify({ success: true, workItem: { id: 'WL-ABC', title: 'Some task' } }), + stdout: JSON.stringify({ success: true, workItem: { id: 'WL-ABC', title: 'Some task', status: 'open' } }), stderr: '', }); setExecFileAsync(mockExec as never); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - await deps.getNextItem('intake_complete'); + await deps.getNextItem('intake_complete', '/repo'); // AC1: the invocation must carry the bounded timeout so a hung wl child // is killed by execFile instead of wedging the dispatch task. @@ -616,7 +672,7 @@ describe('createDowntimeDeps', () => { setExecFileAsync(hungExec as never); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - const resultPromise = deps.getNextItem('idea'); + const resultPromise = deps.getNextItem('idea', '/repo'); await vi.advanceTimersByTimeAsync(DOWNTIME_WL_TIMEOUT_MS - 1); // Still pending just before the timeout — no premature resolution. await vi.advanceTimersByTimeAsync(1); @@ -1066,7 +1122,7 @@ describe('createDowntimeDeps', () => { return Promise.resolve({ stdout: JSON.stringify({ success: true, workItem: null }), stderr: '' }); }); setExecFileAsync(mockExec as never); - const spawnFn = vi.fn(() => ({ unref: vi.fn() })); + const spawnFn = vi.fn(() => ({ unref: vi.fn(), once: vi.fn() })); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map', spawnFn); const cwd = makeTempDir(); @@ -1087,22 +1143,56 @@ describe('createDowntimeDeps', () => { expect(spawnFn).toHaveBeenCalledTimes(1); }); - it('claimItem runs wl update --status in_progress --assignee', async () => { + it('claimItem runs wl update --status in_progress --assignee with CAS guards', async () => { const mockExec = vi.fn().mockResolvedValue({ stdout: '', stderr: '' }); setExecFileAsync(mockExec as never); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - await deps.claimItem('WL-ABC'); + const result = await deps.claimItem('WL-ABC', { status: 'open', stage: 'intake_complete' }); expect(mockExec).toHaveBeenCalledWith( 'wl', - ['update', 'WL-ABC', '--status', 'in_progress', '--assignee', 'Map', '--json'], + [ + 'update', + 'WL-ABC', + '--status', + 'in_progress', + '--assignee', + 'Map', + '--if-status', + 'open', + '--if-stage', + 'intake_complete', + '--json', + ], expect.anything(), ); + expect(result).toEqual({ ok: true }); + }); + + it('claimItem resolves stale when the CAS guard fails (another pane won the race)', async () => { + const mockExec = vi + .fn() + .mockRejectedValue(new Error('{"success":false,"error":"stale","message":"Conditional update skipped"}')); + setExecFileAsync(mockExec as never); + + const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); + const result = await deps.claimItem('WL-ABC', { status: 'open', stage: 'idea' }); + + expect(result).toEqual({ ok: false, reason: 'stale' }); + }); + + it('claimItem resolves error (a strike) when wl fails for any other reason', async () => { + setExecFileAsync(vi.fn().mockRejectedValue(new Error('wl boom')) as never); + + const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); + const result = await deps.claimItem('WL-ABC', { status: 'open', stage: 'idea' }); + + expect(result).toEqual({ ok: false, reason: 'error' }); }); it('spawnAgentPane spawns send-to-pi.sh with the derived pane name and args', async () => { - const spawnFn = vi.fn(() => ({ unref: vi.fn() })); + const spawnFn = vi.fn(() => ({ unref: vi.fn(), once: vi.fn() })); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map', spawnFn); await deps.spawnAgentPane('Run /skill:plan WL-ABC — Some task.', { @@ -1129,7 +1219,7 @@ describe('createDowntimeDeps', () => { }); it('spawnAgentPane derives the intake pane name from the prompt', async () => { - const spawnFn = vi.fn(() => ({ unref: vi.fn() })); + const spawnFn = vi.fn(() => ({ unref: vi.fn(), once: vi.fn() })); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map', spawnFn); await deps.spawnAgentPane('Run /skill:intake WL-DEF — An idea.', { @@ -1145,7 +1235,7 @@ describe('createDowntimeDeps', () => { }); it('spawnAgentPane derives the audit pane name from the prompt', async () => { - const spawnFn = vi.fn(() => ({ unref: vi.fn() })); + const spawnFn = vi.fn(() => ({ unref: vi.fn(), once: vi.fn() })); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map', spawnFn); await deps.spawnAgentPane('Run /skill:audit WL-AUD — Audit me.', { @@ -1253,21 +1343,28 @@ describe('createDowntimeDeps recordDispatch', () => { expect(entry.dispatchedAt).toBe('2026-01-01T00:00:00.000Z'); }); - it('is fail-closed when wl comment add fails (no throw, dispatch unaffected)', async () => { + it('resolves true even when wl comment add fails (comment is not the marker)', async () => { setExecFileAsync(vi.fn().mockRejectedValue(new Error('wl boom')) as never); + const cwd = makeTempDir(); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); + // The comment write failing must NOT abort the dispatch — the rolling-log + // marker is the dispatch-suppression source. The log write succeeds, so + // recordDispatch resolves true (no throw, dispatch proceeds). await expect( deps.recordDispatch({ itemId: 'WL-ABC', kind: 'plan', dispatchedAt: '2026-01-01T00:00:00.000Z', - cwd: '/repo', + cwd, }), - ).resolves.toBeUndefined(); + ).resolves.toBe(true); + // The marker still landed even though the comment failed. + const raw = readFileSync(join(cwd, '.worklog', DOWNTIME_LOG_FILE), 'utf8'); + expect(raw).toContain('WL-ABC'); }); - it('is fail-closed when the log write fails (e.g. .worklog path is a file)', async () => { + it('resolves false (marker write failed → dispatch aborts) when the log write fails', async () => { setExecFileAsync(vi.fn().mockResolvedValue({ stdout: '{}', stderr: '' }) as never); const cwd = makeTempDir(); writeFileSync(join(cwd, '.worklog'), 'not a directory', 'utf8'); @@ -1280,7 +1377,7 @@ describe('createDowntimeDeps recordDispatch', () => { dispatchedAt: '2026-01-01T00:00:00.000Z', cwd, }), - ).resolves.toBeUndefined(); + ).resolves.toBe(false); }); }); diff --git a/packages/herdr/src/index.ts b/packages/herdr/src/index.ts index 9b7932777..72edff201 100644 --- a/packages/herdr/src/index.ts +++ b/packages/herdr/src/index.ts @@ -45,11 +45,12 @@ import { createDowntimePoller, buildDowntimePaneArgs, spawnDowntimePane, - parseNextItemOutput, + parseNextCandidatesOutput, parseAuditCandidatesOutput, parseImplementCandidatesOutput, selectAuditCandidate, selectImplementCandidate, + selectNextCandidate, toDowntimeCandidate, toImplementCandidate, skillKindFromPrompt, @@ -60,6 +61,8 @@ import { type DowntimeDispatchEvent, type DowntimeErrorEvent, type DowntimeNextResult, + type DowntimeClaimExpected, + type DowntimeClaimResult, type DowntimeSpawn, defaultDowntimeSpawn, buildDowntimeDispatchComment, @@ -69,6 +72,8 @@ import { appendDowntimeLogEntry, auditDispatchedItemIds, implementDispatchedItemIds, + planDispatchedItemStages, + intakeDispatchedItemStages, readDowntimeLogEntries, } from './downtime-log.js'; @@ -348,21 +353,41 @@ export function createDowntimeDeps( spawnFn: DowntimeSpawn = defaultDowntimeSpawn, ): DowntimeWorkerDeps { return { - async getNextItem(stage: DowntimeStage): Promise { + async getNextItem(stage: DowntimeStage, cwd: string): Promise { try { // buildWlArgs() prepends the tab's resolved --worklog-dir override // (WL-0MSI7DQL10016QYX): the downtime worker must select candidates // from the SAME worklog root the worklist uses, not the plugin // process's own cwd. Without the override the vector is unchanged. - // The bounded timeout (WL-0MSJIPHD0001L1J9) kills a hung wl child - // so the lookup fails closed to a strike instead of wedging the - // dispatch task until the pane restarts. + // A generous batch (-n 10) is fetched so a marker-excluded top + // candidate does not starve selection of the next one. The bounded + // timeout (WL-0MSJIPHD0001L1J9) kills a hung wl child so the lookup + // fails closed to a strike instead of wedging the dispatch task + // until the pane restarts. const { stdout } = await getExecFileAsync()( 'wl', - buildWlArgs(['next', '--stage', stage, '--json']), + buildWlArgs(['next', '--stage', stage, '-n', '10', '--json']), { encoding: 'utf8', timeout: DOWNTIME_WL_TIMEOUT_MS }, ); - return { ok: true, candidate: parseNextItemOutput(stdout, stage) }; + const candidates = parseNextCandidatesOutput(stdout, stage); + if (candidates === null) return { ok: false }; + // Plan/intake dispatched-marker exclusion with change-guard (RCA + // WL-0MSRBFFLN005W3VT design point 3, RC-2): read the shared rolling + // dispatch log for THIS worklog root and exclude any candidate the + // downtime worker already dispatched for its tier (kind plan at + // intake_complete / kind intake at idea) while the item is still at + // its dispatched-at stage. A stage advancement releases it (the item + // leaves the stage filter anyway). Fail-safe: readDowntimeLogEntries + // never throws — a missing or unreadable log is treated as empty. + const entries = await readDowntimeLogEntries(cwd); + const dispatched = + stage === 'intake_complete' + ? planDispatchedItemStages(entries) + : stage === 'idea' + ? intakeDispatchedItemStages(entries) + : new Map(); + const selected = selectNextCandidate(candidates, dispatched); + return { ok: true, candidate: selected }; } catch { // Transient wl failure → fail closed to busy: no dispatch, and the // worker must NOT treat it as an empty backlog (no cooldown). @@ -452,29 +477,42 @@ export function createDowntimeDeps( return null; } }, - async claimItem(itemId: string): Promise { - // claimWorkItem never throws — failures are returned, but the result is - // deliberately discarded here: a failed claim must not block the - // dispatch. Note this path does NOT log the failure (unlike - // claimItemForAgentCommand), so a claim failure is silent and the - // dispatch is still recorded as a success (known silent path, follow-up - // WL-0MSLWJ310000ND0X; see README "Failure-path logging"). - await claimWorkItem(itemId, assignee); + async claimItem(itemId: string, expected: DowntimeClaimExpected): Promise { + // CAS claim (RCA WL-0MSRBFFLN005W3VT design point 1): the transition + // only applies while the item is still in the state the tier selected + // it in — exactly one concurrent pane wins. A stale result (another + // pane claimed first) or any other claim failure ABORTS the dispatch + // (no pane, no marker, no success record) and is never silently + // discarded (WL-0MSLWJ310000ND0X absorbed): the outcome reason + // ('claim-failed' / 'wl-error') is the durable observable, and the + // failure detail is written to stderr like claimItemForAgentCommand. + const result = await claimWorkItem(itemId, assignee, expected); + if (result.success) return { ok: true }; + process.stderr.write( + `[worklog-plugin] Downtime claim failed for ${itemId}: ` + + `${result.error ?? 'unknown error'}` + + `${result.stale ? ' (another pane won the claim race — dispatch aborted)' : ''}\n`, + ); + return result.stale + ? { ok: false, reason: 'stale' } + : { ok: false, reason: 'error' }; }, - async spawnAgentPane(prompt: string, opts: { model: string; cwd: string }): Promise { + async spawnAgentPane(prompt: string, opts: { model: string; cwd: string }): Promise { const kind = skillKindFromPrompt(prompt); - spawnDowntimePane( + return spawnDowntimePane( scriptPath, buildDowntimePaneArgs(kind, prompt, opts), { cwd: opts.cwd }, spawnFn, ); }, - async recordDispatch(event: DowntimeDispatchEvent): Promise { + async recordDispatch(event: DowntimeDispatchEvent): Promise { // 1. Durable trail: a comment on the item itself (survives wl sync). // buildWlArgs() prepends the resolved --worklog-dir override so the // comment lands on the item in ITS project's DB, not the plugin - // process's own cwd (WL-0MSI7DQL10016QYX). + // process's own cwd (WL-0MSI7DQL10016QYX). A comment failure is + // tolerated — the comment is the durable cross-machine trail, not the + // marker. try { await getExecFileAsync()( 'wl', @@ -493,11 +531,16 @@ export function createDowntimeDeps( } catch { // fail-closed: audit logging must never crash the worker } - // 2. Rolling local log (bounded JSONL under /.worklog). + // 2. Rolling local log (bounded JSONL under /.worklog) — the + // dispatched MARKER. Written BEFORE the pane spawns (RCA design point + // 2): a write failure resolves false so the dispatcher aborts rather + // than dispatching an unmarked item. try { await appendDowntimeLogEntry(event.cwd, JSON.stringify(event)); + return true; } catch { - // fail-closed + // fail-closed: the marker could not be written → abort the dispatch + return false; } }, async recordError(event: DowntimeErrorEvent): Promise { diff --git a/packages/shared/src/database.test.ts b/packages/shared/src/database.test.ts index 9a796da69..a8bb4fa81 100644 --- a/packages/shared/src/database.test.ts +++ b/packages/shared/src/database.test.ts @@ -184,6 +184,74 @@ describe('update() no-op guard (WL-0MSORD6HC005QVZX)', () => { }); }); +describe('updateIfMatches() CAS claim (RCA WL-0MSRBFFLN005W3VT design point 1)', () => { + function seed(status: string = 'open', stage: string = 'intake_complete'): WorkItem { + const item = makeItem({ id: 'WI-CAS1', status: status as WorkItem['status'], stage }); + db.import([item]); + return item; + } + + it('applies the update when status and stage both match', () => { + const seeded = seed('open', 'intake_complete'); + const result = db.updateIfMatches(seeded.id, { status: 'in-progress', assignee: 'Map' }, { + status: 'open', + stage: 'intake_complete', + }); + expect(result.ok).toBe(true); + expect(result.item?.status).toBe('in-progress'); + expect(result.item?.assignee).toBe('Map'); + }); + + it('fails stale (no write) when the status no longer matches', () => { + const seeded = seed('completed', 'in_review'); + // Another pane already claimed it: the guard expects completed, but the + // stored status is in-progress. + db.update(seeded.id, { status: 'in-progress' }); + const result = db.updateIfMatches(seeded.id, { status: 'in-progress', assignee: 'Map' }, { + status: 'completed', + stage: 'in_review', + }); + expect(result.ok).toBe(false); + expect(result.reason).toBe('stale'); + // No write happened: the item is unchanged (still the other pane's claim). + expect(db.get(seeded.id)?.assignee).not.toBe('Map'); + }); + + it('fails stale when the stage no longer matches', () => { + const seeded = seed('open', 'intake_complete'); + db.update(seeded.id, { stage: 'plan_complete' }); + const result = db.updateIfMatches(seeded.id, { status: 'in-progress' }, { + status: 'open', + stage: 'intake_complete', + }); + expect(result.ok).toBe(false); + expect(result.reason).toBe('stale'); + expect(db.get(seeded.id)?.status).toBe('open'); // unchanged + }); + + it('normalizes status spelling (in_progress matches stored in-progress)', () => { + const seeded = seed('open', 'idea'); + const result = db.updateIfMatches(seeded.id, { status: 'in-progress' }, { status: 'open' }); + expect(result.ok).toBe(true); + // And the guard itself is spelling-insensitive on the stored value too. + const r2 = db.updateIfMatches(seeded.id, { assignee: 'Map' }, { status: 'in_progress' }); + expect(r2.ok).toBe(true); + }); + + it('fails not-found for an unknown id', () => { + const result = db.updateIfMatches('WI-NOPE', { status: 'in-progress' }, { status: 'open' }); + expect(result.ok).toBe(false); + expect(result.reason).toBe('not-found'); + }); + + it('applies without a guard when no expected state is given', () => { + const seeded = seed('open', 'idea'); + const result = db.updateIfMatches(seeded.id, { status: 'in-progress' }); + expect(result.ok).toBe(true); + expect(result.item?.status).toBe('in-progress'); + }); +}); + describe('upsertItems() no-op guard (WL-0MSORD6HC005QVZX)', () => { it('preserves updatedAt when only description differs by a trailing newline', () => { const original = makeItem(); diff --git a/packages/shared/src/database.ts b/packages/shared/src/database.ts index 53fd5992e..09e7c008c 100644 --- a/packages/shared/src/database.ts +++ b/packages/shared/src/database.ts @@ -29,6 +29,18 @@ export interface GitTarget { branch: string; } +/** + * Result of a conditional update (`updateIfMatches`, the CAS claim). + * `ok:true` carries the updated item; `ok:false` distinguishes "no such + * item" from "item no longer in the expected status/stage" (another + * process won the race). + */ +export interface UpdateIfMatchesResult { + ok: boolean; + reason?: 'not-found' | 'stale'; + item?: WorkItem; +} + /** * Optional CLI-specific services injected into WorklogDatabase. * When not provided, features that depend on them become no-ops or throw @@ -993,6 +1005,45 @@ export class WorklogDatabase { return this.store.getWorkItem(id); } + /** + * Conditional update (compare-and-swap, RCA WL-0MSRBFFLN005W3VT design + * point 1): apply `input` ONLY if the item's current status/stage match the + * expected guard. Runs the check and the write inside a single `BEGIN + * IMMEDIATE` transaction (see `PersistentStore.transactionImmediate`), so + * two concurrent `wl update --if-status/--if-stage` processes serialize on + * the SQLite write lock: exactly one observes the expected state and wins; + * the loser observes the winner's committed transition and fails `stale` + * without writing. + * + * @returns `{ok:true, item}` on success; `{ok:false, reason:'not-found'}` + * when the id is absent; `{ok:false, reason:'stale'}` when the guard + * mismatch means another process already changed the item. + */ + updateIfMatches( + id: string, + input: UpdateWorkItemInput, + expected: { status?: string; stage?: string } = {}, + ): UpdateIfMatchesResult { + return this.store.transactionImmediate(() => { + const current = this.store.getWorkItem(id); + if (!current) { + return { ok: false, reason: 'not-found' }; + } + if (expected.status !== undefined) { + const want = normalizeStatusValue(expected.status) ?? expected.status; + const have = normalizeStatusValue(current.status) ?? current.status; + if (want !== have) { + return { ok: false, reason: 'stale' }; + } + } + if (expected.stage !== undefined && current.stage !== expected.stage) { + return { ok: false, reason: 'stale' }; + } + const updated = this.update(id, input); + return updated ? { ok: true, item: updated } : { ok: false, reason: 'not-found' }; + }); + } + /** * Update a work item */ diff --git a/packages/shared/src/persistent-store.ts b/packages/shared/src/persistent-store.ts index beeb43d52..7655dfe0f 100644 --- a/packages/shared/src/persistent-store.ts +++ b/packages/shared/src/persistent-store.ts @@ -965,6 +965,22 @@ export class SqlitePersistentStore { return tx(); } + /** + * Execute a function inside a `BEGIN IMMEDIATE` transaction. + * + * The write lock is acquired BEFORE any read inside `fn` (deferred + * transactions acquire it at the first write). This serializes + * check-then-write sequences against OTHER processes' connections: a + * concurrent writer blocks at `BEGIN IMMEDIATE` until this transaction + * commits (busy_timeout), then observes the committed state — which is + * exactly the atomicity the CAS claim (compare-and-swap, RCA + * WL-0MSRBFFLN005W3VT design point 1) needs across herdr panes. + */ + transactionImmediate(fn: () => T): T { + const tx = this.db.transaction(fn); + return tx.immediate(); + } + /** * Create or update a dependency edge */ diff --git a/packages/shared/src/types.ts b/packages/shared/src/types.ts index 3f024880f..00f476a36 100644 --- a/packages/shared/src/types.ts +++ b/packages/shared/src/types.ts @@ -114,6 +114,19 @@ export interface UpdateWorkItemInput { effort?: WorkItemEffortLevel | ''; /** When present, sets the needsProducerReview flag */ needsProducerReview?: boolean; + /** + * CAS guard (compare-and-swap claim, RCA WL-0MSRBFFLN005W3VT design point 1): + * when present, the update only applies if the item's CURRENT status matches + * (normalized hyphenated form). On mismatch the update fails with a `stale` + * result and no row is written — the losing pane of a concurrent claim race + * aborts its dispatch instead of double-claiming. + */ + ifStatus?: WorkItemStatus; + /** + * CAS guard: when present, the update only applies if the item's CURRENT + * stage matches. Composes with `ifStatus` (both must match). + */ + ifStage?: string; } export interface WorkItemQuery { status?: WorkItemStatus[]; diff --git a/src/cli-types.ts b/src/cli-types.ts index abb2da79f..484f1b8cc 100644 --- a/src/cli-types.ts +++ b/src/cli-types.ts @@ -71,6 +71,10 @@ export interface UpdateOptions { description?: string; descriptionFile?: string; status?: WorkItemStatus; + /** CAS guard: only apply the update if the current status matches */ + ifStatus?: string; + /** CAS guard: only apply the update if the current stage matches */ + ifStage?: string; priority?: WorkItemPriority; parent?: string; tags?: string; diff --git a/src/commands/update.ts b/src/commands/update.ts index eacf8a539..6f101642e 100644 --- a/src/commands/update.ts +++ b/src/commands/update.ts @@ -10,6 +10,7 @@ import { humanFormatWorkItem, resolveFormat, extractFilePaths } from './helpers. import { canValidateStatusStage, validateStatusStageCompatibility, validateStatusStageInput } from './status-stage-validation.js'; import { normalizeActionArgs } from './cli-utils.js'; import { buildAuditEntry, formatInvalidAuditFirstLineMessage, inspectAuditFirstLine, redactAuditText } from '../audit.js'; +import { normalizeStatusValue } from '../status-stage-rules.js'; import { submitToOpenBrain } from '../openbrain.js'; import { normalizePriority, CANONICAL_PRIORITIES } from '../validators/priority.js'; @@ -23,6 +24,8 @@ export default function register(ctx: PluginContext): void { .option('-d, --description ', 'New description') .option('--description-file ', 'Read description from a file') .option('-s, --status ', 'New status') + .option('--if-status ', 'Only apply the update if the current status matches (CAS guard)') + .option('--if-stage ', 'Only apply the update if the current stage matches (CAS guard)') .option('-p, --priority ', 'New priority') .option('-P, --parent ', 'New parent ID') .option('--tags ', 'New tags (comma-separated)') @@ -47,14 +50,14 @@ export default function register(ctx: PluginContext): void { // --no-re-sort: skip auto re-sort // --re-sort-sync: force synchronous re-sort (blocking) // Normalize re-sort flags from commander/options - const normalized = normalizeActionArgs(rawArgs, ['title','description','descriptionFile','status','priority','parent','tags','assignee','stage','risk','effort','issueType','createdBy','deletedBy','deleteReason','needsProducerReview','audit','auditText','auditFile','doNotDelegate','prefix','noReSort','reSortSync']); + const normalized = normalizeActionArgs(rawArgs, ['title','description','descriptionFile','status','ifStatus','ifStage','priority','parent','tags','assignee','stage','risk','effort','issueType','createdBy','deletedBy','deleteReason','needsProducerReview','audit','auditText','auditFile','doNotDelegate','prefix','noReSort','reSortSync']); // Robust detection of --no-re-sort that accepts multiple forms Commander // may expose (`noReSort`, `reSort: false`) and also checks raw argv. const cliNoReSort = process.argv.includes('--no-re-sort') || process.argv.includes('--noReSort'); const reSortNo = (((normalized.options as any)?.noReSort === true) || ((normalized.options as any)?.reSort === false) || cliNoReSort); const reSortSync = Boolean((normalized.options as any)?.reSortSync); const knownOptionKeys = [ - 'title','description','descriptionFile','status','priority','parent','tags','assignee','stage','risk','effort','issueType','createdBy','deletedBy','deleteReason','needsProducerReview','audit','auditText','doNotDelegate','prefix','noReSort','reSortSync' + 'title','description','descriptionFile','status','ifStatus','ifStage','priority','parent','tags','assignee','stage','risk','effort','issueType','createdBy','deletedBy','deleteReason','needsProducerReview','audit','auditText','doNotDelegate','prefix','noReSort','reSortSync' ]; const argsHint = rawArgs.map(a => Array.isArray(a) ? `array(${a.length})` : `${typeof a}:${String(a).slice(0,100)}`); if (process.env.WL_DEBUG_UPDATE_ACTION) { @@ -117,6 +120,25 @@ export default function register(ctx: PluginContext): void { } } const statusCandidate = hasProvided('status') ? options.status : undefined; + // CAS guards (compare-and-swap claim, RCA WL-0MSRBFFLN005W3VT design + // point 1): --if-status/--if-stage make the update conditional on the + // item's CURRENT status/stage. A mismatch fails per-id with error + // `stale` (no write), which the herdr downtime worker treats as "another + // pane won the claim race". Status values are normalized the same way + // stored statuses are (underscore → hyphen) so `in_progress` matches + // the stored `in-progress`. + const ifStatusCandidate = hasProvided('ifStatus') ? options.ifStatus : undefined; + const ifStageCandidate = hasProvided('ifStage') ? options.ifStage : undefined; + if (ifStatusCandidate !== undefined) { + const normalized = normalizeStatusValue(String(ifStatusCandidate)); + const valid = + normalized !== undefined && + (['open', 'in-progress', 'completed', 'blocked', 'deleted', 'input-needed'].includes(normalized)); + if (!valid) { + output.error(`Invalid --if-status: "${ifStatusCandidate}". Allowed: open, in_progress, completed, blocked, deleted, input_needed (case-insensitive).`, { success: false, error: 'invalid-arg' }); + process.exit(1); + } + } let priorityCandidate = hasProvided('priority') ? options.priority : undefined; // Validate priority if provided: normalize case, reject P* and unknown tokens if (priorityCandidate !== undefined) { @@ -383,7 +405,42 @@ export default function register(ctx: PluginContext): void { // Capture the previous priority before the update so we can detect a // downgrade away from `critical` and cascade it to children. const oldPriority = (db.get(normalizedId)?.priority) as WorkItemPriority | undefined; - const item = db.update(normalizedId, updates); + // CAS claim path (RCA WL-0MSRBFFLN005W3VT design point 1): when + // --if-status/--if-stage are provided the update applies ONLY if the + // item still matches — the check and the write run in one immediate + // transaction so exactly one concurrent claimer wins. A guard + // mismatch fails per-id with error `stale` (no write); the herdr + // downtime worker maps that to an aborted dispatch (no pane, no + // marker, no success record). + let item: WorkItem | null = null; + if (ifStatusCandidate !== undefined || ifStageCandidate !== undefined) { + const cas = db.updateIfMatches(normalizedId, updates, { + ...(ifStatusCandidate !== undefined + ? { status: String(ifStatusCandidate) } + : {}), + ...(ifStageCandidate !== undefined ? { stage: String(ifStageCandidate) } : {}), + }); + if (!cas.ok) { + if (cas.reason === 'not-found') { + const message = `Work item not found: ${normalizedId}`; + results.push({ id: normalizedId, success: false, error: message }); + } else { + // Guard mismatch — another process (e.g. another herdr pane) + // already claimed or changed the item. Report a distinct error + // code so callers can distinguish a lost race from a real CLI + // failure. + const message = + `Conditional update skipped: ${normalizedId} no longer matches ` + + `--if-status${ifStageCandidate !== undefined ? '/--if-stage' : ''} ` + + `(another process may have claimed it)`; + results.push({ id: normalizedId, success: false, error: 'stale', message }); + } + continue; + } + item = cas.item ?? null; + } else { + item = db.update(normalizedId, updates); + } if (!item) { const message = `Work item not found: ${normalizedId}`; results.push({ id: normalizedId, success: false, error: message }); @@ -487,7 +544,7 @@ export default function register(ctx: PluginContext): void { console.log(`[Parent ${r.demotedParent.parent.id} demoted from ${r.demotedParent.from.status}/${r.demotedParent.from.stage} to ${r.demotedParent.to.status}/${r.demotedParent.to.stage}]`); } } else { - output.error(r.error, { success: false, error: r.error }); + output.error(r.message ?? r.error, { success: false, error: r.error }); } } } diff --git a/tests/cli/update-if-status.test.ts b/tests/cli/update-if-status.test.ts new file mode 100644 index 000000000..6d0e0cc8e --- /dev/null +++ b/tests/cli/update-if-status.test.ts @@ -0,0 +1,99 @@ +/** + * Dedicated tests for `wl update --if-status/--if-stage` — the CAS claim + * guard (compare-and-swap, RCA WL-0MSRBFFLN005W3VT design point 1). + * + * The herdr downtime worker uses the conditional claim so that exactly one + * concurrent pane wins the pre-dispatch claim: the transition to + * `in_progress` only applies while the item is still in the state the tier + * selected it in. A guard mismatch fails per-id with error `stale` (no + * write) and a non-zero exit — the losing pane aborts its dispatch. + * + * Work item: WL-0MSRDEWES0059TZN + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { + cliPath, + execAsync, + enterTempDir, + leaveTempDir, + writeConfig, + writeInitSemaphore, +} from './cli-helpers.js'; + +describe('wl update --if-status/--if-stage (CAS claim)', () => { + let tempState: { tempDir: string; originalCwd: string }; + + beforeEach(() => { + tempState = enterTempDir(); + writeConfig(tempState.tempDir, 'Test Project', 'TEST'); + writeInitSemaphore(tempState.tempDir); + }); + + afterEach(() => { + leaveTempDir(tempState); + }); + + async function createItem(flags = ''): Promise { + const { stdout } = await execAsync( + `tsx ${cliPath} --json create -t "CAS item" ${flags}` + ); + return JSON.parse(stdout).workItem.id; + } + + it('claims (status → in_progress) when the guard matches', async () => { + const id = await createItem(); + const { stdout } = await execAsync( + `tsx ${cliPath} --json update ${id} --status in_progress --assignee Map --if-status open --if-stage idea` + ); + const parsed = JSON.parse(stdout); + expect(parsed.success).toBe(true); + expect(parsed.workItem.status).toBe('in-progress'); + expect(parsed.workItem.assignee).toBe('Map'); + }); + + it('fails stale with a non-zero exit when the status no longer matches (no write)', async () => { + const id = await createItem(); + // Another pane claimed it first. + await execAsync(`tsx ${cliPath} --json update ${id} --status in_progress --assignee Other`); + + const run = await execAsync( + `tsx ${cliPath} --json update ${id} --status in_progress --assignee Map --if-status open --if-stage idea`, + ).catch((e) => e as { stdout: string; stderr: string; exitCode?: number }); + const parsed = JSON.parse((run as { stderr: string }).stderr || (run as { stdout: string }).stdout); + expect(parsed.success).toBe(false); + expect(parsed.error).toBe('stale'); + // No write happened: the winner's claim stands. + const { stdout } = await execAsync(`tsx ${cliPath} --json show ${id}`); + expect(JSON.parse(stdout).workItem.assignee).toBe('Other'); + }); + + it('fails stale when the stage no longer matches', async () => { + const id = await createItem(); + await execAsync(`tsx ${cliPath} --json update ${id} --stage plan_complete`); + + const run = await execAsync( + `tsx ${cliPath} --json update ${id} --status in_progress --if-stage idea`, + ).catch((e) => e as { stderr: string; stdout: string }); + const parsed = JSON.parse((run as { stderr: string }).stderr || (run as { stdout: string }).stdout); + expect(parsed.success).toBe(false); + expect(parsed.error).toBe('stale'); + }); + + it('accepts underscore spelling for the guard (in_progress matches stored in-progress)', async () => { + const id = await createItem(); + // The guard value itself is normalized the same way stored statuses are. + const { stdout } = await execAsync( + `tsx ${cliPath} --json update ${id} --status in_progress --if-status open --if-stage idea` + ); + expect(JSON.parse(stdout).success).toBe(true); + }); + + it('a successful conditional update is not a false success when fields match', async () => { + const id = await createItem(); + const { stdout } = await execAsync( + `tsx ${cliPath} --json update ${id} --status in_progress --if-status open --if-stage idea` + ); + expect(JSON.parse(stdout).success).toBe(true); + }); +}); From 03cc4c7fb7cc7a780ae52113f38ed401e4c98fc7 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 15:25:56 +0100 Subject: [PATCH 04/34] WL-0MSP28F5Z008DAUA: Test suite: per-slot idle tracking (red phase) --- .../herdr/src/downtime-worker.fixtures.ts | 37 ++ packages/herdr/src/downtime-worker.test.ts | 428 ++++++++++++++++++ 2 files changed, 465 insertions(+) diff --git a/packages/herdr/src/downtime-worker.fixtures.ts b/packages/herdr/src/downtime-worker.fixtures.ts index 682a6cc72..77639c13f 100644 --- a/packages/herdr/src/downtime-worker.fixtures.ts +++ b/packages/herdr/src/downtime-worker.fixtures.ts @@ -38,6 +38,43 @@ export const ambiguousZeroTotalSlots: LlamaStatus = { total_slots: 0, }; +// ── Per-slot status payloads (WL-0MSG7P9N8009PCKG) ──────────────────── + +/** All 4 slots free, with per-slot identity served (LP-0MSG5TA7Y002GN39). */ +export const perSlotAllFree: LlamaStatus = { + ...idleAllSlotsFree, + slots: [ + { slot_id: 'slot-1', is_processing: false }, + { slot_id: 'slot-2', is_processing: false }, + { slot_id: 'slot-3', is_processing: false }, + { slot_id: 'slot-4', is_processing: false }, + ], +}; + +/** Only 2 of 4 slots free (slot-3, slot-4 processing) with per-slot identity. */ +export const perSlotTwoFree: LlamaStatus = { + ...idleAllSlotsFree, + available_slots: 2, + slots: [ + { slot_id: 'slot-1', is_processing: false }, + { slot_id: 'slot-2', is_processing: false }, + { slot_id: 'slot-3', is_processing: true }, + { slot_id: 'slot-4', is_processing: true }, + ], +}; + +/** 3 of 4 slots free — only slot-1 processing. */ +export const perSlotOneProcessing: LlamaStatus = { + ...idleAllSlotsFree, + available_slots: 3, + slots: [ + { slot_id: 'slot-1', is_processing: true }, + { slot_id: 'slot-2', is_processing: false }, + { slot_id: 'slot-3', is_processing: false }, + { slot_id: 'slot-4', is_processing: false }, + ], +}; + /** Raw (unparsed) response missing the numeric slot fields → ambiguous. */ export const ambiguousMissingFieldsRaw: Record = { llama_server_running: true, diff --git a/packages/herdr/src/downtime-worker.test.ts b/packages/herdr/src/downtime-worker.test.ts index e0b7f3a8d..897ee8b3e 100644 --- a/packages/herdr/src/downtime-worker.test.ts +++ b/packages/herdr/src/downtime-worker.test.ts @@ -18,6 +18,16 @@ * parseImplementCandidatesOutput / selectImplementCandidate, and * implement prompt/pane helpers. Red phase: these fail until the implement * tier lands (WL-0MSMAYPQP001FLR6). + * + * Per-slot idle-tracking tests (WL-0MSP28F5Z008DAUA, parent + * WL-0MSG7P9N8009PCKG): test-first suite for same-slot idle tracking — + * optional `slots` parsing (LP-0MSG5TA7Y002GN39), fail-closed malformed + * slots, the per-slot idle tracker (Map with + * record()/thresholdMetCount() semantics), per-slot evaluateIdle mode, and + * worker routing (per-slot mode only when slots present AND 0 < N < total; + * global-busy resets all slot timers; all-slots-free fallback without + * per-slot data). Red phase: these fail until the implementation lands + * (WL-0MSP28LSY007NDYX). */ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; @@ -28,6 +38,7 @@ import { fetchLocalStatus, createDowntimePoller, createIdleTracker, + createPerSlotIdleTracker, dispatchDowntimeWork, createDowntimeWorker, buildDowntimePrompt, @@ -55,6 +66,7 @@ import { DEFAULT_DOWNTIME_NO_CANDIDATE_COOLDOWN_MS, DOWNTIME_NO_CANDIDATE_COOLDOWN_FLOOR_MS, type LlamaStatus, + type LlamaSlot, type LlamaStatusFetcher, type DowntimeCandidate, type DowntimeWorkerDeps, @@ -67,6 +79,9 @@ import { statusFixtures, ambiguousMissingFieldsRaw, idleAllSlotsFree, + perSlotAllFree, + perSlotTwoFree, + perSlotOneProcessing, networkErrorFixture, timeoutErrorFixture, httpErrorResponseFixture, @@ -2436,3 +2451,416 @@ describe('three-strike rule on CLI errors (createDowntimeWorker)', () => { expect(deps.recordError).not.toHaveBeenCalled(); // no persistent-error log }); }); + +// ── Per-slot idle tracking (WL-0MSP28F5Z008DAUA, parent WL-0MSG7P9N8009PCKG) ── +// Test-first suite for same-slot idle tracking, deferred follow-up Q7 of +// WL-0MSF49FMW009M06K. The proxy feature LP-0MSG5TA7Y002GN39 serves per-slot +// details (`slots: [{slot_id, is_processing}]`) in GET /llama/local/status; +// when `downtimeRequiredFreeSlots` N < total slots, the worker must track idle +// duration PER SLOT IDENTITY so the SAME N slots are free for the full +// threshold — never any-N transient availability. +// +// Red phase: all tests in this section fail against the current +// implementation (which has no per-slot parsing, no per-slot tracker, and no +// per-slot evaluateIdle/worker routing) until the implementation feature +// WL-0MSP28LSY007NDYX lands. + +// ── parseLlamaStatus: optional per-slot fields (AC1/AC2) ─────────────── + +describe('parseLlamaStatus per-slot slots array', () => { + const base = { + llama_server_running: true, + active_query: false, + model_switch_in_progress: false, + available_slots: 4, + total_slots: 4, + }; + + it('exposes the slots array when the payload serves it', () => { + const status = parseLlamaStatus({ + ...base, + slots: [ + { slot_id: 'slot-1', is_processing: false }, + { slot_id: 'slot-2', is_processing: true }, + ], + }); + expect(status).not.toBeNull(); + expect(status!.slots).toEqual([ + { slot_id: 'slot-1', is_processing: false }, + { slot_id: 'slot-2', is_processing: true }, + ]); + }); + + it('leaves slots undefined when the payload does not serve it (backward compatible)', () => { + const status = parseLlamaStatus(base); + expect(status).not.toBeNull(); + expect(status!.slots).toBeUndefined(); + }); + + it('parses an empty slots array as valid (zero free slots — never dispatches)', () => { + const status = parseLlamaStatus({ ...base, slots: [] }); + expect(status).not.toBeNull(); + expect(status!.slots).toEqual([]); + }); + + it('treats a non-array slots field as ambiguous → null (busy, fail-closed)', () => { + expect(parseLlamaStatus({ ...base, slots: 'not-an-array' })).toBeNull(); + expect(parseLlamaStatus({ ...base, slots: { slot_id: 'x', is_processing: false } })).toBeNull(); + }); + + it('treats an entry missing slot_id as malformed → null', () => { + expect(parseLlamaStatus({ ...base, slots: [{ is_processing: false }] })).toBeNull(); + }); + + it('treats a non-string or empty slot_id as malformed → null', () => { + expect(parseLlamaStatus({ ...base, slots: [{ slot_id: '', is_processing: false }] })).toBeNull(); + expect(parseLlamaStatus({ ...base, slots: [{ slot_id: 42, is_processing: false }] })).toBeNull(); + }); + + it('treats a non-boolean or missing is_processing as malformed → null', () => { + expect(parseLlamaStatus({ ...base, slots: [{ slot_id: 'slot-1', is_processing: 'yes' }] })).toBeNull(); + expect(parseLlamaStatus({ ...base, slots: [{ slot_id: 'slot-1' }] })).toBeNull(); + }); + + it('treats duplicate slot_ids as malformed → null', () => { + expect( + parseLlamaStatus({ + ...base, + slots: [ + { slot_id: 'slot-1', is_processing: false }, + { slot_id: 'slot-1', is_processing: true }, + ], + }), + ).toBeNull(); + }); +}); + +// ── Per-slot idle tracker (AC3/AC4) ─────────────────────────────────── + +describe('per-slot idle tracker (createPerSlotIdleTracker)', () => { + const a: LlamaSlot = { slot_id: 'a', is_processing: false }; + const b: LlamaSlot = { slot_id: 'b', is_processing: false }; + + it('starts a slot timer on its first free poll and continues it across free polls', () => { + const tracker = createPerSlotIdleTracker(); + const start = 1_000_000; + + tracker.record([a, b], start); + expect(tracker.idleSince.get('a')).toBe(start); + expect(tracker.idleSince.get('b')).toBe(start); + + tracker.record([a, b], start + 30_000); + expect(tracker.idleSince.get('a')).toBe(start); // run start stays fixed + expect(tracker.idleSince.get('b')).toBe(start); + }); + + it('a slot starting processing resets ONLY its own timer (other timers continue)', () => { + const tracker = createPerSlotIdleTracker(); + const start = 1_000_000; + + tracker.record([a, b], start); + tracker.record([{ slot_id: 'a', is_processing: true }, b], start + 10_000); + + expect(tracker.idleSince.get('a')).toBeNull(); // own timer reset + expect(tracker.idleSince.get('b')).toBe(start); // other timer continues + }); + + it('a slot absent from the payload is fail-closed to busy (its timer resets)', () => { + const tracker = createPerSlotIdleTracker(); + const start = 1_000_000; + + tracker.record([a, b], start); + tracker.record([a], start + 10_000); // b absent + + expect(tracker.idleSince.get('a')).toBe(start); + expect(tracker.idleSince.get('b')).toBeNull(); + }); + + it('an empty record resets every slot timer (global-busy reset path)', () => { + const tracker = createPerSlotIdleTracker(); + const start = 1_000_000; + + tracker.record([a, b], start); + tracker.record([], start + 10_000); + + expect(tracker.idleSince.get('a')).toBeNull(); + expect(tracker.idleSince.get('b')).toBeNull(); + }); + + it('thresholdMetCount counts only slots continuously free for the full threshold', () => { + const tracker = createPerSlotIdleTracker(); + const start = 1_000_000; + + tracker.record([a, b], start); + expect(tracker.thresholdMetCount(240_000, start + 239_999)).toBe(0); + expect(tracker.thresholdMetCount(240_000, start + 240_000)).toBe(2); + expect(tracker.thresholdMetCount(240_000, start + 500_000)).toBe(2); + }); + + it('a reset slot needs a fresh full run before it counts toward the threshold again', () => { + const tracker = createPerSlotIdleTracker(); + const start = 1_000_000; + + tracker.record([a, b], start); + // a goes processing at +10s (own timer reset) and free again at +20s. + tracker.record([{ slot_id: 'a', is_processing: true }, b], start + 10_000); + tracker.record([a, b], start + 20_000); + + // Only b has a full continuous run at start+threshold+5s. + expect(tracker.thresholdMetCount(240_000, start + 240_000 + 5_000)).toBe(1); + // a completes its fresh run at start+20s+threshold → both count. + expect(tracker.thresholdMetCount(240_000, start + 20_000 + 240_000)).toBe(2); + }); +}); + +// ── evaluateIdle: per-slot mode (AC5 + per-slot free count) ─────────── + +describe('evaluateIdle per-slot mode (0 < N < total with slots present)', () => { + const free = (id: string): LlamaSlot => ({ slot_id: id, is_processing: false }); + const busy = (id: string): LlamaSlot => ({ slot_id: id, is_processing: true }); + const perSlot = (slots: LlamaSlot[], available: number, total: number): LlamaStatus => ({ + llama_server_running: true, + active_query: false, + model_switch_in_progress: false, + local_lease_active: false, + available_slots: available, + total_slots: total, + slots, + }); + // 4 slots, 2 free (slot-1, slot-2). + const twoFree = perSlot([free('s1'), free('s2'), busy('s3'), busy('s4')], 2, 4); + + it('uses the per-slot free count when per-slot data is present and 0 < N < total', () => { + expect(evaluateIdle(twoFree, 2)).toBe(true); // 2 of 4 free — enough + expect(evaluateIdle(twoFree, 3)).toBe(false); // needs 3 — not enough + }); + + it('per-slot mode still requires the base idle checks (global busy → false)', () => { + expect(evaluateIdle({ ...twoFree, active_query: true }, 2)).toBe(false); + expect(evaluateIdle({ ...twoFree, model_switch_in_progress: true }, 2)).toBe(false); + expect(evaluateIdle({ ...twoFree, local_lease_active: true }, 2)).toBe(false); + expect(evaluateIdle({ ...twoFree, llama_server_running: false }, 2)).toBe(false); + }); + + it('N=0 (default) still requires ALL slots free even when per-slot data is present', () => { + expect(evaluateIdle(twoFree, 0)).toBe(false); // 2 < 4 → busy + expect(evaluateIdle(perSlot([free('s1'), free('s2'), free('s3'), free('s4')], 4, 4), 0)).toBe(true); + }); + + it('N == total still requires ALL slots free even when per-slot data is present', () => { + expect(evaluateIdle(perSlot([free('s1'), free('s2'), free('s3'), free('s4')], 4, 4), 4)).toBe(true); + expect(evaluateIdle(perSlot([free('s1'), free('s2'), busy('s3'), free('s4')], 3, 4), 4)).toBe(false); + }); + + it('N > total never idles even with per-slot data', () => { + expect(evaluateIdle(perSlot([free('s1'), free('s2'), free('s3'), free('s4')], 4, 4), 5)).toBe(false); + }); +}); + +// ── Worker routing: per-slot mode vs fallback (WL-0MSG7P9N8009PCKG AC) ─ + +describe('downtime worker per-slot routing (createDowntimeWorker)', () => { + function makePerSlotWorker(overrides: { + requiredFreeSlots?: number; + thresholdMs?: number; + status?: unknown; + deps?: Partial; + } = {}) { + const cfg = { + enabled: true, + thresholdMs: overrides.thresholdMs ?? 240_000, + requiredFreeSlots: overrides.requiredFreeSlots ?? 2, + model: 'plan', + cwd: '/repo', + noCandidateCooldownMs: 3_600_000, + }; + const fetcher = vi + .fn() + .mockResolvedValue(jsonResponseFixture(overrides.status ?? perSlotTwoFree)); + const poller = createDowntimePoller('http://proxy:8000', fetcher); + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete' }, + }), + ...overrides.deps, + }); + const worker = createDowntimeWorker({ + poller, + deps, + config: () => ({ ...cfg }), + }); + return { worker, deps, cfg, fetcher }; + } + + beforeEach(() => { + vi.useFakeTimers(); + }); + + afterEach(() => { + vi.useRealTimers(); + }); + + it('routes to per-slot mode: dispatches with only N of total slots free (same slots, full threshold)', async () => { + // perSlotTwoFree: 2 of 4 slots free. Legacy all-slots-free mode would + // NEVER dispatch (available 2 < 4); per-slot mode dispatches after the + // threshold because the SAME 2 slots have been free throughout. + const { worker, deps, cfg } = makePerSlotWorker({ requiredFreeSlots: 2 }); + const start = 1_000_000; + vi.setSystemTime(start); + await worker.tick(); + expect(worker.idleSince ?? null).not.toBeNull(); // idle run starts + + vi.setSystemTime(start + cfg.thresholdMs - 1); + const before = await worker.tick(); + expect(before.dispatched).toBe(false); + expect(deps.spawnAgentPane).not.toHaveBeenCalled(); + + vi.setSystemTime(start + cfg.thresholdMs); + const at = await worker.tick(); + expect(at.dispatched).toBe(true); + expect(deps.spawnAgentPane).toHaveBeenCalledTimes(1); + }); + + it('a slot starting processing resets only its own timer: other slots keep their run and can dispatch', async () => { + // N=1. slot-1 goes processing at +10s while slot-2 stays free. The poll + // stays per-slot idle (free count 3 ≥ 1), slot-2's timer continues, and + // dispatch fires at the original threshold — not delayed by slot-1. + const { worker, deps, cfg, fetcher } = makePerSlotWorker({ requiredFreeSlots: 1 }); + const start = 1_000_000; + vi.setSystemTime(start); + await worker.tick(); // all 4 free → timers start + + fetcher.mockResolvedValueOnce(jsonResponseFixture(perSlotOneProcessing)); + vi.setSystemTime(start + 10_000); + const mid = await worker.tick(); + expect(mid.idle).toBe(true); // free count 3 ≥ N=1 → still idle + + vi.setSystemTime(start + cfg.thresholdMs); + const at = await worker.tick(); + expect(at.dispatched).toBe(true); + expect(deps.spawnAgentPane).toHaveBeenCalledTimes(1); + }); + + it('a processing slot that must count toward N delays dispatch by its own fresh run', async () => { + // N=2 with exactly 2 slots. slot-1 is processing from +10s to +20s: its + // timer resets, so dispatch cannot fire at the original threshold (only + // slot-2 has a full run); it fires only after slot-1's fresh run. + const { worker, deps, cfg, fetcher } = makePerSlotWorker({ requiredFreeSlots: 2 }); + const start = 1_000_000; + vi.setSystemTime(start); + await worker.tick(); // slot-1, slot-2 free → timers start + + const twoSlots = (s1: boolean, s2: boolean): LlamaStatus => ({ + ...perSlotAllFree, + available_slots: (s1 ? 0 : 1) + (s2 ? 0 : 1), + total_slots: 2, + slots: [ + { slot_id: 'slot-1', is_processing: s1 }, + { slot_id: 'slot-2', is_processing: s2 }, + ], + }); + fetcher.mockResolvedValueOnce(jsonResponseFixture(twoSlots(true, false))); + vi.setSystemTime(start + 10_000); + await worker.tick(); // slot-1 processing → its timer resets + + fetcher.mockResolvedValueOnce(jsonResponseFixture(twoSlots(false, false))); + vi.setSystemTime(start + 20_000); + await worker.tick(); // both free again → slot-1 restarts its run + + vi.setSystemTime(start + cfg.thresholdMs); + const at = await worker.tick(); + expect(at.dispatched).toBe(false); // only slot-2 has a full continuous run + + vi.setSystemTime(start + 20_000 + cfg.thresholdMs); + const after = await worker.tick(); + expect(after.dispatched).toBe(true); // slot-1 completed its fresh run + expect(deps.spawnAgentPane).toHaveBeenCalledTimes(1); + }); + + it('a global busy poll resets ALL per-slot timers (fresh full idle period required)', async () => { + const { worker, deps, cfg, fetcher } = makePerSlotWorker({ requiredFreeSlots: 2 }); + const start = 1_000_000; + vi.setSystemTime(start); + await worker.tick(); // all free → timers start + + fetcher.mockResolvedValueOnce(jsonResponseFixture({ ...perSlotAllFree, active_query: true })); + vi.setSystemTime(start + 10_000); + await worker.tick(); // global busy → every slot timer reset + + fetcher.mockResolvedValueOnce(jsonResponseFixture(perSlotAllFree)); + vi.setSystemTime(start + 20_000); + await worker.tick(); // free again → timers restart at +20s + + vi.setSystemTime(start + cfg.thresholdMs); + const at = await worker.tick(); + expect(at.dispatched).toBe(false); // no slot has a full run since +20s + + vi.setSystemTime(start + 20_000 + cfg.thresholdMs); + const after = await worker.tick(); + expect(after.dispatched).toBe(true); + expect(deps.spawnAgentPane).toHaveBeenCalledTimes(1); + }); + + it('an ambiguous poll (null status) resets ALL per-slot timers (fail-closed)', async () => { + const { worker, deps, cfg, fetcher } = makePerSlotWorker({ requiredFreeSlots: 2 }); + const start = 1_000_000; + vi.setSystemTime(start); + await worker.tick(); // all free → timers start + + fetcher.mockResolvedValueOnce(jsonResponseFixture(null)); + vi.setSystemTime(start + 10_000); + await worker.tick(); // ambiguous → busy → every slot timer reset + + fetcher.mockResolvedValueOnce(jsonResponseFixture(perSlotAllFree)); + vi.setSystemTime(start + 20_000); + await worker.tick(); // free again → timers restart at +20s + + vi.setSystemTime(start + cfg.thresholdMs); + const at = await worker.tick(); + expect(at.dispatched).toBe(false); + + vi.setSystemTime(start + 20_000 + cfg.thresholdMs); + const after = await worker.tick(); + expect(after.dispatched).toBe(true); + expect(deps.spawnAgentPane).toHaveBeenCalledTimes(1); + }); + + it('without per-slot data and 0 < N < total, falls back to all-slots-free (never any N slots)', async () => { + // No slots array: evaluateIdle degrades to all-slots-free, so 2 of 4 + // free is BUSY even though N=2 — the worker never dispatches on + // any-N availability without per-slot identity. + const { worker, deps, cfg, fetcher } = makePerSlotWorker({ + requiredFreeSlots: 2, + status: { ...idleAllSlotsFree, available_slots: 2 }, + }); + const start = 1_000_000; + vi.setSystemTime(start); + const first = await worker.tick(); + expect(first.idle).toBe(false); // 2 of 4 free without identity → busy + + vi.setSystemTime(start + cfg.thresholdMs); + const at = await worker.tick(); + expect(at.dispatched).toBe(false); + expect(deps.spawnAgentPane).not.toHaveBeenCalled(); + expect(fetcher).toHaveBeenCalled(); // still polling — never any-N dispatch + }); + + it('with per-slot data but N=0, falls back to all-slots-free (per-slot mode only when 0 < N < total)', async () => { + // Slots are present but N=0 (all slots): per-slot mode is NOT active — + // all-slots-free applies, so 2 of 4 free is busy and never dispatches. + const { worker, deps, cfg } = makePerSlotWorker({ + requiredFreeSlots: 0, + status: perSlotTwoFree, + }); + const start = 1_000_000; + vi.setSystemTime(start); + const first = await worker.tick(); + expect(first.idle).toBe(false); + + vi.setSystemTime(start + cfg.thresholdMs); + const at = await worker.tick(); + expect(at.dispatched).toBe(false); + expect(deps.spawnAgentPane).not.toHaveBeenCalled(); + }); +}); From ea25eab137e647457655be3d2633b19b863ecd52 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 15:50:16 +0100 Subject: [PATCH 05/34] WL-0MSP28LSY007NDYX: Implementation complete --- packages/herdr/src/downtime-worker.test.ts | 27 +-- packages/herdr/src/downtime-worker.ts | 199 +++++++++++++++++++-- 2 files changed, 198 insertions(+), 28 deletions(-) diff --git a/packages/herdr/src/downtime-worker.test.ts b/packages/herdr/src/downtime-worker.test.ts index 897ee8b3e..5de41e58e 100644 --- a/packages/herdr/src/downtime-worker.test.ts +++ b/packages/herdr/src/downtime-worker.test.ts @@ -2743,30 +2743,33 @@ describe('downtime worker per-slot routing (createDowntimeWorker)', () => { }); it('a processing slot that must count toward N delays dispatch by its own fresh run', async () => { - // N=2 with exactly 2 slots. slot-1 is processing from +10s to +20s: its - // timer resets, so dispatch cannot fire at the original threshold (only - // slot-2 has a full run); it fires only after slot-1's fresh run. + // Per-slot mode with 4 slots, N=2: slot-1 is processing from +10s to + // +20s while slot-2 stays free. slot-1's OWN timer resets, so dispatch + // cannot fire at the original threshold (only slot-2 has a full + // continuous run); it fires only after slot-1's fresh run completes. const { worker, deps, cfg, fetcher } = makePerSlotWorker({ requiredFreeSlots: 2 }); const start = 1_000_000; vi.setSystemTime(start); - await worker.tick(); // slot-1, slot-2 free → timers start + await worker.tick(); // slot-1, slot-2 free → their timers start - const twoSlots = (s1: boolean, s2: boolean): LlamaStatus => ({ + const fourSlots = (s1: boolean): LlamaStatus => ({ ...perSlotAllFree, - available_slots: (s1 ? 0 : 1) + (s2 ? 0 : 1), - total_slots: 2, + available_slots: (s1 ? 0 : 1) + 1, // slot-2 free + slot-1 free + total_slots: 4, slots: [ { slot_id: 'slot-1', is_processing: s1 }, - { slot_id: 'slot-2', is_processing: s2 }, + { slot_id: 'slot-2', is_processing: false }, + { slot_id: 'slot-3', is_processing: true }, + { slot_id: 'slot-4', is_processing: true }, ], }); - fetcher.mockResolvedValueOnce(jsonResponseFixture(twoSlots(true, false))); + fetcher.mockResolvedValueOnce(jsonResponseFixture(fourSlots(true))); vi.setSystemTime(start + 10_000); - await worker.tick(); // slot-1 processing → its timer resets + await worker.tick(); // slot-1 processing → its own timer resets - fetcher.mockResolvedValueOnce(jsonResponseFixture(twoSlots(false, false))); + fetcher.mockResolvedValueOnce(jsonResponseFixture(fourSlots(false))); vi.setSystemTime(start + 20_000); - await worker.tick(); // both free again → slot-1 restarts its run + await worker.tick(); // slot-1 free again → restarts its run vi.setSystemTime(start + cfg.thresholdMs); const at = await worker.tick(); diff --git a/packages/herdr/src/downtime-worker.ts b/packages/herdr/src/downtime-worker.ts index f9082722c..3f33abb44 100644 --- a/packages/herdr/src/downtime-worker.ts +++ b/packages/herdr/src/downtime-worker.ts @@ -156,10 +156,44 @@ export interface LlamaStatus { current_model?: string; local_owner_session_id?: string | null; local_owner_lease_remaining_seconds?: number | null; + /** + * Per-slot identity (LP-0MSG5TA7Y002GN39): served by proxies that expose + * slot-level detail. ABSENT on pre-feature proxies — the worker then + * falls back to the count-based all-slots-free logic. When present, the + * downtime worker tracks idle duration PER SLOT ID so a configured N + * requires the SAME N slots continuously free. + */ + slots?: LlamaSlot[]; +} + +/** Per-slot detail served inside `LlamaStatus.slots`. */ +export interface LlamaSlot { + slot_id: string; + is_processing: boolean; } // ── Idle detection (implemented) ────────────────────────────────────── +/** + * The global (slot-count-independent) idle checks shared by `isIdleStatus` + * and the per-slot branch of `evaluateIdle`: llama-server up, no active + * query (local signal preferred), no model switch, no local lease. + */ +function globalIdleChecks(status: LlamaStatus): boolean { + if (!status.llama_server_running) return false; + // Prefer the local-only signal when the proxy exposes it (LP-0MSL2ZLLS009RVKR): + // remote streams keep the GLOBAL active_query true while the local model is + // idle with free slots, so they must not block downtime dispatch. Fall back + // to the global active_query for pre-fix proxies that do not serve + // local_active_query (backward compatible). + const queryActive = + status.local_active_query !== undefined ? status.local_active_query : status.active_query; + if (queryActive) return false; + if (status.model_switch_in_progress) return false; + if (status.local_lease_active) return false; + return true; +} + /** * True when the proxy reports an idle state for the required free-slot * count: @@ -174,17 +208,7 @@ export interface LlamaStatus { * busy (fail-closed). */ export function isIdleStatus(status: LlamaStatus, requiredFreeSlots: number): boolean { - if (!status.llama_server_running) return false; - // Prefer the local-only signal when the proxy exposes it (LP-0MSL2ZLLS009RVKR): - // remote streams keep the GLOBAL active_query true while the local model is - // idle with free slots, so they must not block downtime dispatch. Fall back - // to the global active_query for pre-fix proxies that do not serve - // local_active_query (backward compatible). - const queryActive = - status.local_active_query !== undefined ? status.local_active_query : status.active_query; - if (queryActive) return false; - if (status.model_switch_in_progress) return false; - if (status.local_lease_active) return false; + if (!globalIdleChecks(status)) return false; const total = status.total_slots; if (!Number.isFinite(total) || total <= 0) return false; // ambiguous → busy @@ -204,10 +228,32 @@ export function isIdleStatus(status: LlamaStatus, requiredFreeSlots: number): bo * 0 < N < total slots degrades to requiring ALL slots free — stricter than * configured, never "any N slots" without same-slot identity. N > total * slots never dispatches (carried through `isIdleStatus`). + * + * Per-slot mode (LP-0MSG5TA7Y002GN39): when the payload serves per-slot + * identity AND 0 < N < total, the free count comes from the slots array + * (which identifies WHICH slots are free) so the same N slots can be + * required; the count-based logic is unchanged for N ≤ 0 / N ≥ total. */ export function evaluateIdle(status: LlamaStatus, requiredFreeSlots: number): boolean { const total = status.total_slots; if (!Number.isFinite(total) || total <= 0) return false; // ambiguous → busy + + // Per-slot mode: per-slot identity present AND 0 < N < total. Base global + // checks still apply; the slot requirement is the per-slot free count. + if ( + Array.isArray(status.slots) && + requiredFreeSlots > 0 && + requiredFreeSlots < total + ) { + if (!globalIdleChecks(status)) return false; + // Fail-closed counting: an entry without an explicit boolean + // `is_processing` is treated as processing (busy), never free. + const free = status.slots.filter( + (s) => typeof s.is_processing === 'boolean' && !s.is_processing, + ).length; + return free >= requiredFreeSlots; + } + const effective = requiredFreeSlots > 0 && requiredFreeSlots < total ? total : requiredFreeSlots; @@ -254,6 +300,68 @@ export function createIdleTracker(): IdleTracker { }; } +// ── Per-slot idle tracker (WL-0MSG7P9N8009PCKG) ─────────────────────── + +/** + * Per-slot idle-duration tracker: one timer per `slot_id` so a configured + * N < total requires the SAME N slots continuously free for the full + * threshold (LP-0MSG5TA7Y002GN39) — never transient any-N availability. + */ +export interface PerSlotIdleTracker { + /** idle-since timestamp per slot_id; null = slot busy / timer reset. */ + readonly idleSince: ReadonlyMap; + /** + * Record one per-slot poll. A free slot starts (first free poll) or + * continues (subsequent free polls) its OWN timer; a processing slot + * resets ONLY its own timer; a slot absent from the payload is + * fail-closed to busy (its timer resets). An empty payload resets every + * known slot timer (the global-busy reset path). + */ + record(slots: LlamaSlot[], now?: number): void; + /** + * Number of slots continuously free for `thresholdMs` or longer. + */ + thresholdMetCount(thresholdMs: number, now?: number): number; +} + +/** + * Map-backed per-slot idle tracker. `idleSince` is set on each slot's first + * free poll and kept fixed across consecutive free polls; the slot's own + * timer resets (null) when it reports processing or disappears from a poll. + */ +export function createPerSlotIdleTracker(): PerSlotIdleTracker { + const idleSince = new Map(); + return { + get idleSince(): ReadonlyMap { + return idleSince; + }, + record(slots: LlamaSlot[], now: number = Date.now()): void { + const reported = new Set(); + for (const slot of slots) { + reported.add(slot.slot_id); + if (slot.is_processing) { + idleSince.set(slot.slot_id, null); // own timer reset only + } else { + idleSince.set(slot.slot_id, idleSince.get(slot.slot_id) ?? now); + } + } + // Fail-closed: any known slot not reported this poll is treated as + // busy (its timer resets) — a slot is only ever considered free when + // the proxy explicitly says so. + for (const slotId of [...idleSince.keys()]) { + if (!reported.has(slotId)) idleSince.set(slotId, null); + } + }, + thresholdMetCount(thresholdMs: number, now: number = Date.now()): number { + let count = 0; + for (const since of idleSince.values()) { + if (since !== null && now - since >= thresholdMs) count += 1; + } + return count; + }, + }; +} + // ── Poller (implemented — F2) ──────────────────────────────────────── /** @@ -309,6 +417,29 @@ export function parseLlamaStatus(raw: unknown): LlamaStatus | null { (typeof leaseSeconds === 'number' && Number.isFinite(leaseSeconds) && leaseSeconds > 0); } + // Optional per-slot identity (LP-0MSG5TA7Y002GN39): absent on pre-feature + // proxies (slots stays undefined — backward compatible). A malformed + // array (non-array, entry missing/empty/non-string slot_id, non-boolean + // is_processing, duplicate slot_ids) is ambiguous → null (busy, + // fail-closed): per-slot tracking must never run on identity it cannot + // trust. An empty array is valid (zero slots reported free). + let slots: LlamaSlot[] | undefined; + if (o.slots !== undefined) { + if (!Array.isArray(o.slots)) return null; + const parsed: LlamaSlot[] = []; + const seen = new Set(); + for (const entry of o.slots) { + if (typeof entry !== 'object' || entry === null) return null; + const slot = entry as Record; + if (typeof slot.slot_id !== 'string' || slot.slot_id.length === 0) return null; + if (typeof slot.is_processing !== 'boolean') return null; + if (seen.has(slot.slot_id)) return null; // duplicate identity → ambiguous + seen.add(slot.slot_id); + parsed.push({ slot_id: slot.slot_id, is_processing: slot.is_processing }); + } + slots = parsed; + } + return { llama_server_running: o.llama_server_running, active_query: o.active_query, @@ -324,6 +455,7 @@ export function parseLlamaStatus(raw: unknown): LlamaStatus | null { typeof o.local_owner_lease_remaining_seconds === 'number' ? o.local_owner_lease_remaining_seconds : undefined, + slots, }; } @@ -954,6 +1086,7 @@ export interface DowntimeWorker { */ export function createDowntimeWorker(opts: DowntimeWorkerConfig): DowntimeWorker { const tracker = createIdleTracker(); + const perSlotTracker = createPerSlotIdleTracker(); let dispatching = false; let lastDispatchAt: number | null = null; // No-candidate cooldown (WL-0MSI7DQL10016QYX): timestamp until which the @@ -1003,15 +1136,46 @@ export function createDowntimeWorker(opts: DowntimeWorkerConfig): DowntimeWorker const status = await opts.poller.poll(); if (status === null) { tracker.record(false); // endpoint failure/ambiguity → busy + perSlotTracker.record([]); // fail-closed: ambiguous poll resets every slot timer return { polled: true, dispatched: false, idle: false }; } - const idle = evaluateIdle(status, cfg.requiredFreeSlots); - tracker.record(idle); - if (!idle) return { polled: true, dispatched: false, idle: false }; - if (!tracker.isThresholdMet(cfg.thresholdMs)) { - return { polled: true, dispatched: false, idle: true }; + // Per-slot routing (LP-0MSG5TA7Y002GN39): when the proxy serves + // per-slot identity AND the config asks for 0 < N < total, idle + // duration is tracked PER SLOT so dispatch requires the SAME N slots + // continuously free for the full threshold. Any global busy condition + // (query / model switch / lease / server down / ambiguous) resets ALL + // slot timers; a slot reporting processing resets only its own. + const perSlotMode = + Array.isArray(status.slots) && + cfg.requiredFreeSlots > 0 && + cfg.requiredFreeSlots < status.total_slots; + + let idle: boolean; + let ready: boolean; + if (perSlotMode && Array.isArray(status.slots)) { + const globalIdle = globalIdleChecks(status); + // Keep the worker's idleSince view in sync with the run state. + tracker.record(globalIdle); + if (globalIdle) { + perSlotTracker.record(status.slots); + idle = true; + ready = + perSlotTracker.thresholdMetCount(cfg.thresholdMs) >= cfg.requiredFreeSlots; + } else { + perSlotTracker.record([]); // global busy → reset every slot timer + idle = false; + ready = false; + } + } else { + // Single global tracker (no per-slot data, or N ≤ 0 / N ≥ total): + // unchanged all-slots-free behaviour (parent AC2 parity). + idle = evaluateIdle(status, cfg.requiredFreeSlots); + tracker.record(idle); + ready = idle && tracker.isThresholdMet(cfg.thresholdMs); } + if (!idle) return { polled: true, dispatched: false, idle: false }; + if (!ready) return { polled: true, dispatched: false, idle: true }; if (dispatching) return { polled: true, dispatched: false, idle: true }; dispatching = true; @@ -1027,6 +1191,7 @@ export function createDowntimeWorker(opts: DowntimeWorkerConfig): DowntimeWorker // report busy, require a fresh full idle period before the next // dispatch (AC5). tracker.record(false); + perSlotTracker.record([]); } else if (outcome.reason === 'no-candidate') { // Genuine empty backlog (both stages answered with no candidate): // pause entirely so the worker stops burning cycles (proxy polling @@ -1035,6 +1200,7 @@ export function createDowntimeWorker(opts: DowntimeWorkerConfig): DowntimeWorker errorStrikes = 0; // the CLI answered — it is healthy cooldownUntil = Date.now() + cfg.noCandidateCooldownMs; tracker.record(false); + perSlotTracker.record([]); } else if (outcome.reason === 'wl-error') { // Three-strike rule on CLI errors: a dispatch attempt ending in a // wl failure is one strike. Three consecutive strikes pause the @@ -1057,6 +1223,7 @@ export function createDowntimeWorker(opts: DowntimeWorkerConfig): DowntimeWorker cooldownUntil = Date.now() + cfg.noCandidateCooldownMs; errorStrikes = 0; tracker.record(false); + perSlotTracker.record([]); } } // Any other non-dispatch outcome (dispatch-in-flight, code-freeze From a46d25b6ce65e915b8941233fa179cddb3afd361 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 15:53:07 +0100 Subject: [PATCH 06/34] WL-0MSP28Q4E0028GP5: Implementation complete --- packages/herdr/README.md | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/packages/herdr/README.md b/packages/herdr/README.md index f8424fec2..c9ba65c4e 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -204,8 +204,10 @@ Settings (all re-read each poll, so changes apply without a plugin restart): - `downtimeIdleThresholdMs` — Minimum continuous idle duration before a dispatch (default: `240000` = 4 minutes, floor 1s) - `downtimeRequiredFreeSlots` — Required free slots; `0` means **all** slots -must be free (default). A positive integer N is accepted; without per-slot -identity data the worker fails closed to all-slots-free for `0 < N < total`; +must be free (default). A positive integer N is accepted; with per-slot +identity data the worker requires the **same N slots** continuously free (see +per-slot idle tracking below) for `0 < N < total`; without per-slot data it +fails closed to all-slots-free for `0 < N < total` (never any-N dispatch); `N > total` never dispatches. - `downtimePollIntervalMs` — Poll interval for the proxy status endpoint (default: `30000`, hard floor `10000`) @@ -227,6 +229,23 @@ Endpoint failures, timeouts, and ambiguous responses are treated as **busy** (no dispatch) and never crash the plugin. Each poll is single-flight with a per-poll timeout. +**Per-slot idle tracking** (LP-0MSG5TA7Y002GN39) — when the proxy serves +per-slot detail (`slots: [{slot_id, is_processing}]`) in the status payload +AND `downtimeRequiredFreeSlots` is `0 < N < total`, the worker tracks the +idle duration of **each slot individually** and dispatches only when the +**same N slots** have each been continuously free for the full +`downtimeIdleThresholdMs` — a slot that starts processing resets only its +*own* idle timer, so transient any-N availability never counts. Any *global* +busy condition (active query, model switch, local lease, server down, or an +ambiguous/unparseable poll) resets **all** slot timers, requiring a fresh +full idle period. This assumes `slot_id` values are stable across polls +(they identify the physical slots on the llama-server). Malformed per-slot +data (non-array, missing/empty `slot_id`, non-boolean `is_processing`, or +duplicate ids) is treated as busy (fail-closed). Without per-slot data — or +when N is `0` or ≥ `total` — the worker falls back to the count-based +all-slots-free logic: N of `total` slots free never dispatches without +per-slot identity. + **Dispatch behaviour** — once idle has been continuous for the threshold, the worker first runs `wl list --status completed --stage in_review --json` and selects the first completed/in_review item **without** a valid audit that was From 34cbfa8fb98b35b52cac8069dbf141749bc30cb5 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 19:50:09 +0100 Subject: [PATCH 07/34] WL-0MSQPPKO3006RW5F: Author docs/AGENTS-INSTALL.md documenting wl init AGENTS install flow - New docs/AGENTS-INSTALL.md: install flow diagram, pointer-line semantics, O/A/M prompt + --agents-template/--workflow-inline flags, idempotence, local-vs-global relationship/precedence, global-install interaction, standalone vs global environments, drift history - Cross-linked from README.md and docs/ARCHITECTURE.md - Extended docs/tutorials/01-your-first-work-item.md AGENTS.md prompt note - Fact base verified against src/commands/init.ts and tests/cli/init.test.ts (26 tests pass) --- README.md | 1 + docs/AGENTS-INSTALL.md | 200 ++++++++++++++++++++++ docs/ARCHITECTURE.md | 1 + docs/tutorials/01-your-first-work-item.md | 2 +- 4 files changed, 203 insertions(+), 1 deletion(-) create mode 100644 docs/AGENTS-INSTALL.md diff --git a/README.md b/README.md index 5422c3174..d0073d0aa 100644 --- a/README.md +++ b/README.md @@ -114,6 +114,7 @@ You can get a lot of value from using Worklog as a memory for your agents. But y | [API.md](API.md) | REST API endpoints and usage | | [docs/FILE_PATH_CONVENTION.md](docs/FILE_PATH_CONVENTION.md) | File path convention for work item descriptions | | [docs/CROSS_PROJECT_POLLUTION_CLEANUP.md](docs/CROSS_PROJECT_POLLUTION_CLEANUP.md) | Detecting and removing cross-project worklog pollution (`wl doctor foreign-items`) | +| [docs/AGENTS-INSTALL.md](docs/AGENTS-INSTALL.md) | How `wl init` installs AGENTS.md / workflow guidance and how local rules relate to the global agent file | ### Reference diff --git a/docs/AGENTS-INSTALL.md b/docs/AGENTS-INSTALL.md new file mode 100644 index 000000000..767361497 --- /dev/null +++ b/docs/AGENTS-INSTALL.md @@ -0,0 +1,200 @@ +# AGENTS.md Install Model + +How `wl init` installs agent guidance into a project, how that local guidance +relates to the global agent file, and where duplication/conflict risk remains. + +> **Status:** documents the current behavior (verified against +> `src/commands/init.ts` and `tests/cli/init.test.ts`). +> A recommendation to change this model is tracked in +> WL-0MSKEJK4G008BMS0 and the implementation vehicle is +> WL-0MSIXMKOX0052514 (open). + +## Overview + +Running `wl init` in a fresh project performs two agent-guidance installs: + +1. **`templates/AGENTS.md` → `/AGENTS.md`** — the project-local + agent instruction file (258 lines of work-item tracking rules, CRITICAL + RULES, a CLI cheat-sheet, and architecture notes). +2. **Workflow guidance → inlined into `AGENTS.md`** — the optional workflow + template (`templates/WORKFLOW.md`) is *inlined* between + `` / `` markers inside the + project `AGENTS.md`; a standalone `WORKFLOW.md` file is **never** written + to the repository. + +Both installs are prefixed with a **pointer line** that defers to the global +`~/.pi/agent/AGENTS.md` file when one is present: + +``` +Follow the global AGENTS.md in addition to the rules below. The local rules below take priority in the event of a conflict. +``` + +## Install flow + +``` +wl init + ├─ locateAgentTemplate() + │ templates/AGENTS.md (packaged) → fallback /templates/AGENTS.md + │ + ├─ ensureAgentTemplateInstalled() + │ │ + │ ├─ AGENTS.md does NOT exist ──► write "{pointer}\n\n{template}" (always installs) + │ │ + │ └─ AGENTS.md EXISTS + │ │ + │ ├─ pointer present ──────► no-op ("pointer already present") ← idempotent + │ │ + │ └─ pointer absent + │ ├─ action=overwrite ► replace with "{pointer}\n\n{template}" + │ ├─ action=append ───► prepend "{pointer}\n\n{existing}" + │ ├─ action=skip ─────► no-op ("user chose to manage manually") + │ └─ no action ───────► interactive O/A/M prompt → same actions + │ + └─ ensureWorkflowTemplateInstalled() + └─ --workflow-inline yes (or prompt answer) + └─ insertWorkflowLoaderIntoAgents(AGENTS.md) + │ repo WORKFLOW.md preferred, else packaged template + └─ insert "…" + if not already present ← idempotent +``` + +### Pointer line + +The pointer line is the constant `WORKLOG_AGENT_POINTER_LINE` in +`src/commands/init.ts`: + +```ts +const WORKLOG_AGENT_POINTER_LINE = + 'Follow the global AGENTS.md in addition to the rules below. The local rules below take priority in the event of a conflict.'; +``` + +It is always written as the first non-empty line of the installed `AGENTS.md`. +Its semantics: + +- **Defer to the global file** — when `~/.pi/agent/AGENTS.md` exists (the + SorraAgents global install), agents should read it for core workflow + instructions. +- **Local priority on conflict** — project-local rules take precedence over + global ones when they disagree. + +The pointer is also the idempotence key: `analyzeAgentContent()` detects it as +`firstNonEmpty === WORKLOG_AGENT_POINTER_LINE`, and `ensureAgentTemplateInstalled()` +returns `skipped: true, reason: 'pointer already present'` without touching the +file — re-running `wl init` never duplicates the pointer or template. + +### O/A/M prompt + +When `AGENTS.md` exists without the pointer and no `--agents-template` flag is +given, `promptAgentTemplateAction()` asks: + +| Choice | Meaning | Effect | +|--------|---------|--------| +| **O** – Overwrite | Replace the existing AGENTS.md entirely | Destructive; no chance of conflict with existing content | +| **A** – Add pointer | Prepend the pointer line, keep existing content | Non-destructive; retains existing instructions | +| **M** – Manual | Skip; user manages AGENTS.md themselves | No-op; Worklog agent guidance is not installed | + +### CLI flags + +| Flag | Values | Behavior | +|------|--------|----------| +| `--agents-template` | `overwrite` (or `o`), `append` (or `a`), `skip` (or `m`/`manual`/`manage`) | Non-interactive action when AGENTS.md exists without the pointer | +| `--workflow-inline` | `yes`/`true`/`1`, `no`/`false`/`0` | Inline workflow template into AGENTS.md (`yes`) or not (`no`); omitted → interactive prompt | + +`--agents-template skip` is the common choice for unattended init +(see [WL-0MKVRI3580RXZ54H], the `--agents-template` flag item). + +### Idempotence + +| Scenario | Result | +|----------|--------| +| No `AGENTS.md` | Installs `{pointer}\n\n{template}` | +| `AGENTS.md` with pointer | No-op — pointer already present | +| `AGENTS.md` without pointer, `--agents-template skip` | No-op | +| `AGENTS.md` without pointer, `--agents-template overwrite` | Replaces file | +| `AGENTS.md` without pointer, `--agents-template append` | Prepends pointer, keeps content | +| `--workflow-inline yes` on re-run | No-op — markers already present | + +### WORKFLOW.md: inlining only + +`ensureWorkflowTemplateInstalled()` never writes a standalone `WORKFLOW.md` +into the project. Instead it **inlines** the workflow content (repo +`WORKFLOW.md` preferred, else packaged template) into `AGENTS.md` between +`` and `` markers, at the top of +the file. Insertion is skipped when the markers are already present. + +## Local vs global relationship + +Two sources of agent guidance exist: + +| | Local (`templates/AGENTS.md`) | Global (`AGENTS_GLOBAL.md`) | +|---|---|---| +| **Install target** | `/AGENTS.md` via `wl init` | `~/.pi/agent/AGENTS.md` via SorraAgents `scripts/install_pi.sh` | +| **Install mechanism** | Copied (with pointer prefix) | Symlinked | +| **Scope** | One project | All projects on the machine | +| **Content** | 258 lines: CRITICAL RULES, work-item types/priorities, CLI cheat-sheet, architecture notes | Full agent workflow: work-item lifecycle, workflow steps, push policy, types/priorities | +| **Drift risk** | High — duplicated content must be edited twice | Low — single source | + +### Precedence + +The pointer line (installed by `wl init`) makes the relationship explicit: + +> *"Follow the global AGENTS.md **in addition to** the rules below. The local +> rules below take priority **in the event of a conflict**."* + +So precedence is: **local project rules > global file**, with both being read. + +### Duplication and conflict risk + +The two files overlap substantially — both carry CRITICAL RULES, work-item +Types, priorities, and workflow guidance. That duplication is a real drift +risk: a change (e.g. adding an issue type) must be applied to both files. +Confirmed example: the recent `docs` issue-type addition had to be applied to +`templates/AGENTS.md` **and** `AGENTS_GLOBAL.md` (both carry +`--issue-type: … docs …` today). + +## Interaction with the global install + +- `wl init` does **not** detect whether `~/.pi/agent/AGENTS.md` exists, and + does **not** delegate to it. It unconditionally installs its own template + content (258 lines), prefixed with the pointer line. +- The pointer line is the *only* acknowledgement of the global file: it tells + agents to also read the global file, but the local template content is still + installed in full. +- There is **no delegation today** — no code path that says "if the global + file exists, skip the template and just write a reference." + +### Environments + +| Environment | What agents see | +|-------------|-----------------| +| **Standalone** (no SorraAgents global install) | Only the project `AGENTS.md` (pointer line points at a file that does not exist — harmless, but no global workflow is loaded) | +| **Global** (SorraAgents `install_pi.sh` run) | Both the global `~/.pi/agent/AGENTS.md` (symlinked from `AGENTS_GLOBAL.md`) and the project `AGENTS.md`, with local rules taking precedence | + +## Drift history + +- **`docs` issue-type change** — applied to both `templates/AGENTS.md` and + `AGENTS_GLOBAL.md` (drift risk confirmed in practice). +- **SorraAgents dedup items** (SA-0MSITKHPW002XG4G, SA-0MSIUUYRD002GC8W, + SA-0MSITKOXI007XD4N — completed): SorraAgents adopted a + "reference global instead of duplicating" model. Its own project + `AGENTS.md` now starts with a short pointer to the global file instead of + duplicating the full content. ContextHub's template has **not** followed. + +## Recommendation pointer + +The full install-model recommendation is the deliverable of the parent item +WL-0MSKEJK4G008BMS0; implementation is tracked separately in +WL-0MSIXMKOX0052514 ("Align wl init with SorraAgents install: delegate +agent-guidance/workflow setup instead of duplicating"). This document only +describes the current model. + +## Related + +- `src/commands/init.ts` — `ensureAgentTemplateInstalled`, + `promptAgentTemplateAction`, `ensureWorkflowTemplateInstalled`, + `WORKLOG_AGENT_POINTER_LINE`, `WORKLOG_AGENT_TEMPLATE_RELATIVE_PATH` +- `templates/AGENTS.md`, `templates/WORKFLOW.md` — the installed templates +- `tests/cli/init.test.ts` — behavior tests (pointer insertion, idempotence, + `--agents-template`, `--workflow-inline`) +- [docs/tutorials/01-your-first-work-item.md](tutorials/01-your-first-work-item.md) — + the init walkthrough that mentions the AGENTS.md prompt diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 1f8dc6356..abbfeec57 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -342,3 +342,4 @@ simulates a 6-pane refresh and asserts ≥60% fewer work spawns. - [Data Syncing Guide](../DATA_SYNCING.md) - [CLI Reference](../CLI.md) - [Migration Guide](./migrations.md) +- [AGENTS.md Install Model](./AGENTS-INSTALL.md) — how `wl init` installs project agent guidance and its relationship to the global agent file diff --git a/docs/tutorials/01-your-first-work-item.md b/docs/tutorials/01-your-first-work-item.md index adc0d7f53..19434eb8f 100644 --- a/docs/tutorials/01-your-first-work-item.md +++ b/docs/tutorials/01-your-first-work-item.md @@ -41,7 +41,7 @@ Navigate to the Git repository where you want to track work, then run: wl init ``` -Worklog will prompt you for a project name and an ID prefix (e.g. `WI` or `PROJ`). Accept the defaults or choose your own. When asked about `AGENTS.md`, choose the option that suits your project. +Worklog will prompt you for a project name and an ID prefix (e.g. `WI` or `PROJ`). Accept the defaults or choose your own. When asked about `AGENTS.md`, choose the option that suits your project (O = overwrite, A = add pointer, M = manual). See [AGENTS.md Install Model](../AGENTS-INSTALL.md) for what gets installed and how it relates to the global agent file. After initialization you will have a `.worklog/` directory containing configuration and a local SQLite database. This directory is typically added to `.gitignore` since data is shared via `wl sync` rather than through direct file commits. From dd72cfa869dd6bcb3eb008c068834d246f98af34 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 19:51:56 +0100 Subject: [PATCH 08/34] WL-0MSQPPOI500687BS: Evidence-based recommendation + handoff to sibling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - docs/AGENTS-INSTALL.md: replace pointer stub with full Recommendation section — adopt single-source-of-truth (reference-global) model backed by 5 evidence points from F1/F2 findings - WL-0MSIXMKOX0052514 explicitly referenced as the implementation vehicle - Scope boundary stated: no behavioral change to wl init in this item - Implementation ideas logged as comment on WL-0MSIXMKOX0052514 --- docs/AGENTS-INSTALL.md | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/docs/AGENTS-INSTALL.md b/docs/AGENTS-INSTALL.md index 767361497..ec37f3c12 100644 --- a/docs/AGENTS-INSTALL.md +++ b/docs/AGENTS-INSTALL.md @@ -180,13 +180,23 @@ Confirmed example: the recent `docs` issue-type addition had to be applied to `AGENTS.md` now starts with a short pointer to the global file instead of duplicating the full content. ContextHub's template has **not** followed. -## Recommendation pointer +## Recommendation -The full install-model recommendation is the deliverable of the parent item -WL-0MSKEJK4G008BMS0; implementation is tracked separately in -WL-0MSIXMKOX0052514 ("Align wl init with SorraAgents install: delegate -agent-guidance/workflow setup instead of duplicating"). This document only -describes the current model. +**Adopt the single-source-of-truth model: delegate agent-guidance/workflow setup to the SorraAgents global install** (the reference-global pattern), with implementation tracked in WL-0MSIXMKOX0052514. The current duplicated model should be retired. + +### Evidence + +1. **Confirmed duplication.** `templates/AGENTS.md` (258 lines) and `AGENTS_GLOBAL.md` both carry CRITICAL RULES, work-item Types, priorities, and workflow guidance. The pointer line acknowledges the global file but the full template content is still installed — the pointer defers, the content duplicates. +2. **Confirmed drift in practice.** The `docs` issue-type change had to be applied to both files. Any future guidance change carries the same two-file maintenance burden (or worse, is applied to only one, silently diverging behavior between projects and machines). +3. **SorraAgents already adopted the model.** SA-0MSITKHPW002XG4G, SA-0MSIUUYRD002GC8W, SA-0MSITKOXI007XD4N (completed) moved SorraAgents to reference-global: its project `AGENTS.md` now starts with a short pointer to `~/.pi/agent/AGENTS.md` instead of duplicating content. ContextHub's template has not followed. +4. **ContextHub's own project AGENTS.md already uses the pattern.** The installed `/AGENTS.md` starts with a "Global agent guidance" section referencing `~/.pi/agent/AGENTS.md` — the template is out of step with the repo's own practice. +5. **Delegation has no downsides for the global environment.** When the SorraAgents install is present, the global file carries the canonical workflow; the project file only needs project-specific rules plus a reference line. This matches what `wl init`'s pointer line already half-intends. + +### What the change looks like (scope boundary — NOT implemented here) + +- When the SorraAgents global install is detected (`~/.pi/agent/AGENTS.md` resolves to a SorraAgents symlink), `wl init` should emit the canonical reference structure (a `## Global agent guidance` section referencing `~/.pi/agent/AGENTS.md`, plus a `## Project-specific guidance` section), rather than installing the full 258-line template. +- Standalone environments (no global install) keep a self-contained `AGENTS.md` so Worklog remains usable without SorraAgents. +- **No behavioral change is made as part of this item (WL-0MSKEJK4G008BMS0)** — the investigation only documents and recommends. Implementation lands in WL-0MSIXMKOX0052514. ## Related From 29df7a8c5c057078fe8d9a84969e00fa1316b534 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 19:53:51 +0100 Subject: [PATCH 09/34] feat(herdr): release Local Proxy model lease when a pi session closes The Pi extension proactively releases the previous session's model lease via POST {baseUrl}/leases/release when a new session starts (/new). Herdr now does the same when a pi agent pane closes (normal exit or prefix+x). - Extract releaseLease() into the shared module @worklog/shared/lease-release so the Pi extension and Herdr never drift; the extension entry point re-exports it. Optional timeoutMs bounds the request (a hung fetch must never delay pane teardown). Empty session ids are skipped (proxy 400s). - send-to-pi.sh / open-pi-agent.sh now launch pi via run-pi-agent.sh with a deterministic --session-id (herdr---) and register EXIT/TERM/HUP/INT traps that fire release-lease-on-exit.mjs (best-effort, silently discarded failures, 5s timeout, exit status propagated). - Tests: shared module (vitest), mjs executor (vitest), bash wiring (tests/test_lease_release.sh), updated test_scripts.sh and open-pi-agent.test.ts. Docs updated (extensions README, herdr README). Refs: WL-0MSGI7UIH008USVB --- packages/herdr/README.md | 5 +- packages/herdr/shared/open-pi-agent.sh | 12 +- .../herdr/shared/release-lease-on-exit.mjs | 60 ++++ .../shared/release-lease-on-exit.test.mjs | 60 ++++ packages/herdr/shared/run-pi-agent.sh | 72 +++++ packages/herdr/shared/send-to-pi.sh | 17 +- .../herdr/shared/tests/test_lease_release.sh | 223 ++++++++++++++ packages/herdr/shared/tests/test_scripts.sh | 4 +- packages/shared/package.json | 4 + packages/shared/src/lease-release.test.ts | 291 ++++++++++++++++++ packages/shared/src/lease-release.ts | 181 +++++++++++ packages/tui/extensions/README.md | 18 +- .../extensions/Worklog/lease-release.test.ts | 112 +------ .../tui/extensions/Worklog/lease-release.ts | 142 +-------- tests/herdr/open-pi-agent.test.ts | 8 +- 15 files changed, 958 insertions(+), 251 deletions(-) create mode 100644 packages/herdr/shared/release-lease-on-exit.mjs create mode 100644 packages/herdr/shared/release-lease-on-exit.test.mjs create mode 100644 packages/herdr/shared/run-pi-agent.sh create mode 100644 packages/herdr/shared/tests/test_lease_release.sh create mode 100644 packages/shared/src/lease-release.test.ts create mode 100644 packages/shared/src/lease-release.ts diff --git a/packages/herdr/README.md b/packages/herdr/README.md index c9ba65c4e..e1e0fb850 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -720,9 +720,9 @@ packages/herdr/ │ ├── open.sh # Open the worklist pane │ ├── open-podcast-editor-tab.sh # Open the worklist pane in a tab renamed "Podcast Editing" │ ├── toggle.sh # Toggle the worklist pane -│ ├── send-to-pi.sh # Split pane to right, launch pi with agent command +│ ├── send-to-pi.sh # Split pane to right, launch pi with agent command (via run-pi-agent.sh) │ ├── run-in-pane.sh # Run a shell command visibly in a new pane (stays open for inspection) -│ └── open-pi-agent.sh # Open a fresh interactive pi agent pane +│ └── open-pi-agent.sh # Open a fresh interactive pi agent pane (via run-pi-agent.sh) └── tests/herdr/ # Test files ``` @@ -737,6 +737,7 @@ packages/herdr/ - `!!`/`!` prefixed commands (shell-executed shortcuts such as audit approve/reject, priority updates, close/delete) are run **visibly in a new herdr pane** via `scripts/run-in-pane.sh` — the wrapper keeps the pane's process alive so the pane stays open (exit status reported; dismiss with Enter or close with `prefix+x`) so the user can inspect the command output. - Everything else is written to stdout with a `CMD:` prefix for the calling framework (Herdr) to execute. - **Pi agent dispatch** — Agent commands (`/skill:*`, `/intake`, `/plan`) are intercepted by the entry point and routed to a new pi agent pane. The `send-to-pi.sh` script splits the current pane to the right, creates a new pane, runs `pi` with the command as the initial prompt, and renames the pane to "Pi Agent". Agent commands are routed before any prefix handling, so they are unaffected by `!!`/`!` processing. +- **Model lease release on pane close** — Pi agent panes launched by `send-to-pi.sh` or `open-pi-agent.sh` run pi via `shared/run-pi-agent.sh`, which gives the session a deterministic id (`pi --session-id herdr---`) and registers EXIT/TERM/HUP/INT traps. When the pi session ends — normal exit or pane close (`prefix+x`) — the wrapper runs `shared/release-lease-on-exit.mjs`, which posts to the Local Proxy's `POST {baseUrl}/leases/release` using the **same shared implementation** as the Pi extension (`@worklog/shared/lease-release`), so the proxy's dispatch lease is reclaimed promptly instead of lingering until timeout. The release is strictly best-effort: failures (unreachable proxy, missing `~/.pi/agent/models.json`, unconfigured provider) are silently discarded, a 5s request timeout bounds the pane-close path, and the wrapper always propagates pi's exit status (WL-0MSGI7UIH008USVB). - **Podcast Editing tab naming** — `herdr plugin pane open` creates tabs with generated numeric labels. The `open-podcast-editor-tab` action wraps the same pane-open command and renames the created tab to "Podcast Editing" via `herdr tab rename` (socket API, not session-state editing), so podcast production is instantly recognisable in the tab row. Each press still opens a new tab; only the label changes. - **Model selection per shortcut** — Each LLM-bound shortcut entry in `shortcuts.json` may carry an optional `model` field (a pi model pattern such as `plan`, `code`, or `author`). When the command is dispatched to the agent channel, `--model ` is forwarded to the spawned `pi` CLI (e.g. `pi --model code '/skill:implement '`), so every workflow runs on an appropriately specialised model without manual model switching. Agent-bound entries without a `model` field default to `plan`; shell (`!!`) and `/wl` filter entries never carry a model and never receive a `--model` flag. The default mapping in `src/shortcuts.json`: `/plan`, `/intake`, `/skill:audit`, `/prompt:` → `plan`; `/skill:implement` → `code`. - **Free-form prompts via `/prompt:`** — Commands starting with `/prompt:` are also routed to the agent pane, but the `/prompt:` routing prefix is stripped before `send-to-pi.sh` runs, so pi receives only the bare prompt text (e.g. `pi "What are the audit gaps reported in the most recent audit for WL-123"`). This lets a chord shortcut open a new pi instance with an arbitrary injected prompt, not just a skill/workflow invocation. The `P-p` chord opens the command input form so you can type any free-form prompt, `P-a` opens pi with `What are the audit gaps reported in the most recent audit for ` (the selected item's ID is substituted automatically), and `P-n` opens a brand-new blank session (`/prompt:` with an empty prompt — no form dialog, no injected text, and no work-item association). Edit `src/shortcuts.json` to bind your own prompt text to any free chord. diff --git a/packages/herdr/shared/open-pi-agent.sh b/packages/herdr/shared/open-pi-agent.sh index 91fc1f0e0..af69e2bae 100755 --- a/packages/herdr/shared/open-pi-agent.sh +++ b/packages/herdr/shared/open-pi-agent.sh @@ -145,7 +145,17 @@ if [ -z "$np" ]; then fi # ── Start pi interactively in the new pane ────────────────────────── -"$herdr_bin" pane run "$np" exec pi +# The pi session is launched via run-pi-agent.sh so the pane's pi session +# id is deterministic (--session-id) and the Local Proxy model lease is +# released when the session ends (normal exit or pane close). +# (WL-0MSGI7UIH008USVB) +wrapper_script="$(cd "$(dirname "$0")" && pwd)/run-pi-agent.sh" +# Session id must match pi's session-id rules (alphanumeric, '-', '_', '.'). +# Unique per launch: timestamp + launcher pid + random suffix. +lease_session_id="herdr-$(date +%s)-$$-$RANDOM" +quoted_wrapper="$(printf '%q' "$wrapper_script")" +quoted_lease_id="$(printf '%q' "$lease_session_id")" +"$herdr_bin" pane run "$np" exec bash "$quoted_wrapper" "$quoted_lease_id" # ── Rename the pane ──────────────────────────────────────────────── "$herdr_bin" pane rename "$np" "$pane_name" >/dev/null 2>&1 || true diff --git a/packages/herdr/shared/release-lease-on-exit.mjs b/packages/herdr/shared/release-lease-on-exit.mjs new file mode 100644 index 000000000..f9f9e49d1 --- /dev/null +++ b/packages/herdr/shared/release-lease-on-exit.mjs @@ -0,0 +1,60 @@ +#!/usr/bin/env node +/** + * release-lease-on-exit.mjs — release the Local Proxy model lease for a + * pi session that has ended (WL-0MSGI7UIH008USVB). + * + * Invoked by packages/herdr/shared/run-pi-agent.sh when the pi session + * running in a Herdr pane terminates (normal exit, or pane close via + * `prefix+x` → TERM/HUP). It forwards the pane's pi session id to the + * shared `releaseLease()` implementation in `@worklog/shared/lease-release` + * — the SAME code path the Pi extension uses — so the two consumers never + * drift. + * + * Best-effort / fire-and-forget by design: + * - Exit code is always 0; failures (missing models.json, network errors, + * non-2xx responses, unresolvable shared module) are silently ignored + * and never surface in the Herdr UI. + * - If the "Local Proxy" provider is not configured, no request is sent. + * + * Usage: + * node release-lease-on-exit.mjs + */ + +import { releaseLease } from '@worklog/shared/lease-release'; +import { pathToFileURL } from 'node:url'; + +/** + * Abort the release HTTP request after this long. The pane's shell waits + * for this executor before tearing down, so a hung fetch (e.g. an + * unreachable proxy) must never delay pane close (WL-0MSGI7UIH008USVB). + */ +const RELEASE_TIMEOUT_MS = 5000; + +/** + * Release the lease for the given pi session id (best-effort, never throws). + * + * @param {string|undefined} sessionId - Pi session id from the closed pane; + * empty/undefined means "nothing to release". + */ +export async function releaseLeaseOnExit(sessionId) { + if (!sessionId) { + return; + } + try { + // releaseLease never throws, but belt-and-suspenders: this must never + // propagate so the pane's shell wrapper is never blocked. + await releaseLease(sessionId, { timeoutMs: RELEASE_TIMEOUT_MS }); + } catch { + // ignored — best-effort only + } +} + +// Run only when executed directly (not when imported by tests). +const isEntryPoint = + process.argv[1] !== undefined && + pathToFileURL(process.argv[1]).href === import.meta.url; + +if (isEntryPoint) { + await releaseLeaseOnExit(process.argv[2]); + process.exit(0); +} diff --git a/packages/herdr/shared/release-lease-on-exit.test.mjs b/packages/herdr/shared/release-lease-on-exit.test.mjs new file mode 100644 index 000000000..d0ef0cb41 --- /dev/null +++ b/packages/herdr/shared/release-lease-on-exit.test.mjs @@ -0,0 +1,60 @@ +/** + * Unit tests for packages/herdr/shared/release-lease-on-exit.mjs + * (WL-0MSGI7UIH008USVB). + * + * The script is the thin executor the Herdr pane wrapper + * (run-pi-agent.sh) invokes when a pi session ends. Its only job is to + * forward the pane's session id to the shared `releaseLease()` + * implementation in `@worklog/shared/lease-release` — the same code path + * the Pi extension uses (the HTTP contract itself is covered by + * packages/shared/src/lease-release.test.ts). + */ + +import { describe, it, expect, vi, beforeEach } from 'vitest'; + +// ── Mock the shared lease-release module ─────────────────────────────── + +const mockReleaseLease = vi.fn(); +vi.mock('@worklog/shared/lease-release', () => ({ + releaseLease: mockReleaseLease, +})); + +// ── Module under test (loaded lazily) ───────────────────────────────── + +const { releaseLeaseOnExit } = await import('./release-lease-on-exit.mjs'); + +describe('release-lease-on-exit.mjs', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + it('forwards the session id to the shared releaseLease', async () => { + await releaseLeaseOnExit('herdr-1723456789-1234-5678'); + expect(mockReleaseLease).toHaveBeenCalledWith('herdr-1723456789-1234-5678', { + timeoutMs: expect.any(Number), + }); + }); + + it('passes a bounded timeout so a hung fetch never delays pane teardown', async () => { + await releaseLeaseOnExit('herdr-1723456789-1234-5678'); + const timeoutMs = mockReleaseLease.mock.calls[0][1].timeoutMs; + expect(typeof timeoutMs).toBe('number'); + expect(timeoutMs).toBeGreaterThan(0); + expect(timeoutMs).toBeLessThanOrEqual(60_000); + }); + + it('does nothing when the session id is empty', async () => { + await releaseLeaseOnExit(''); + expect(mockReleaseLease).not.toHaveBeenCalled(); + }); + + it('does nothing when the session id is undefined', async () => { + await releaseLeaseOnExit(undefined); + expect(mockReleaseLease).not.toHaveBeenCalled(); + }); + + it('swallows errors from the shared releaseLease (never throws)', async () => { + mockReleaseLease.mockRejectedValue(new Error('boom')); + await expect(releaseLeaseOnExit('s1')).resolves.toBeUndefined(); + }); +}); diff --git a/packages/herdr/shared/run-pi-agent.sh b/packages/herdr/shared/run-pi-agent.sh new file mode 100644 index 000000000..e9fd08804 --- /dev/null +++ b/packages/herdr/shared/run-pi-agent.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# run-pi-agent.sh — Run pi in a Herdr pane and release the Local Proxy +# model lease when the session ends (WL-0MSGI7UIH008USVB). +# +# Invoked by send-to-pi.sh / open-pi-agent.sh via: +# herdr pane run exec bash \ +# [--model ] [...] +# +# The first positional argument is the pi session id used for BOTH the pi +# session (forwarded as `pi --session-id `) and the lease release +# (`node release-lease-on-exit.mjs `). Because the proxy keys dispatch +# leases by the session id pi sends in its session-affinity headers, a +# deterministic id per launch lets the release match the lease exactly. +# +# Release trigger (best-effort, never blocks the pane): +# - pi exits normally → EXIT trap +# - pane closed via prefix+x → TERM/HUP/INT trap (herdr tears the pane +# down when the primary process exits) +# Failures are silent: stdout/stderr of the release are discarded and a +# missing release script or node simply skips the release. +# +# The wrapper propagates pi's exit status so pane teardown behaves exactly +# as it did when pi was the pane's direct command. + +set -uo pipefail + +lease_session_id="${1:-}" +shift || true + +model_arg="" +if [ "${1:-}" = "--model" ]; then + model_arg="$2" + shift 2 || true +fi + +# Everything remaining is the initial prompt for pi (empty for interactive). +command_args="$*" + +wrapper_dir="$(cd "$(dirname "$0")" && pwd)" +release_script="$wrapper_dir/release-lease-on-exit.mjs" + +# ── Lease release on session end ──────────────────────────────────────── +release_lease() { + if [ -n "$lease_session_id" ] && [ -f "$release_script" ]; then + node "$release_script" "$lease_session_id" >/dev/null 2>&1 || true + fi +} + +# EXIT covers normal pi exit; TERM/HUP/INT cover pane close (prefix+x). +trap release_lease EXIT TERM HUP INT + +# ── Run pi in the pane ─────────────────────────────────────────────────── +# An empty lease_session_id (manual invocation without one) means no +# deterministic session id: run pi exactly as before (no --session-id flag). +if [ -z "$lease_session_id" ]; then + if [ -n "$model_arg" ]; then + pi --model "$model_arg" ${command_args:+"$command_args"} + else + pi ${command_args:+"$command_args"} + fi + status=$? + exit "$status" +fi + +if [ -n "$model_arg" ]; then + pi --model "$model_arg" --session-id "$lease_session_id" ${command_args:+"$command_args"} +else + pi --session-id "$lease_session_id" ${command_args:+"$command_args"} +fi +status=$? + +exit "$status" diff --git a/packages/herdr/shared/send-to-pi.sh b/packages/herdr/shared/send-to-pi.sh index 46ae36a72..b2d8aeebe 100755 --- a/packages/herdr/shared/send-to-pi.sh +++ b/packages/herdr/shared/send-to-pi.sh @@ -201,13 +201,22 @@ if [ -n "$pane_id_file" ]; then fi # ── Run pi with the command in the new pane ────────────────────────── +# The pi session is launched via run-pi-agent.sh so the pane's pi session +# id is deterministic (--session-id) and the Local Proxy model lease is +# released when the session ends (normal exit or pane close). +# (WL-0MSGI7UIH008USVB) +wrapper_script="$(cd "$(dirname "$0")" && pwd)/run-pi-agent.sh" +# Session id must match pi's session-id rules (alphanumeric, '-', '_', '.'). +# Unique per launch: timestamp + launcher pid + random suffix. +lease_session_id="herdr-$(date +%s)-$$-$RANDOM" +quoted_wrapper="$(printf '%q' "$wrapper_script")" +quoted_lease_id="$(printf '%q' "$lease_session_id")" quoted_cmd="$(printf '%q' "$COMMAND")" if [ -n "$model_arg" ]; then - # Forward the model pattern so the pi CLI opens with the requested model - # (e.g. `pi --model code '/skill:implement '`). - "$herdr_bin" pane run "$np" exec pi --model "$model_arg" "$quoted_cmd" + quoted_model="$(printf '%q' "$model_arg")" + "$herdr_bin" pane run "$np" exec bash "$quoted_wrapper" "$quoted_lease_id" --model "$quoted_model" "$quoted_cmd" else - "$herdr_bin" pane run "$np" exec pi "$quoted_cmd" + "$herdr_bin" pane run "$np" exec bash "$quoted_wrapper" "$quoted_lease_id" "$quoted_cmd" fi # ── Rename the pane ──────────────────────────────────────────────── diff --git a/packages/herdr/shared/tests/test_lease_release.sh b/packages/herdr/shared/tests/test_lease_release.sh new file mode 100644 index 000000000..c65b454c0 --- /dev/null +++ b/packages/herdr/shared/tests/test_lease_release.sh @@ -0,0 +1,223 @@ +#!/usr/bin/env bash +# test_lease_release.sh — bash tests for the Herdr pi-pane lease-release +# wiring (WL-0MSGI7UIH008USVB): +# +# packages/herdr/shared/run-pi-agent.sh — in-pane wrapper +# packages/herdr/shared/release-lease-on-exit.mjs — release executor +# +# Covers: +# - wrapper runs pi with a deterministic --session-id (command + interactive) +# - wrapper forwards --model +# - wrapper invokes the release executor with the same session id on exit +# - wrapper propagates pi's exit status +# - no release when no session id is provided +# - send-to-pi.sh / open-pi-agent.sh launch pi via the wrapper with a +# valid herdr- session id (mock herdr CLI) +# +# Uses mock `pi` / `node` / `herdr` binaries on PATH — no live pi session +# and no real HTTP requests. +# +# Run from the repo root (or anywhere): +# bash packages/herdr/shared/tests/test_lease_release.sh + +set -uo pipefail + +SHARED_DIR="$(cd "$(dirname "$0")/.." && pwd)" +RUN_PI_AGENT="$SHARED_DIR/run-pi-agent.sh" +SEND_TO_PI="$SHARED_DIR/send-to-pi.sh" +OPEN_PI_AGENT="$SHARED_DIR/open-pi-agent.sh" +PASS=0 +FAIL=0 + +pass() { PASS=$((PASS+1)); echo " PASS: $1"; } +fail() { FAIL=$((FAIL+1)); echo " FAIL: $1"; } + +# Temporary sandbox for mocks and logs +SANDBOX="$(mktemp -d)" +trap 'rm -rf "$SANDBOX"' EXIT + +# ── Mock pi / node / herdr binaries ──────────────────────────────────── + +# Mock pi: records every invocation, exits with $MOCK_PI_EXIT (default 0). +MOCK_PI="$SANDBOX/bin/pi" +MOCK_NODE="$SANDBOX/bin/node" +MOCK_HERDR="$SANDBOX/bin/herdr" +mkdir -p "$SANDBOX/bin" +cat > "$MOCK_PI" <> "$SANDBOX/pi.log" +exit "\${MOCK_PI_EXIT:-0}" +MOCK +# Mock node: records every invocation (release executor only). +cat > "$MOCK_NODE" <> "$SANDBOX/node.log" +exit 0 +MOCK +# Mock herdr CLI: records invocations, answers pane current / split / run. +cat > "$MOCK_HERDR" <> "$SANDBOX/herdr.log" +case "\$1" in + pane) + case "\$2" in + current) + echo '{"pane_id":"anchor-pane-9","success":true}' + ;; + split) + echo '{"pane_id":"pane-1","success":true}' + ;; + run|rename|zoom) + exit 0 + ;; + *) + exit 0 + ;; + esac + ;; + *) + exit 0 + ;; +esac +MOCK +chmod +x "$MOCK_PI" "$MOCK_NODE" "$MOCK_HERDR" + +# Capture the real node before the mock bin is prepended to PATH. +REAL_NODE="$(command -v node)" + +export PATH="$SANDBOX/bin:$PATH" + +# ── Tests: run-pi-agent.sh (in-pane wrapper) ────────────────────────── + +echo "=== Test: wrapper runs pi with --session-id and the command ===" +rm -f "$SANDBOX/pi.log" "$SANDBOX/node.log" +bash "$RUN_PI_AGENT" "herdr-1723456789-1234-5678" "/skill:implement WL-123" >/dev/null 2>&1 +if grep -q "^pi:--session-id herdr-1723456789-1234-5678 /skill:implement WL-123$" "$SANDBOX/pi.log" 2>/dev/null; then + pass "pi invoked with --session-id and command" +else + fail "pi should be invoked with --session-id and command" + echo " pi log: $(cat "$SANDBOX/pi.log" 2>/dev/null)" +fi +if grep -q "^node:.*release-lease-on-exit.mjs herdr-1723456789-1234-5678$" "$SANDBOX/node.log" 2>/dev/null; then + pass "release executor invoked with the same session id on exit" +else + fail "release executor should be invoked with the session id on exit" + echo " node log: $(cat "$SANDBOX/node.log" 2>/dev/null)" +fi + +echo "" +echo "=== Test: wrapper runs pi interactively (no command) ===" +rm -f "$SANDBOX/pi.log" "$SANDBOX/node.log" +bash "$RUN_PI_AGENT" "herdr-1723456789-1234-5678" >/dev/null 2>&1 +if grep -q "^pi:--session-id herdr-1723456789-1234-5678$" "$SANDBOX/pi.log" 2>/dev/null; then + pass "interactive pi invoked with --session-id only" +else + fail "interactive pi should be invoked with --session-id only" + echo " pi log: $(cat "$SANDBOX/pi.log" 2>/dev/null)" +fi + +echo "" +echo "=== Test: wrapper forwards --model to pi ===" +rm -f "$SANDBOX/pi.log" "$SANDBOX/node.log" +bash "$RUN_PI_AGENT" "herdr-1723456789-1234-5678" --model "code" "/skill:implement WL-123" >/dev/null 2>&1 +if grep -q "^pi:--model code --session-id herdr-1723456789-1234-5678 /skill:implement WL-123$" "$SANDBOX/pi.log" 2>/dev/null; then + pass "--model forwarded before --session-id" +else + fail "--model should be forwarded to pi" + echo " pi log: $(cat "$SANDBOX/pi.log" 2>/dev/null)" +fi + +echo "" +echo "=== Test: wrapper propagates pi's exit status ===" +rm -f "$SANDBOX/pi.log" "$SANDBOX/node.log" +MOCK_PI_EXIT=3 bash "$RUN_PI_AGENT" "herdr-1723456789-1234-5678" "cmd" >/dev/null 2>&1 +if [ $? -eq 3 ]; then + pass "wrapper propagates pi exit status 3" +else + fail "wrapper should propagate pi exit status 3 (got $?)" +fi +if grep -q "^node:" "$SANDBOX/node.log" 2>/dev/null; then + pass "release still fires when pi exits non-zero" +else + fail "release should fire even when pi exits non-zero" +fi + +echo "" +echo "=== Test: no release invocation when session id is empty ===" +rm -f "$SANDBOX/pi.log" "$SANDBOX/node.log" +bash "$RUN_PI_AGENT" "" "cmd" >/dev/null 2>&1 +if [ ! -s "$SANDBOX/node.log" ]; then + pass "no release when session id is empty" +else + fail "release should be skipped when session id is empty" + echo " node log: $(cat "$SANDBOX/node.log" 2>/dev/null)" +fi + +echo "" +echo "=== Test: no release invocation when release script is missing ===" +rm -f "$SANDBOX/pi.log" "$SANDBOX/node.log" +mkdir -p "$SANDBOX/empty-wrapper-dir" +cat > "$SANDBOX/empty-wrapper-dir/run-pi-agent.sh" <<'WRAPPER' +#!/usr/bin/env bash +set -uo pipefail +release_script="$(cd "$(dirname "$0")" && pwd)/release-lease-on-exit.mjs" +if [ -n "${1:-}" ] && [ -f "$release_script" ]; then + node "$release_script" "$1" >/dev/null 2>&1 || true +fi +pi --session-id "${1:-}" "${2:-}" +exit $? +WRAPPER +chmod +x "$SANDBOX/empty-wrapper-dir/run-pi-agent.sh" +# Run a copy of the wrapper logic with a missing release script: +bash "$SANDBOX/empty-wrapper-dir/run-pi-agent.sh" "herdr-x" "cmd" >/dev/null 2>&1 +if [ ! -s "$SANDBOX/node.log" ]; then + pass "missing release script skips release silently" +else + fail "missing release script should skip the release" + echo " node log: $(cat "$SANDBOX/node.log" 2>/dev/null)" +fi + +# ── Tests: send-to-pi.sh / open-pi-agent.sh launch via the wrapper ───── + +echo "" +echo "=== Test: send-to-pi.sh launches pi via the wrapper with a herdr- session id ===" +rm -f "$SANDBOX/herdr.log" +HERDR_BIN_PATH="$MOCK_HERDR" HERDR_GRID_BIN="$SANDBOX/missing-grid.py" \ + bash "$SEND_TO_PI" --no-resize "hello" >/dev/null 2>&1 +if grep -q "pane run pane-1 exec bash .*run-pi-agent.sh herdr-[0-9]*-[0-9]*-[0-9]*" "$SANDBOX/herdr.log" 2>/dev/null; then + pass "send-to-pi runs pi via the lease-release wrapper with a herdr- id" +else + fail "send-to-pi should run pi via the lease-release wrapper" + echo " herdr log: $(cat "$SANDBOX/herdr.log" 2>/dev/null)" +fi + +echo "" +echo "=== Test: open-pi-agent.sh launches pi via the wrapper with a herdr- session id ===" +rm -f "$SANDBOX/herdr.log" +HERDR_BIN_PATH="$MOCK_HERDR" HERDR_GRID_BIN="$SANDBOX/missing-grid.py" \ + bash "$OPEN_PI_AGENT" --no-resize >/dev/null 2>&1 +if grep -q "pane run pane-1 exec bash .*run-pi-agent.sh herdr-[0-9]*-[0-9]*-[0-9]*$" "$SANDBOX/herdr.log" 2>/dev/null; then + pass "open-pi-agent runs pi via the lease-release wrapper with a herdr- id" +else + fail "open-pi-agent should run pi via the lease-release wrapper" + echo " herdr log: $(cat "$SANDBOX/herdr.log" 2>/dev/null)" +fi + +echo "" +echo "=== Test: release-lease-on-exit.mjs runs standalone and exits 0 ===" +rm -f "$SANDBOX/node.log" +# Run the real script with the real node (mock models.json HOME is unset, +# so the shared module simply finds no provider and no request is sent). +if HOME="$SANDBOX/nonexistent-home" "$REAL_NODE" "$SHARED_DIR/release-lease-on-exit.mjs" "herdr-1723456789-1234-5678" >/dev/null 2>&1; then + pass "release script exits 0 standalone" +else + fail "release script should exit 0 standalone (got $?)" +fi + +echo "" +echo "==========================================" +echo "PASS: $PASS FAIL: $FAIL" +if [ "$FAIL" -gt 0 ]; then + exit 1 +fi +exit 0 diff --git a/packages/herdr/shared/tests/test_scripts.sh b/packages/herdr/shared/tests/test_scripts.sh index ad971530d..931e3623a 100755 --- a/packages/herdr/shared/tests/test_scripts.sh +++ b/packages/herdr/shared/tests/test_scripts.sh @@ -237,7 +237,7 @@ fi echo "" echo "=== Test: existing options still work (--pane-name, --focus, --no-focus, --cwd, --model) ===" out="$(run_send --pane-name "My Agent" --focus --cwd /tmp/proj --model code "hello")" || true -if grep -q "pane run grid-pane-5 exec pi --model code" "$HERDR_LOG" 2>/dev/null; then +if grep -q "pane run grid-pane-5 exec bash .*run-pi-agent.sh herdr-.* --model code" "$HERDR_LOG" 2>/dev/null; then pass "--model forwarded to pi invocation" else fail "--model should be forwarded to pi" @@ -292,7 +292,7 @@ else fail "send-to-pi should tolerate an unwritable --pane-id-file path" fi -if grep -q "pane run grid-pane-5 exec pi" "$HERDR_LOG" 2>/dev/null; then +if grep -q "pane run grid-pane-5 exec bash .*run-pi-agent.sh herdr-" "$HERDR_LOG" 2>/dev/null; then pass "send-to-pi --pane-id-file still spawns pi after writing the file" else fail "send-to-pi --pane-id-file should still spawn pi" diff --git a/packages/shared/package.json b/packages/shared/package.json index ee39001e6..83e638607 100644 --- a/packages/shared/package.json +++ b/packages/shared/package.json @@ -17,6 +17,10 @@ "./worklog-paths": { "types": "./dist/worklog-paths.d.ts", "import": "./dist/worklog-paths.js" + }, + "./lease-release": { + "types": "./dist/lease-release.d.ts", + "import": "./dist/lease-release.js" } }, "files": [ diff --git a/packages/shared/src/lease-release.test.ts b/packages/shared/src/lease-release.test.ts new file mode 100644 index 000000000..7b78848de --- /dev/null +++ b/packages/shared/src/lease-release.test.ts @@ -0,0 +1,291 @@ +/** + * Unit tests for packages/shared/src/lease-release.ts — the shared Local + * Proxy model-lease release helper (WL-0MSGI7UIH008USVB). + * + * The helper is the single implementation shared by: + * - the Pi extension (packages/tui/extensions/Worklog/lease-release.ts) + * - the Herdr plugin's pane-close release executor + * (packages/herdr/shared/release-lease-on-exit.mjs) + * + * Verifies that: + * 1. releaseLease() reads ~/.pi/agent/models.json to find the Local Proxy baseUrl. + * 2. releaseLease() sends POST {baseUrl}/leases/release with correct body. + * 3. The base URL is cached at module level (no repeated filesystem reads), + * and _resetLeaseReleaseState() clears the cache. + * 4. Errors (network failures, non-2xx) are handled gracefully + * (no throw, debug log only). + * 5. Missing "Local Proxy" provider / missing models.json file: no request. + * + * Run: npx vitest run packages/shared/src/lease-release.test.ts + */ + +import { describe, it, expect, vi, beforeEach, afterEach, afterAll } from 'vitest'; + +// ── Mock node:fs/promises ──────────────────────────────────────────── + +const mockReadFile = vi.fn(); +vi.mock('node:fs/promises', () => ({ + readFile: mockReadFile, +})); + +// ── Mock global fetch ──────────────────────────────────────────────── + +const mockFetch = vi.fn(); +vi.stubGlobal('fetch', mockFetch); + +// ── Mock console.debug ─────────────────────────────────────────────── + +const mockConsoleDebug = vi.fn(); +const origConsoleDebug = console.debug; + +// ── Module under test (loaded lazily) ──────────────────────────────── + +let mod: typeof import('./lease-release.js'); + +describe('shared lease-release', () => { + beforeEach(async () => { + vi.clearAllMocks(); + process.env.HOME = '/home/test-user'; + // Patch console.debug for this test + console.debug = mockConsoleDebug; + // Re-import fresh each test so module-level state is reset + mod = await import('./lease-release.js'); + mod._resetLeaseReleaseState(); + }); + + afterEach(() => { + console.debug = origConsoleDebug; + }); + + afterAll(() => { + console.debug = origConsoleDebug; + }); + + describe('releaseLease', () => { + it('reads models.json and sends POST to correct endpoint with session ID', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + mockFetch.mockResolvedValue({ ok: true, status: 200 }); + + await mod.releaseLease('session-abc-123'); + + expect(mockReadFile).toHaveBeenCalledWith( + '/home/test-user/.pi/agent/models.json', + 'utf-8', + ); + expect(mockFetch).toHaveBeenCalledWith( + 'http://192.168.0.199:8000/v1/leases/release', + expect.objectContaining({ + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ session_id: 'session-abc-123' }), + }), + ); + }); + + it('strips a trailing slash from the baseUrl before appending the path', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1/' }, + }, + })); + mockFetch.mockResolvedValue({ ok: true, status: 200 }); + + await mod.releaseLease('s1'); + + expect(mockFetch).toHaveBeenCalledWith( + 'http://192.168.0.199:8000/v1/leases/release', + expect.any(Object), + ); + }); + + it('does not throw on network failure (fetch rejects) and logs at debug level', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + mockFetch.mockRejectedValue(new Error('Network error')); + + await expect(mod.releaseLease('session-abc-123')).resolves.toBeUndefined(); + expect(mockConsoleDebug).toHaveBeenCalled(); + }); + + it('does not throw on non-2xx response', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + mockFetch.mockResolvedValue({ ok: false, status: 500 }); + + await expect(mod.releaseLease('session-abc-123')).resolves.toBeUndefined(); + expect(mockConsoleDebug).toHaveBeenCalled(); + }); + + it('does not send request when "Local Proxy" provider is missing', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Other Provider': { baseUrl: 'http://other:8000/v1' }, + }, + })); + + await mod.releaseLease('session-abc-123'); + expect(mockFetch).not.toHaveBeenCalled(); + }); + + it('does not send request when models.json cannot be read', async () => { + mockReadFile.mockRejectedValue(new Error('ENOENT: file not found')); + + await mod.releaseLease('session-abc-123'); + expect(mockFetch).not.toHaveBeenCalled(); + expect(mockConsoleDebug).toHaveBeenCalled(); + }); + + it('does not send request when Local Proxy provider has no baseUrl', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': {}, + }, + })); + + await mod.releaseLease('session-abc-123'); + expect(mockFetch).not.toHaveBeenCalled(); + }); + + it('does not send request when models.json is not valid JSON', async () => { + mockReadFile.mockResolvedValue('not json at all'); + + await mod.releaseLease('session-abc-123'); + expect(mockFetch).not.toHaveBeenCalled(); + }); + + it('aborts the request when timeoutMs is provided and the fetch hangs', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + // Simulate a fetch that never resolves but respects the abort signal. + mockFetch.mockImplementation((_url: string, init: { signal?: AbortSignal }) => + new Promise((_resolve, reject) => { + init.signal?.addEventListener('abort', () => + reject(new DOMException('The operation was aborted', 'AbortError')), + ); + }), + ); + + await mod.releaseLease('session-abc-123', { timeoutMs: 50 }); + + expect(mockFetch).toHaveBeenCalledWith( + expect.any(String), + expect.objectContaining({ signal: expect.any(AbortSignal) }), + ); + // Failure is swallowed — debug log only, never throws. + expect(mockConsoleDebug).toHaveBeenCalled(); + }); + + it('passes the session ID as-is to the proxy', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + mockFetch.mockResolvedValue({ ok: true, status: 200 }); + + await mod.releaseLease('herdr-1723456789-1234-5678'); + + expect(mockFetch).toHaveBeenCalledWith( + expect.any(String), + expect.objectContaining({ + body: JSON.stringify({ session_id: 'herdr-1723456789-1234-5678' }), + }), + ); + }); + + it('skips the request when session_id is empty (proxy would 400)', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + + await expect(mod.releaseLease('')).resolves.toBeUndefined(); + expect(mockFetch).not.toHaveBeenCalled(); + }); + + it('skips the request when session_id is whitespace only', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + + await mod.releaseLease(' '); + expect(mockFetch).not.toHaveBeenCalled(); + }); + + it('only calls console.debug on error (not console.error or console.warn)', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + mockFetch.mockRejectedValue(new Error('Network error')); + const mockConsoleError = vi.fn(); + const origConsoleError = console.error; + console.error = mockConsoleError; + + await mod.releaseLease('session-abc-123'); + + expect(mockConsoleDebug).toHaveBeenCalled(); + expect(mockConsoleError).not.toHaveBeenCalled(); + console.error = origConsoleError; + }); + }); + + describe('module-level baseUrl cache', () => { + it('reuses the cached baseUrl on subsequent calls (single file read)', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + mockFetch.mockResolvedValue({ ok: true, status: 200 }); + + await mod.releaseLease('s1'); + await mod.releaseLease('s2'); + + expect(mockReadFile).toHaveBeenCalledTimes(1); + expect(mockFetch).toHaveBeenCalledTimes(2); + }); + + it('re-reads models.json after _resetLeaseReleaseState() clears the cache', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ + providers: { + 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, + }, + })); + mockFetch.mockResolvedValue({ ok: true, status: 200 }); + + await mod.releaseLease('s1'); + mod._resetLeaseReleaseState(); + await mod.releaseLease('s2'); + + expect(mockReadFile).toHaveBeenCalledTimes(2); + }); + + it('caches a negative result (no provider) without re-reading', async () => { + mockReadFile.mockResolvedValue(JSON.stringify({ providers: {} })); + + await mod.releaseLease('s1'); + await mod.releaseLease('s2'); + + expect(mockReadFile).toHaveBeenCalledTimes(1); + expect(mockFetch).not.toHaveBeenCalled(); + }); + }); +}); diff --git a/packages/shared/src/lease-release.ts b/packages/shared/src/lease-release.ts new file mode 100644 index 000000000..c548f11b5 --- /dev/null +++ b/packages/shared/src/lease-release.ts @@ -0,0 +1,181 @@ +/** + * Shared Local Proxy model-lease release helper (WL-0MSGI7UIH008USVB). + * + * Canonical implementation of the best-effort lease release used by both: + * - the Pi extension (packages/tui/extensions/Worklog/lease-release.ts) + * - the Herdr plugin's pane-close release executor + * (packages/herdr/shared/release-lease-on-exit.mjs) + * + * Reads the Local Proxy provider configuration from `~/.pi/agent/models.json` + * and sends a best-effort HTTP POST to `{baseUrl}/leases/release` with the + * session identifier of the pi session whose model lease should be released. + * + * The call is fire-and-forget: + * - It never throws and never blocks the caller. + * - Failures (network errors, non-2xx responses, unreadable config) are + * silently logged at debug level only — no user-visible errors. + * - If the "Local Proxy" provider is not configured, no request is sent. + */ + +import { readFile } from 'node:fs/promises'; +import { homedir } from 'node:os'; +import { join } from 'node:path'; + +// ── Constants ──────────────────────────────────────────────────────────── + +/** Path to Pi's model/provider configuration file. */ +const MODELS_JSON_PATH = join(homedir(), '.pi', 'agent', 'models.json'); + +/** The name of the Local Proxy provider in models.json. */ +const LOCAL_PROVIDER_NAME = 'Local Proxy'; + +/** The lease release API endpoint path (appended to the provider's baseUrl). */ +const LEASE_RELEASE_PATH = '/leases/release'; + +// ── Module-level cache ─────────────────────────────────────────────────── + +/** + * Cached base URL for the Local Proxy provider. + * Set once on first successful read (or a negative `null` result), or + * cleared to force re-read. Module-level cache reduces filesystem reads + * during repeated calls within the same process lifecycle. + */ +let _cachedBaseUrl: string | null | undefined = undefined; + +/** + * Reset the cached base URL (for testing). + * + * @internal Used by tests to ensure isolation between test cases. + */ +export function _resetLeaseReleaseState(): void { + _cachedBaseUrl = undefined; +} + +// ── Internal helpers ───────────────────────────────────────────────────── + +/** + * Read the Local Proxy base URL from Pi's models.json. + * + * Reads `~/.pi/agent/models.json`, parses the provider entries, and + * extracts the `baseUrl` of the `"Local Proxy"` provider. + * + * Returns `null` if: + * - The file does not exist or cannot be read. + * - The file is not valid JSON. + * - The `"Local Proxy"` provider is not found. + * - The found provider has no `baseUrl` field. + * + * @returns The base URL string, or `null` if unavailable. + */ +async function readLocalProxyBaseUrl(): Promise { + // Return cached value if previously resolved (including negative results) + if (_cachedBaseUrl !== undefined) { + return _cachedBaseUrl; + } + + try { + const content = await readFile(MODELS_JSON_PATH, 'utf-8'); + const config = JSON.parse(content); + + const provider = config?.providers?.[LOCAL_PROVIDER_NAME]; + const baseUrl = provider?.baseUrl; + + if (!baseUrl || typeof baseUrl !== 'string') { + console.debug( + `[lease-release] Local Proxy provider "${LOCAL_PROVIDER_NAME}" has no baseUrl in ${MODELS_JSON_PATH}; skipping lease release`, + ); + _cachedBaseUrl = null; + return null; + } + + _cachedBaseUrl = baseUrl; + return baseUrl; + } catch (err: unknown) { + const message = err instanceof Error ? err.message : String(err); + console.debug( + `[lease-release] Could not read proxy config from ${MODELS_JSON_PATH}: ${message}`, + ); + _cachedBaseUrl = null; + return null; + } +} + +// ── Public API ─────────────────────────────────────────────────────────── + +/** + * Options for {@link releaseLease}. + */ +export interface ReleaseLeaseOptions { + /** + * Abort the HTTP request after this many milliseconds. The Pi extension + * path (fire-and-forget inside a running pi process) omits it; the Herdr + * pane-close executor passes a modest value so a hung fetch (unreachable + * proxy) never delays pane teardown. + */ + timeoutMs?: number; +} + +/** + * Proactively release the model lease for a pi session. + * + * Reads the Local Proxy base URL from `~/.pi/agent/models.json` and sends + * a best-effort `POST {baseUrl}/leases/release` with the session identifier. + * + * The call is fire-and-forget: + * - Never throws; failures are logged at debug level only. + * - If the Local Proxy is not configured, no request is sent. + * - If the models.json file cannot be read, no request is sent. + * - If `timeoutMs` is provided, the request aborts after that duration + * (logged at debug level, no throw). + * + * @param sessionId - The pi session identifier whose lease should be + * released (the value pi sends in its session-affinity headers — the + * same value the proxy keys the dispatch lease with). Passed as-is to + * the proxy. + * @param options - Optional release options (e.g. a request timeout). + */ +export async function releaseLease( + sessionId: string, + options: ReleaseLeaseOptions = {}, +): Promise { + // The proxy rejects empty/missing session ids with 400 (LP-0MRFOF7XO003T7CT); + // skip the pointless request instead of firing a guaranteed-failing POST. + if (!sessionId || !sessionId.trim()) { + console.debug('[lease-release] Empty session id; skipping lease release'); + return; + } + + const baseUrl = await readLocalProxyBaseUrl(); + + if (!baseUrl) { + return; + } + + const url = `${baseUrl.replace(/\/+$/, '')}${LEASE_RELEASE_PATH}`; + const body = JSON.stringify({ session_id: sessionId }); + + const signal = + options.timeoutMs !== undefined + ? AbortSignal.timeout(options.timeoutMs) + : undefined; + + try { + const response = await fetch(url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body, + ...(signal !== undefined ? { signal } : {}), + }); + + if (!response.ok) { + console.debug( + `[lease-release] Lease release request to ${url} returned status ${response.status}`, + ); + } + } catch (err: unknown) { + const message = err instanceof Error ? err.message : String(err); + console.debug( + `[lease-release] Lease release request to ${url} failed: ${message}`, + ); + } +} diff --git a/packages/tui/extensions/README.md b/packages/tui/extensions/README.md index 177d5daa5..5e4abec0c 100644 --- a/packages/tui/extensions/README.md +++ b/packages/tui/extensions/README.md @@ -275,6 +275,11 @@ The extension includes a **proactive lease release** module that automatically releases the previous session's model lease when a new Pi session is created (via `/new`). This speeds up model reclamation on the Local Proxy provider. +The release logic lives in the **shared module** `@worklog/shared/lease-release` +(`packages/shared/src/lease-release.ts`) so the Pi extension and the Herdr +plugin's pane-close release executor share a single implementation and never +drift (WL-0MSGI7UIH008USVB). + ### How It Works 1. When Pi fires a `session_start` event with reason `"new"` (indicating a @@ -299,8 +304,13 @@ releases the previous session's model lease when a new Pi session is created ### Technical Notes -- Implemented in `Worklog/lease-release.ts`. -- The proxy configuration is read at runtime from `~/.pi/agent/models.json`. -- Results are cached per-extension-lifecycle to avoid repeated filesystem reads. +- The release logic is implemented in the shared module + `packages/shared/src/lease-release.ts` and re-exported by + `Worklog/lease-release.ts`. +- The proxy configuration is read at runtime from `~/.pi/agent/models.json` + and cached at module level (a single filesystem read per process). - Registered in `Worklog/index.ts` via `registerLeaseRelease(pi)`. -- Tests are in `Worklog/lease-release.test.ts`. +- Tests are in `Worklog/lease-release.test.ts` (extension wiring) and + `packages/shared/src/lease-release.test.ts` (shared HTTP behavior). +- The Herdr plugin runs the same release on pi-pane close — see + `packages/herdr/README.md` ("Pi agent dispatch"). diff --git a/packages/tui/extensions/Worklog/lease-release.test.ts b/packages/tui/extensions/Worklog/lease-release.test.ts index eced3d461..5ca92d7f0 100644 --- a/packages/tui/extensions/Worklog/lease-release.test.ts +++ b/packages/tui/extensions/Worklog/lease-release.test.ts @@ -57,9 +57,15 @@ describe('lease-release', () => { }); // ── releaseLease() function tests ───────────────────────────────── + // + // The HTTP behavior is implemented in (and fully covered by) the shared + // module `@worklog/shared/lease-release` (packages/shared/src/lease- + // release.test.ts). This extension entry point merely re-exports it, so + // a single smoke test verifies the re-export path stays wired and the + // remaining tests focus on the extension-specific registerLeaseRelease(). describe('releaseLease', () => { - it('reads models.json and sends POST to correct endpoint with session ID', async () => { + it('re-exports the shared releaseLease (reads models.json, POSTs to the proxy)', async () => { mockReadFile.mockResolvedValue(JSON.stringify({ providers: { 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, @@ -82,110 +88,6 @@ describe('lease-release', () => { }), ); }); - - it('does not throw on network failure (fetch rejects)', async () => { - mockReadFile.mockResolvedValue(JSON.stringify({ - providers: { - 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, - }, - })); - mockFetch.mockRejectedValue(new Error('Network error')); - - await expect(mod.releaseLease('session-abc-123')).resolves.toBeUndefined(); - expect(mockConsoleDebug).toHaveBeenCalled(); - }); - - it('does not throw on non-2xx response', async () => { - mockReadFile.mockResolvedValue(JSON.stringify({ - providers: { - 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, - }, - })); - mockFetch.mockResolvedValue({ ok: false, status: 500 }); - - await expect(mod.releaseLease('session-abc-123')).resolves.toBeUndefined(); - }); - - it('does not send request when "Local Proxy" provider is missing', async () => { - mockReadFile.mockResolvedValue(JSON.stringify({ - providers: { - 'Other Provider': { baseUrl: 'http://other:8000/v1' }, - }, - })); - - await mod.releaseLease('session-abc-123'); - expect(mockFetch).not.toHaveBeenCalled(); - }); - - it('does not send request when models.json cannot be read', async () => { - mockReadFile.mockRejectedValue(new Error('ENOENT: file not found')); - - await mod.releaseLease('session-abc-123'); - expect(mockFetch).not.toHaveBeenCalled(); - }); - - it('does not send request when Local Proxy provider has no baseUrl', async () => { - mockReadFile.mockResolvedValue(JSON.stringify({ - providers: { - 'Local Proxy': {}, - }, - })); - - await mod.releaseLease('session-abc-123'); - expect(mockFetch).not.toHaveBeenCalled(); - }); - - it('passes the session ID as-is to the proxy', async () => { - mockReadFile.mockResolvedValue(JSON.stringify({ - providers: { - 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, - }, - })); - mockFetch.mockResolvedValue({ ok: true, status: 200 }); - - await mod.releaseLease('/home/test-user/.pi/sessions/session-xyz.json'); - - expect(mockFetch).toHaveBeenCalledWith( - expect.any(String), - expect.objectContaining({ - body: JSON.stringify({ session_id: '/home/test-user/.pi/sessions/session-xyz.json' }), - }), - ); - }); - - it('still sends request when session_id is empty string (proxy handles validation)', async () => { - mockReadFile.mockResolvedValue(JSON.stringify({ - providers: { - 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, - }, - })); - mockFetch.mockResolvedValue({ ok: true, status: 200 }); - - await expect(mod.releaseLease('')).resolves.toBeUndefined(); - expect(mockFetch).toHaveBeenCalled(); - }); - - it('only calls console.debug on error (not console.error or console.warn)', async () => { - mockReadFile.mockResolvedValue(JSON.stringify({ - providers: { - 'Local Proxy': { baseUrl: 'http://192.168.0.199:8000/v1' }, - }, - })); - mockFetch.mockRejectedValue(new Error('Network error')); - - await mod.releaseLease('session-abc-123'); - - expect(mockConsoleDebug).toHaveBeenCalled(); - }); - - it('survives missing Local Proxy in models.json (no crash)', async () => { - mockReadFile.mockResolvedValue(JSON.stringify({ - providers: {}, - })); - - await expect(mod.releaseLease('session-abc-123')).resolves.toBeUndefined(); - expect(mockFetch).not.toHaveBeenCalled(); - }); }); // ── registerLeaseRelease() integration tests ────────────────────── diff --git a/packages/tui/extensions/Worklog/lease-release.ts b/packages/tui/extensions/Worklog/lease-release.ts index 1127c0e19..fd0955055 100644 --- a/packages/tui/extensions/Worklog/lease-release.ts +++ b/packages/tui/extensions/Worklog/lease-release.ts @@ -2,9 +2,12 @@ * Lease release module for the Worklog Pi extension. * * Proactively releases the previous session's model lease when a new Pi - * session is created (via `/new`). Reads the Local Proxy provider - * configuration from `~/.pi/agent/models.json` and sends a best-effort - * HTTP POST to `{baseUrl}/leases/release`. + * session is created (via `/new`). The release logic itself (reading the + * Local Proxy provider configuration from `~/.pi/agent/models.json` and + * sending a best-effort HTTP POST to `{baseUrl}/leases/release`) lives in + * the shared module `@worklog/shared/lease-release` (WL-0MSGI7UIH008USVB) + * so the Pi extension and the Herdr plugin's pane-close release executor + * never drift. * * The call is fire-and-forget: * - It does not block or delay session startup. @@ -13,135 +16,12 @@ * - If the "Local Proxy" provider is not configured, no request is sent. */ -import { readFile } from 'node:fs/promises'; -import { homedir } from 'node:os'; -import { join } from 'node:path'; +import { releaseLease, _resetLeaseReleaseState } from '@worklog/shared/lease-release'; -// ── Constants ──────────────────────────────────────────────────────────── - -/** Path to Pi's model/provider configuration file. */ -const MODELS_JSON_PATH = join(homedir(), '.pi', 'agent', 'models.json'); - -/** The name of the Local Proxy provider in models.json. */ -const LOCAL_PROVIDER_NAME = 'Local Proxy'; - -/** The lease release API endpoint path (appended to the provider's baseUrl). */ -const LEASE_RELEASE_PATH = '/leases/release'; - -// ── Module-level cache ─────────────────────────────────────────────────── - -/** - * Cached base URL for the Local Proxy provider. - * Set once on first successful read, or cleared to force re-read. - * Module-level cache reduces filesystem reads during repeated calls - * within the same extension lifecycle. - */ -let _cachedBaseUrl: string | null | undefined = undefined; - -/** - * Reset the cached base URL (for testing). - * - * @internal Used by tests to ensure isolation between test cases. - */ -export function _resetLeaseReleaseState(): void { - _cachedBaseUrl = undefined; -} - -// ── Internal helpers ───────────────────────────────────────────────────── - -/** - * Read the Local Proxy base URL from Pi's models.json. - * - * Reads `~/.pi/agent/models.json`, parses the provider entries, and - * extracts the `baseUrl` of the `"Local Proxy"` provider. - * - * Returns `null` if: - * - The file does not exist or cannot be read. - * - The file is not valid JSON. - * - The `"Local Proxy"` provider is not found. - * - The found provider has no `baseUrl` field. - * - * @returns The base URL string, or `null` if unavailable. - */ -async function readLocalProxyBaseUrl(): Promise { - // Return cached value if previously resolved - if (_cachedBaseUrl !== undefined) { - return _cachedBaseUrl; - } - - try { - const content = await readFile(MODELS_JSON_PATH, 'utf-8'); - const config = JSON.parse(content); - - const provider = config?.providers?.[LOCAL_PROVIDER_NAME]; - const baseUrl = provider?.baseUrl; - - if (!baseUrl || typeof baseUrl !== 'string') { - console.debug( - `[lease-release] Local Proxy provider "${LOCAL_PROVIDER_NAME}" has no baseUrl in ${MODELS_JSON_PATH}; skipping lease release`, - ); - _cachedBaseUrl = null; - return null; - } - - _cachedBaseUrl = baseUrl; - return baseUrl; - } catch (err: unknown) { - const message = err instanceof Error ? err.message : String(err); - console.debug( - `[lease-release] Could not read proxy config from ${MODELS_JSON_PATH}: ${message}`, - ); - _cachedBaseUrl = null; - return null; - } -} - -// ── Public API ─────────────────────────────────────────────────────────── - -/** - * Proactively release the model lease for a previous session. - * - * Reads the Local Proxy base URL from `~/.pi/agent/models.json` and sends - * a best-effort `POST {baseUrl}/leases/release` with the session identifier. - * - * The call is fire-and-forget: - * - Failures are logged at debug level only (no user-visible errors). - * - If the Local Proxy is not configured, no request is sent. - * - If the models.json file cannot be read, no request is sent. - * - * @param previousSessionId - The session identifier from the previous - * session (value of `previousSessionFile` from the session_start event). - * Passed as-is to the proxy. - */ -export async function releaseLease(previousSessionId: string): Promise { - const baseUrl = await readLocalProxyBaseUrl(); - - if (!baseUrl) { - return; - } - - const url = `${baseUrl.replace(/\/+$/, '')}${LEASE_RELEASE_PATH}`; - const body = JSON.stringify({ session_id: previousSessionId }); - - try { - const response = await fetch(url, { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body, - }); - - if (!response.ok) { - console.debug( - `[lease-release] Lease release request to ${url} returned status ${response.status}`, - ); - } - } catch (err: unknown) { - const message = err instanceof Error ? err.message : String(err); - console.debug( - `[lease-release] Lease release request to ${url} failed: ${message}`, - ); - } -} +// Re-export the shared implementation so existing consumers (and the unit +// tests in this directory) keep importing this module as the single entry +// point for the extension's lease-release behavior. +export { releaseLease, _resetLeaseReleaseState }; /** * Register the lease release session_start handler with a Pi extension instance. diff --git a/tests/herdr/open-pi-agent.test.ts b/tests/herdr/open-pi-agent.test.ts index 08ab11b03..8e74a4c51 100644 --- a/tests/herdr/open-pi-agent.test.ts +++ b/tests/herdr/open-pi-agent.test.ts @@ -150,9 +150,13 @@ describe('open-pi-agent.sh --cwd propagation', () => { expect(split).toContain(cwd); }); - it('starts pi interactively in the new pane', () => { + it('starts pi interactively in the new pane via the lease-release wrapper', () => { const { status, log } = runScript(['--no-resize', '--cwd', '/tmp/project-root']); expect(status).toBe(0); - expect(log.some((line) => line.includes('pane run') && line.includes('pi'))).toBe(true); + // The pane runs the lease-release wrapper (run-pi-agent.sh) with a + // deterministic --session-id so the Local Proxy lease can be released + // when the interactive pi session ends (WL-0MSGI7UIH008USVB). + expect(log.some((line) => line.includes('pane run') && line.includes('run-pi-agent.sh'))).toBe(true); + expect(log.some((line) => line.includes('pane run') && /herdr-\d+-\d+-\d+/.test(line))).toBe(true); }); }); From 79abe8e9df565a90816f1887dfb580822a89e674 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 20:12:06 +0100 Subject: [PATCH 10/34] =?UTF-8?q?WL-0MSQPPRXR00809LM:=20Docs=20sweep=20?= =?UTF-8?q?=E2=80=94=20fix=20CLI.md=20init=20description=20to=20match=20ve?= =?UTF-8?q?rified=20behavior?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - CLI.md: corrected the init command description — O/A/M prompt does not always preserve content (O = destructive overwrite); no prompt when the pointer is already present (idempotent). Added link to docs/AGENTS-INSTALL.md. - Verified: no contradictory code comments in src/commands/init.ts - tutorial/README/ARCHITECTURE cross-links already in place from F2 --- CLI.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CLI.md b/CLI.md index 8e571e58d..c67b8e341 100644 --- a/CLI.md +++ b/CLI.md @@ -1060,7 +1060,7 @@ Other commands cover repository bootstrap and local system status. Use these to ### `init` -Initialize Worklog configuration in the repository (creates `.worklog` and default config). `wl init` also installs `AGENTS.md` in the project root with a pointer line to the global `AGENTS.md`. If `AGENTS.md` already exists, it prompts before inserting the pointer and preserves the existing content (unless you pass `--agents-template` for unattended runs). When workflow templates are available, `wl init` prompts you to choose between no formal workflow, a basic Worklog-aware workflow, or manual management (unless you pass `--workflow-inline` for unattended runs). +Initialize Worklog configuration in the repository (creates `.worklog` and default config). `wl init` also installs `AGENTS.md` in the project root, prefixed with a pointer line to the global `AGENTS.md`. If `AGENTS.md` already contains the pointer line, installation is skipped (idempotent, no prompt). If `AGENTS.md` exists without the pointer, it prompts O/A/M — **O**verwrite (destructive), **A**dd pointer (keeps existing content), **M**anual (skip) — unless you pass `--agents-template` for unattended runs. When workflow templates are available, `wl init` prompts you to choose between no formal workflow, a basic Worklog-aware workflow, or manual management (unless you pass `--workflow-inline` for unattended runs). See [AGENTS.md Install Model](docs/AGENTS-INSTALL.md) for the full install flow. Options: From fa0d601b833704bbcf9acd0cf9a0ffe21ad25876 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 22:51:38 +0100 Subject: [PATCH 11/34] WL-0MSMGGPWR000MZ0O: Auto-continue after mid-session compaction (session_compact event handling) - register-recovery.ts: register pi.on('session_compact') handler; classify mid-session vs end-of-session compaction; reuse the invisible-continue loop (agent.prompt([])) via triggerCompactionContinue() with abort/mutex/ continuation-in-flight guards. - recovery.ts: add pure shouldAutoContinueAfterCompaction() (skips willRetry:true overflow, detects pending messages / length|error stop / agent-running) and shouldTriggerCompactionContinue() guard checks. - session-compact.test.ts: 14 unit tests covering the classification cases and safety guards. - README.md: document the Mid-Session Compaction Auto-Continue behavior. --- packages/tui/extensions/README.md | 24 ++- .../Worklog/lib/recovery/recovery.ts | 74 ++++++++++ .../Worklog/lib/recovery/register-recovery.ts | 54 ++++++- .../lib/recovery/session-compact.test.ts | 139 ++++++++++++++++++ 4 files changed, 289 insertions(+), 2 deletions(-) create mode 100644 packages/tui/extensions/Worklog/lib/recovery/session-compact.test.ts diff --git a/packages/tui/extensions/README.md b/packages/tui/extensions/README.md index 5e4abec0c..2db01bb89 100644 --- a/packages/tui/extensions/README.md +++ b/packages/tui/extensions/README.md @@ -255,6 +255,28 @@ The module registers a `/retry` command with the following subcommands: error messages, is-retrying flags, continuation count - `/retry reset` — Resets all retry counters and state +### Mid-Session Compaction Auto-Continue + +When Pi performs a **mid-session** compaction (threshold auto-compaction while the +agent still has work in flight), the module automatically resumes the agent via +the invisible-continue loop (`agent.prompt([])`) — no manual "continue" needed. + +- **Auto-continues when**: queued messages are pending (`ctx.hasPendingMessages()`), + the last assistant turn was interrupted (`stopReason` `length`/`error`), or the + agent was still running when compaction started. +- **Does not auto-continue when**: the compaction is overflow recovery + (`willRetry: true` — Pi retries the aborted turn natively, continuing again + would double-continue) or an end-of-session compaction (agent settled, nothing + pending — e.g. manual `/compact` after a completed turn). Manual `/compact` + auto-continues only when demonstrably mid-session (pending work). +- **Safety guards**: user abort (ESC), session switches, the retry-loop mutex, and + the continuation-in-flight flag are all respected — no continuation starts when + the user has aborted, the session changed, or another continuation/retry is + already running. + +The classification lives in `shouldAutoContinueAfterCompaction()` in +`recovery.ts`; the `session_compact` handler is wired in `register-recovery.ts`. + ### Architecture The recovery module is implemented in `Worklog/lib/recovery/` and consists of: @@ -265,7 +287,7 @@ The recovery module is implemented in `Worklog/lib/recovery/` and consists of: | `retry-logic.ts` | Exponential backoff, state managers, interruptible sleep | | `recovery.ts` | Compact-and-continue and checkpoint-and-terminate handlers | | `retry-command.ts` | `/retry` command interface (status, reset, manual-trigger) | -| `register-recovery.ts` | Extension lifecycle wiring (agent_end, turn_end, session_start) | +| `register-recovery.ts` | Extension lifecycle wiring (agent_end, turn_end, session_start, session_compact) | The module is auto-registered during extension initialization in `index.ts`. diff --git a/packages/tui/extensions/Worklog/lib/recovery/recovery.ts b/packages/tui/extensions/Worklog/lib/recovery/recovery.ts index 9f584b693..eb6f212df 100644 --- a/packages/tui/extensions/Worklog/lib/recovery/recovery.ts +++ b/packages/tui/extensions/Worklog/lib/recovery/recovery.ts @@ -45,6 +45,80 @@ export function hasContextLengthStop(message: AgentMessage | unknown): boolean { return false; } +// ── Compaction auto-continue (session_compact) ───────────────────────── + +/** + * Decide whether a `session_compact` event should trigger an automatic + * continuation (mid-session compaction) or leave the agent stopped + * (end-of-session compaction). + * + * Pi natively auto-continues ONLY overflow recovery (`willRetry: true`); + * threshold compaction deliberately stops ("user continues manually"). This + * classification fills that gap without racing Pi's own flows: + * + * - `willRetry: true` (overflow) — Pi retries the aborted turn itself after + * compaction; continuing again would double-continue → never auto-continue. + * - `hasPendingMessages` — queued work was in flight when compaction ran + * (e.g. pre-prompt threshold compaction) → mid-session. + * - Last assistant `stopReason` is `length` or `error` — an interrupted + * turn (e.g. max output tokens) preceded the compaction → mid-session. + * - `!isIdle` — the agent was still streaming when compaction started + * → mid-session. + * - Otherwise (agent settled, nothing pending, last turn completed with + * `stop`) — end-of-session compaction (e.g. manual `/compact` after a + * completed turn) → no auto-continue. + * + * @param event - The `session_compact` event (`reason` and `willRetry`) + * @param signals - Live extension-context signals captured at event time + * @returns true when the agent should auto-continue after compaction + */ +export function shouldAutoContinueAfterCompaction( + event: { reason: string; willRetry: boolean }, + signals: { hasPendingMessages: boolean; isIdle: boolean; lastAssistantStopReason?: string }, +): boolean { + // Pi natively retries the aborted turn after overflow compaction — + // a second continuation would race it. + if (event.willRetry) return false; + + // Queued messages mean the agent had work in flight → mid-session. + if (signals.hasPendingMessages) return true; + + // An interrupted turn (max output tokens / error) is mid-session work. + const stopReason = signals.lastAssistantStopReason; + if (stopReason === 'length' || stopReason === 'error') return true; + + // The agent was running when compaction started → mid-session. + if (!signals.isIdle) return true; + + // Settled agent, no pending work → end-of-session, no auto-continue. + return false; +} + +/** + * Guard checks for starting a compaction auto-continue. + * + * Mirrors the safety guards already enforced inside `triggerInvisibleContinue` + * (user abort, session switches) at the compaction entry point, plus the + * retry-loop mutex and the continuation-in-flight flag so the `session_compact` + * path never races the CONTEXT_LENGTH path or Pi's own overflow retry. + * + * @param guards - Current recovery-module state + * @returns true when a compaction auto-continue may start + */ +export function shouldTriggerCompactionContinue(guards: { + /** User pressed ESC — never auto-continue. */ + userAborted: boolean; + /** The invisible-continue retry loop is already running (mutex). */ + continueInProgress: boolean; + /** A context-length continuation is already in flight (double-trigger guard). */ + continuationInFlight: boolean; +}): boolean { + if (guards.userAborted) return false; + if (guards.continueInProgress) return false; + if (guards.continuationInFlight) return false; + return true; +} + // ── Terminal error categories ───────────────────────────────────────── /** diff --git a/packages/tui/extensions/Worklog/lib/recovery/register-recovery.ts b/packages/tui/extensions/Worklog/lib/recovery/register-recovery.ts index 1eea4dbb3..b941eed1e 100644 --- a/packages/tui/extensions/Worklog/lib/recovery/register-recovery.ts +++ b/packages/tui/extensions/Worklog/lib/recovery/register-recovery.ts @@ -6,11 +6,12 @@ * - agent_end → error detection → category dispatch * - turn_end → state management (reset on success, abort flag) * - session_start → state reset + * - session_compact → auto-continue after mid-session compaction * - Built-in retry suppression (monkey-patching _prepareRetry) * - /retry command registration */ -import type { ExtensionAPI, ExtensionCommandContext } from '@earendil-works/pi-coding-agent'; +import type { ExtensionAPI, ExtensionCommandContext, SessionCompactEvent } from '@earendil-works/pi-coding-agent'; import { classifyError, ErrorCategory } from './error-patterns.js'; import { @@ -22,6 +23,10 @@ import { type RetryCommandOptions, } from './retry-command.js'; import { calculateDelay, formatDuration } from './retry-logic.js'; +import { + shouldAutoContinueAfterCompaction, + shouldTriggerCompactionContinue, +} from './recovery.js'; let _recoveryRegistered = false; @@ -148,6 +153,28 @@ export function registerRecoveryModule(pi: ExtensionAPI): void { } }); + // ── session_compact: auto-continue after mid-session compaction ── + // Pi auto-continues only overflow recovery (willRetry) natively; threshold + // compaction stops. When a compaction is demonstrably mid-session (queued + // messages pending, an interrupted turn with stopReason length/error, or the + // agent still streaming), reuse the invisible-continue loop so the agent + // resumes without the user typing "continue". + pi.on('session_compact', async (event: SessionCompactEvent, ctx) => { + _notifyFn = (message, level) => ctx.ui.notify(message, level); + + const entries = ctx.sessionManager.getEntries(); + const lastAssistant = findLastAssistantMessage(entries); + + const signals = { + hasPendingMessages: ctx.hasPendingMessages(), + isIdle: ctx.isIdle(), + lastAssistantStopReason: lastAssistant?.stopReason, + }; + + if (!shouldAutoContinueAfterCompaction(event, signals)) return; + void triggerCompactionContinue(); + }); + // ── turn_end: manage state on successful turns ───────────────── pi.on('turn_end', async (event) => { const msg = (event as any).message; @@ -346,6 +373,31 @@ async function triggerInvisibleContinue(): Promise { } } +/** + * Auto-continue the agent after a mid-session compaction. + * + * Applies the entry-point guards (user abort, retry-loop mutex, and + * continuation-in-flight so this path never double-triggers with the + * CONTEXT_LENGTH path), then fires the invisible-continue loop + * (`agent.prompt([])`) that the CONTEXT_LENGTH branch already uses. + * The loop itself re-verifies abort/session-change throughout. + */ +async function triggerCompactionContinue(): Promise { + if (!shouldTriggerCompactionContinue({ + userAborted: interruptibleState.userAborted, + continueInProgress: _continueInProgress, + continuationInFlight: continuationState.getIsContinuing() || continuationState.getCount() > 0, + })) return; + + continuationState.startContinuation(); + if (_notifyFn) { + _notifyFn('Compaction complete — continuing automatically...', 'info'); + } + continuationState.endContinuation(); + + void triggerInvisibleContinue(); +} + /** Notify user about a retry attempt */ function _notifyRetryAttempt(attempt: number, duration: string): void { if (_notifyFn) { diff --git a/packages/tui/extensions/Worklog/lib/recovery/session-compact.test.ts b/packages/tui/extensions/Worklog/lib/recovery/session-compact.test.ts new file mode 100644 index 000000000..c8e7e59fc --- /dev/null +++ b/packages/tui/extensions/Worklog/lib/recovery/session-compact.test.ts @@ -0,0 +1,139 @@ +/** + * Tests for mid-session compaction auto-continue (session_compact event). + * + * Covers: + * - shouldAutoContinueAfterCompaction — mid-session vs end-of-session + * classification, overflow (`willRetry: true`) exclusion, manual + * /compact semantics + * - shouldTriggerCompactionContinue — abort / retry-mutex / continuation + * guards respected at the compaction entry point + * + * Session-switch guards (sessionGeneration capture + interruptible sleep) + * are enforced inside the reused `triggerInvisibleContinue()` loop and are + * covered by retry-loop.test.ts (interruptibleSleep). + * + * Run: cd /home/rgardler/projects/ContextHub && npx vitest run packages/tui/extensions/Worklog/lib/recovery/session-compact.test.ts + */ + +import { describe, it, expect } from 'vitest'; +import { + shouldAutoContinueAfterCompaction, + shouldTriggerCompactionContinue, +} from './recovery.js'; + +// ── shouldAutoContinueAfterCompaction ───────────────────────────────── + +describe('shouldAutoContinueAfterCompaction', () => { + // AC2: mid-session threshold compaction → auto-continue fires + it('continues when queued messages are pending (pre-prompt threshold compaction)', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'threshold', willRetry: false }, + { hasPendingMessages: true, isIdle: true, lastAssistantStopReason: undefined }, + )).toBe(true); + }); + + it('continues when the last assistant turn hit max output tokens (stopReason "length")', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'threshold', willRetry: false }, + { hasPendingMessages: false, isIdle: true, lastAssistantStopReason: 'length' }, + )).toBe(true); + }); + + it('continues when the last assistant turn errored (stopReason "error")', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'threshold', willRetry: false }, + { hasPendingMessages: false, isIdle: true, lastAssistantStopReason: 'error' }, + )).toBe(true); + }); + + it('continues when the agent was still running when compaction started', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'threshold', willRetry: false }, + { hasPendingMessages: false, isIdle: false, lastAssistantStopReason: undefined }, + )).toBe(true); + }); + + // AC3: overflow compaction → no continuation (Pi retries the turn natively) + it('never continues on overflow recovery (willRetry: true) — even with pending work', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'overflow', willRetry: true }, + { hasPendingMessages: true, isIdle: false, lastAssistantStopReason: 'length' }, + )).toBe(false); + }); + + it('does not continue on overflow compaction without retry (completed turn)', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'overflow', willRetry: false }, + { hasPendingMessages: false, isIdle: true, lastAssistantStopReason: 'stop' }, + )).toBe(false); + }); + + // AC4: end-of-session compaction → no auto-continue + it('does not continue on end-of-session manual /compact after a completed turn', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'manual', willRetry: false }, + { hasPendingMessages: false, isIdle: true, lastAssistantStopReason: 'stop' }, + )).toBe(false); + }); + + it('does not continue when the agent is settled with no assistant message', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'manual', willRetry: false }, + { hasPendingMessages: false, isIdle: true, lastAssistantStopReason: undefined }, + )).toBe(false); + }); + + it('does not continue after an aborted turn (user-initiated stop)', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'manual', willRetry: false }, + { hasPendingMessages: false, isIdle: true, lastAssistantStopReason: 'aborted' }, + )).toBe(false); + }); + + // Documented working assumption: manual /compact mid-session with pending work + it('continues on manual /compact when demonstrably mid-session (pending work)', () => { + expect(shouldAutoContinueAfterCompaction( + { reason: 'manual', willRetry: false }, + { hasPendingMessages: true, isIdle: true, lastAssistantStopReason: 'stop' }, + )).toBe(true); + }); +}); + +// ── shouldTriggerCompactionContinue (guards) ────────────────────────── + +describe('shouldTriggerCompactionContinue', () => { + // AC5: user abort (ESC) respected + it('does not continue when the user has aborted', () => { + expect(shouldTriggerCompactionContinue({ + userAborted: true, + continueInProgress: false, + continuationInFlight: false, + })).toBe(false); + }); + + // AC5: retry-loop mutex respected — no concurrent loop + it('does not continue when a retry/continue loop is already in progress', () => { + expect(shouldTriggerCompactionContinue({ + userAborted: false, + continueInProgress: true, + continuationInFlight: false, + })).toBe(false); + }); + + // AC5: no double-trigger with the CONTEXT_LENGTH path + it('does not continue when a continuation is already in flight', () => { + expect(shouldTriggerCompactionContinue({ + userAborted: false, + continueInProgress: false, + continuationInFlight: true, + })).toBe(false); + }); + + it('continues when all guards are clear', () => { + expect(shouldTriggerCompactionContinue({ + userAborted: false, + continueInProgress: false, + continuationInFlight: false, + })).toBe(true); + }); +}); From 034abd649ebfd88c652226d3203cb690ed236734 Mon Sep 17 00:00:00 2001 From: pi Date: Thu, 13 Aug 2026 23:37:09 +0100 Subject: [PATCH 12/34] WL-0MSNKWLER003EGE0: Auto-recover from JSON parse errors in audit processing Add a dedicated PARSE_ERROR error category to the Worklog extension's recovery module so JSON parse failures (V8/Node JSON.parse, Python JSONDecodeError, generic invalid-JSON mentions) no longer halt sessions via UNKNOWN classification. - error-patterns.ts: PARSE_ERROR enum member + default patterns for all three families, DEFAULT_RECOVERY_CONFIG.parseError entry (enabled, no backoff), isParseError() classifier wired into classifyError() - register-recovery.ts: agent_end dispatches PARSE_ERROR to a single-shot continue (wait-for-idle -> prompt(['continue'])); no exponential backoff per operator decision; error message stays in agent state - recovery.ts: executeParseErrorContinue() pure-logic harness - retry-command.ts: parseError retry state + /retry manual continue - tests: classification (3 families + no false positives), settings 7->8 categories, single-shot continue + repeated-error behavior, /retry - docs: extension README error-category table and architecture notes --- packages/tui/extensions/README.md | 7 +- .../lib/recovery/error-classification.test.ts | 58 +++++++ .../Worklog/lib/recovery/error-patterns.ts | 44 +++++- .../lib/recovery/parse-error-continue.test.ts | 147 ++++++++++++++++++ .../Worklog/lib/recovery/recovery.ts | 66 ++++++++ .../Worklog/lib/recovery/register-recovery.ts | 66 +++++++- .../lib/recovery/retry-command.test.ts | 31 ++++ .../Worklog/lib/recovery/retry-command.ts | 8 + .../Worklog/lib/recovery/settings.test.ts | 23 ++- 9 files changed, 443 insertions(+), 7 deletions(-) create mode 100644 packages/tui/extensions/Worklog/lib/recovery/parse-error-continue.test.ts diff --git a/packages/tui/extensions/README.md b/packages/tui/extensions/README.md index 2db01bb89..4178227d7 100644 --- a/packages/tui/extensions/README.md +++ b/packages/tui/extensions/README.md @@ -221,6 +221,7 @@ configuration. | `quotaExhausted` | NOT retried | Checkpoint saved + terminal error displayed | | `timeout` | Retried | Exponential backoff with configurable delay | | `terminated` | NOT retried | Checkpoint saved + terminal error displayed | +| `parseError` (JSON) | Single-shot continue | One plain "continue" prompt, no backoff loop | ### Configuration @@ -250,7 +251,7 @@ Recovery behaviour is driven by `DEFAULT_RECOVERY_CONFIG` in The module registers a `/retry` command with the following subcommands: - `/retry` — Manual trigger: auto-detects the last error and applies the correct - recovery strategy (retry, compact+continue, or warning) + recovery strategy (retry, compact+continue, single-shot continue, or warning) - `/retry status` — Displays diagnostics: per-category attempt counts, last error messages, is-retrying flags, continuation count - `/retry reset` — Resets all retry counters and state @@ -283,9 +284,9 @@ The recovery module is implemented in `Worklog/lib/recovery/` and consists of: | File | Purpose | |------|---------| -| `error-patterns.ts` | Error classification patterns for all 7 categories | +| `error-patterns.ts` | Error classification patterns for all 8 categories | | `retry-logic.ts` | Exponential backoff, state managers, interruptible sleep | -| `recovery.ts` | Compact-and-continue and checkpoint-and-terminate handlers | +| `recovery.ts` | Compact-and-continue, checkpoint-and-terminate, and single-shot parse-error continue handlers | | `retry-command.ts` | `/retry` command interface (status, reset, manual-trigger) | | `register-recovery.ts` | Extension lifecycle wiring (agent_end, turn_end, session_start, session_compact) | diff --git a/packages/tui/extensions/Worklog/lib/recovery/error-classification.test.ts b/packages/tui/extensions/Worklog/lib/recovery/error-classification.test.ts index 00e75d2c1..7a900276e 100644 --- a/packages/tui/extensions/Worklog/lib/recovery/error-classification.test.ts +++ b/packages/tui/extensions/Worklog/lib/recovery/error-classification.test.ts @@ -17,6 +17,7 @@ import { isQuotaExhausted, isTimeout, isTerminated, + isParseError, ErrorCategory, type RecoveryConfig, } from './error-patterns.js'; @@ -224,6 +225,53 @@ describe('isTerminated', () => { }); }); +// ── Parse Error (JSON) ──────────────────────────────────────────────── + +describe('isParseError', () => { + it('detects V8/Node JSON.parse messages', () => { + expect(isParseError(makeErrorMsg('Expected \'"\' or \'}\' after property value in JSON at position 123'))).toBe(true); + expect(isParseError(makeErrorMsg('Unexpected end of JSON input'))).toBe(true); + expect(isParseError(makeErrorMsg("Unexpected token '}' ... is not valid JSON"))).toBe(true); + expect(isParseError(makeErrorMsg('Unexpected number in JSON'))).toBe(true); + expect(isParseError(makeErrorMsg('Unexpected string in JSON at position 42'))).toBe(true); + expect(isParseError(makeErrorMsg('Unexpected non-whitespace character after JSON at position 7'))).toBe(true); + expect(isParseError(makeErrorMsg('Unterminated string in JSON at position 10'))).toBe(true); + expect(isParseError(makeErrorMsg('Bad control character in string literal in JSON'))).toBe(true); + }); + + it('detects Python JSONDecodeError texts', () => { + expect(isParseError(makeErrorMsg('Expecting value: line 1 column 123 (char 122)'))).toBe(true); + expect(isParseError(makeErrorMsg("Expecting ',' delimiter: line 1 column 5 (char 4)"))).toBe(true); + expect(isParseError(makeErrorMsg("Expecting ':' delimiter: line 1 column 9 (char 8)"))).toBe(true); + expect(isParseError(makeErrorMsg('Expecting property name enclosed in double quotes: line 2'))).toBe(true); + expect(isParseError(makeErrorMsg('Unterminated string starting at: line 1 column 3 (char 2)'))).toBe(true); + expect(isParseError(makeErrorMsg('Invalid control character at: line 1 column 3 (char 2)'))).toBe(true); + expect(isParseError(makeErrorMsg('Extra data: line 2 column 1 (line 1)'))).toBe(true); + }); + + it('detects generic JSON parse mentions', () => { + expect(isParseError(makeErrorMsg('JSON parse error: Unexpected token o'))).toBe(true); + expect(isParseError(makeErrorMsg('Invalid JSON received from provider'))).toBe(true); + expect(isParseError(makeErrorMsg('SyntaxError: JSON.parse: unexpected character at line 1'))).toBe(true); + }); + + it('returns false for unrelated errors', () => { + expect(isParseError(makeErrorMsg('500 Internal Server Error'))).toBe(false); + expect(isParseError(makeErrorMsg('Request timed out'))).toBe(false); + expect(isParseError(makeErrorMsg('429 Too Many Requests'))).toBe(false); + expect(isParseError(makeErrorMsg('context length exceeded'))).toBe(false); + }); + + it('returns false for non-error messages', () => { + expect(isParseError(makeErrorMsg('', 'stop'))).toBe(false); + expect(isParseError({ role: 'assistant', stopReason: 'stop' } as any)).toBe(false); + }); + + it('returns false when errorMessage is missing', () => { + expect(isParseError({ role: 'assistant', stopReason: 'error' } as any)).toBe(false); + }); +}); + // ── classifyError (unified dispatch) ────────────────────────────────── describe('classifyError', () => { @@ -263,6 +311,14 @@ describe('classifyError', () => { expect(classifyError(makeErrorMsg('Response terminated by content policy'))).toBe(ErrorCategory.TERMINATED); }); + it('classifies JSON parse errors', () => { + expect(classifyError(makeErrorMsg('Expected \'"\' or \'}\' after property value'))).toBe(ErrorCategory.PARSE_ERROR); + expect(classifyError(makeErrorMsg('Unexpected end of JSON input'))).toBe(ErrorCategory.PARSE_ERROR); + expect(classifyError(makeErrorMsg('Expecting value: line 1 column 5'))).toBe(ErrorCategory.PARSE_ERROR); + expect(classifyError(makeErrorMsg('JSON parse error: unexpected token'))).toBe(ErrorCategory.PARSE_ERROR); + expect(classifyError(makeErrorMsg('Invalid JSON'))).toBe(ErrorCategory.PARSE_ERROR); + }); + it('returns UNKNOWN for unrecognized errors', () => { expect(classifyError(makeErrorMsg('Some weird error nobody has seen'))).toBe(ErrorCategory.UNKNOWN); expect(classifyError(makeErrorMsg(''))).toBe(ErrorCategory.UNKNOWN); @@ -305,12 +361,14 @@ describe('configurable error patterns', () => { quotaExhausted: { enabled: false, patterns: [/credit/i, /quota/i], baseDelayMs: 1000, maxDelayMs: 30000 }, timeout: { enabled: true, patterns: [/timeout/i, /timed out/i], baseDelayMs: 2000, maxDelayMs: 60000 }, terminated: { enabled: false, patterns: [/terminated/i, /content.filter/i], baseDelayMs: 1000, maxDelayMs: 30000 }, + parseError: { enabled: true, patterns: [/unexpected end of json/i, /expecting value/i], baseDelayMs: 0, maxDelayMs: 0 }, }; expect(fullConfig.rateLimit.enabled).toBe(false); expect(fullConfig.serverError.enabled).toBe(true); expect(fullConfig.serverError.baseDelayMs).toBe(2000); expect(fullConfig.contextLength.enabled).toBe(true); + expect(fullConfig.parseError.enabled).toBe(true); }); }); diff --git a/packages/tui/extensions/Worklog/lib/recovery/error-patterns.ts b/packages/tui/extensions/Worklog/lib/recovery/error-patterns.ts index e6520c058..fd1b612d2 100644 --- a/packages/tui/extensions/Worklog/lib/recovery/error-patterns.ts +++ b/packages/tui/extensions/Worklog/lib/recovery/error-patterns.ts @@ -5,7 +5,7 @@ * per-category detection with configurable regex patterns that can be * overridden via settings. * - * Seven error categories are defined: + * Eight error categories are defined: * - Rate limits (429) → NOT retried (informative error) * - Server errors (5xx) → retried with configurable backoff * - Auth errors (401/403) → NOT retried (checkpoint + terminal) @@ -13,6 +13,7 @@ * - Quota exhausted → NOT retried (checkpoint + terminal) * - Timeout → retried with configurable backoff * - Terminated → NOT retried (checkpoint + terminal) + * - Parse errors (JSON) → single-shot continue (no backoff) */ import type { AgentMessage } from '@earendil-works/pi-agent-core'; @@ -27,6 +28,7 @@ export enum ErrorCategory { QUOTA_EXHAUSTED = 'quotaExhausted', TIMEOUT = 'timeout', TERMINATED = 'terminated', + PARSE_ERROR = 'parseError', UNKNOWN = 'unknown', } @@ -62,6 +64,7 @@ export interface RecoveryConfig { quotaExhausted: RecoveryCategoryConfig; timeout: RecoveryCategoryConfig; terminated: RecoveryCategoryConfig; + parseError: RecoveryCategoryConfig; } // ── Type guard ───────────────────────────────────────────────────────── @@ -154,6 +157,31 @@ const DEFAULT_TERMINATED_PATTERNS: RegExp[] = [ /no\s*such\s*model/i, ]; +const DEFAULT_PARSE_ERROR_PATTERNS: RegExp[] = [ + // V8 / Node.js JSON.parse messages + /expected\s+['\"]|['\"]\s+after\s+property\s+value/i, + /unexpected\s+end\s+of\s+json\s+input/i, + /unexpected\s+token/i, + /unexpected\s+number\s+in\s+json/i, + /unexpected\s+string\s+in\s+json/i, + /unexpected\s+non-whitespace\s+character\s+after\s+json/i, + /unterminated\s+string\s+in\s+json/i, + /bad\s+control\s+character\s+in\s+string\s+literal\s+in\s+json/i, + /json\s+parse\s+error/i, + // Python json.JSONDecodeError texts + /expecting\s+value/i, + /expecting\s+','\s*delimiter/i, + /expecting\s+':'\s*delimiter/i, + /expecting\s+property\s+name\s+enclosed\s+in\s+double\s+quotes/i, + /unterminated\s+string\s+starting\s+at/i, + /invalid\s+control\s+character\s+at/i, + /extra\s+data/i, + /json\s+decode\s+error/i, + // Generic JSON parse mentions + /invalid\s+json/i, + /json\.parse/i, +]; + // ── Default config ──────────────────────────────────────────────────── export const DEFAULT_RECOVERY_CONFIG: RecoveryConfig = { @@ -200,6 +228,13 @@ export const DEFAULT_RECOVERY_CONFIG: RecoveryConfig = { baseDelayMs: 1000, maxDelayMs: 30000, }, + parseError: { + enabled: true, + patterns: DEFAULT_PARSE_ERROR_PATTERNS, + baseDelayMs: 0, + maxDelayMs: 0, + continuationPrompt: 'continue', + }, }; // ── Match helper ────────────────────────────────────────────────────── @@ -262,6 +297,11 @@ export function isTerminated(message: unknown): boolean { return matchesAny(message.errorMessage, DEFAULT_TERMINATED_PATTERNS); } +export function isParseError(message: unknown): boolean { + if (!isAssistantMessage(message)) return false; + return matchesAny(message.errorMessage, DEFAULT_PARSE_ERROR_PATTERNS); +} + // ── Unified classifier ──────────────────────────────────────────────── /** @@ -287,6 +327,7 @@ export function classifyError(message: unknown): ErrorCategory { if (isTimeout(message)) return ErrorCategory.TIMEOUT; if (isServerError(message)) return ErrorCategory.SERVER_ERROR; if (isTerminated(message)) return ErrorCategory.TERMINATED; + if (isParseError(message)) return ErrorCategory.PARSE_ERROR; return ErrorCategory.UNKNOWN; } catch { @@ -309,6 +350,7 @@ export function getDefaultPatterns(category: ErrorCategory): RegExp[] { case ErrorCategory.QUOTA_EXHAUSTED: return DEFAULT_QUOTA_PATTERNS; case ErrorCategory.TIMEOUT: return DEFAULT_TIMEOUT_PATTERNS; case ErrorCategory.TERMINATED: return DEFAULT_TERMINATED_PATTERNS; + case ErrorCategory.PARSE_ERROR: return DEFAULT_PARSE_ERROR_PATTERNS; default: return []; } } diff --git a/packages/tui/extensions/Worklog/lib/recovery/parse-error-continue.test.ts b/packages/tui/extensions/Worklog/lib/recovery/parse-error-continue.test.ts new file mode 100644 index 000000000..5a3c52267 --- /dev/null +++ b/packages/tui/extensions/Worklog/lib/recovery/parse-error-continue.test.ts @@ -0,0 +1,147 @@ +/** + * Tests for the single-shot parse-error continue handler. + * + * Covers the operator decision: a JSON parse error resumes the session + * with exactly one plain "continue" prompt — no exponential-backoff + * retry loop — and a repeated parse error triggers a new continue on the + * next agent_end (one prompt per occurrence, no unbounded loop). + * + * Run: cd /home/rgardler/projects/ContextHub && npx vitest run packages/tui/extensions/Worklog/lib/recovery/parse-error-continue.test.ts + */ + +import { describe, it, expect, vi } from 'vitest'; +import { executeParseErrorContinue } from './recovery.js'; +import { classifyError, ErrorCategory } from './error-patterns.js'; + +// ── executeParseErrorContinue ───────────────────────────────────────── + +describe('executeParseErrorContinue', () => { + it('sends exactly one continue prompt (single-shot, no retry loop)', async () => { + const sendContinue = vi.fn().mockResolvedValue(undefined); + + const result = await executeParseErrorContinue({ sendContinue }); + + expect(sendContinue).toHaveBeenCalledTimes(1); + expect(result.success).toBe(true); + expect(result.promptCount).toBe(1); + }); + + it('waits for the agent to become idle before sending the prompt', async () => { + const order: string[] = []; + const waitForIdle = vi.fn(async () => { + order.push('waitForIdle'); + }); + const sendContinue = vi.fn(async () => { + order.push('sendContinue'); + }); + + await executeParseErrorContinue({ waitForIdle, sendContinue }); + + expect(order).toEqual(['waitForIdle', 'sendContinue']); + expect(waitForIdle).toHaveBeenCalledTimes(1); + }); + + it('does not loop on failure — one prompt per call', async () => { + const sendContinue = vi.fn().mockResolvedValue(undefined); + + // Simulate a repeated parse error across two agent_end events: + // each occurrence gets exactly one continue prompt. + await executeParseErrorContinue({ sendContinue }); + await executeParseErrorContinue({ sendContinue }); + + expect(sendContinue).toHaveBeenCalledTimes(2); + }); + + it('skips the prompt when shouldAbort returns true before idle', async () => { + const waitForIdle = vi.fn().mockResolvedValue(undefined); + const sendContinue = vi.fn().mockResolvedValue(undefined); + + const result = await executeParseErrorContinue({ + waitForIdle, + sendContinue, + shouldAbort: () => true, + }); + + expect(sendContinue).not.toHaveBeenCalled(); + expect(waitForIdle).not.toHaveBeenCalled(); + expect(result.success).toBe(false); + expect(result.promptCount).toBe(0); + expect(result.error).toBe('aborted'); + }); + + it('skips the prompt when abort happens during the idle wait', async () => { + const waitForIdle = vi.fn().mockResolvedValue(undefined); + const sendContinue = vi.fn().mockResolvedValue(undefined); + + const result = await executeParseErrorContinue({ + waitForIdle, + sendContinue, + // Abort only after the idle wait (second check) + shouldAbort: vi.fn().mockReturnValueOnce(false).mockReturnValueOnce(true), + }); + + expect(waitForIdle).toHaveBeenCalledTimes(1); + expect(sendContinue).not.toHaveBeenCalled(); + expect(result.success).toBe(false); + expect(result.promptCount).toBe(0); + }); + + it('handles exceptions from sendContinue gracefully', async () => { + const sendContinue = vi.fn().mockRejectedValue(new Error('agent busy')); + + const result = await executeParseErrorContinue({ sendContinue }); + + expect(result.success).toBe(false); + expect(result.promptCount).toBe(0); + expect(result.error).toContain('agent busy'); + }); + + it('handles non-Error exceptions gracefully', async () => { + const sendContinue = vi.fn().mockRejectedValue('string error'); + + const result = await executeParseErrorContinue({ sendContinue }); + + expect(result.success).toBe(false); + expect(result.error).toBe('Unknown error during parse-error continue'); + }); +}); + +// ── Integration: classification → single-shot continue ─────────────── + +describe('parse-error classification → continue integration', () => { + it('full flow: classify parse error → one continue prompt', async () => { + // Simulate an agent_end message from audit_debug JSONL processing + const message = { + role: 'assistant', + stopReason: 'error', + errorMessage: "Expected ',' or '}' after property value in JSON at position 42", + }; + + expect(classifyError(message)).toBe(ErrorCategory.PARSE_ERROR); + + const sendContinue = vi.fn().mockResolvedValue(undefined); + const result = await executeParseErrorContinue({ sendContinue }); + + expect(result.success).toBe(true); + expect(result.promptCount).toBe(1); + expect(sendContinue).toHaveBeenCalledTimes(1); + }); + + it('Python JSONDecodeError classifies as PARSE_ERROR', () => { + const message = { + role: 'assistant', + stopReason: 'error', + errorMessage: 'Expecting value: line 1 column 123 (char 122)', + }; + expect(classifyError(message)).toBe(ErrorCategory.PARSE_ERROR); + }); + + it('unrelated errors do not trigger the parse-error continue path', () => { + const message = { + role: 'assistant', + stopReason: 'error', + errorMessage: 'Some weird provider error', + }; + expect(classifyError(message)).toBe(ErrorCategory.UNKNOWN); + }); +}); diff --git a/packages/tui/extensions/Worklog/lib/recovery/recovery.ts b/packages/tui/extensions/Worklog/lib/recovery/recovery.ts index eb6f212df..e9bf275da 100644 --- a/packages/tui/extensions/Worklog/lib/recovery/recovery.ts +++ b/packages/tui/extensions/Worklog/lib/recovery/recovery.ts @@ -292,3 +292,69 @@ export async function executeCompactAndContinue( }; } } + +// ── Parse-error continue handler ───────────────────────────────────── + +/** + * Result of a single-shot parse-error continue operation. + */ +export interface ParseErrorContinueResult { + /** Whether the continue prompt was sent */ + success: boolean; + /** + * Number of continue prompts sent (always 1 on success — single-shot; + * 0 when skipped or failed). + */ + promptCount: number; + /** Optional error message if the operation failed */ + error?: string; +} + +/** + * Execute the single-shot continue flow for JSON parse errors. + * + * Per operator decision, a JSON parse error resumes the session with + * exactly one plain "continue" prompt — no exponential-backoff retry + * loop. The error message stays in agent state so the agent can decide + * how to proceed (e.g., skip the malformed record). A repeated parse + * error on the next `agent_end` triggers a new single-shot continue + * (one prompt per occurrence, never an unbounded loop within a turn). + * + * This is a pure-logic harness — it takes the functions it needs as + * parameters so it can be tested without a live agent. + * + * @param options.waitForIdle - Optional; waits for the agent to become idle + * @param options.sendContinue - Sends the plain "continue" prompt + * @param options.shouldAbort - Optional; checked before/after idle wait; + * returns true when the continue should be skipped (e.g. user abort) + * @returns Promise with the result + */ +export async function executeParseErrorContinue( + options: { + waitForIdle?: () => Promise; + sendContinue: () => Promise; + shouldAbort?: () => boolean; + }, +): Promise { + try { + if (options.shouldAbort?.()) { + return { success: false, promptCount: 0, error: 'aborted' }; + } + if (options.waitForIdle) { + await options.waitForIdle(); + } + if (options.shouldAbort?.()) { + return { success: false, promptCount: 0, error: 'aborted' }; + } + // Single-shot: exactly one continue prompt, then return. No backoff, + // no loop — a persistent parse error re-triggers on the next agent_end. + await options.sendContinue(); + return { success: true, promptCount: 1 }; + } catch (err) { + return { + success: false, + promptCount: 0, + error: err instanceof Error ? err.message : 'Unknown error during parse-error continue', + }; + } +} diff --git a/packages/tui/extensions/Worklog/lib/recovery/register-recovery.ts b/packages/tui/extensions/Worklog/lib/recovery/register-recovery.ts index b941eed1e..67927c6e0 100644 --- a/packages/tui/extensions/Worklog/lib/recovery/register-recovery.ts +++ b/packages/tui/extensions/Worklog/lib/recovery/register-recovery.ts @@ -13,7 +13,8 @@ import type { ExtensionAPI, ExtensionCommandContext, SessionCompactEvent } from '@earendil-works/pi-coding-agent'; -import { classifyError, ErrorCategory } from './error-patterns.js'; +import { classifyError, DEFAULT_RECOVERY_CONFIG, ErrorCategory } from './error-patterns.js'; +import { executeParseErrorContinue } from './recovery.js'; import { retryStates, continuationState, @@ -140,6 +141,27 @@ export function registerRecoveryModule(pi: ExtensionAPI): void { break; } + case ErrorCategory.PARSE_ERROR: { + // Parse errors: single-shot auto-continue with a plain "continue" + // prompt. Per operator decision there is NO exponential-backoff + // loop — one "continue" per detected parse error. The error message + // stays in agent state so the agent decides how to proceed (e.g., + // skip the malformed record). A repeated parse error triggers a new + // continue on the next agent_end (single-shot per occurrence). + const errorMsg = lastAssistant.errorMessage || 'Unknown error'; + const state = retryStates[category as string]; + if (state) { + state.startRetry(errorMsg); + state.endRetry(); + } + ctx.ui.notify( + `JSON parse error — auto-continuing: ${errorMsg.substring(0, 100)}...`, + 'info', + ); + void triggerParseErrorContinue(); + break; + } + case ErrorCategory.UNKNOWN: default: // Unknown errors: show message, no retry @@ -228,6 +250,9 @@ export function registerRecoveryModule(pi: ExtensionAPI): void { triggerCompactContinue: () => { ctx.ui.notify('Continuing after context-length...', 'info'); }, + triggerParseErrorContinue: () => { + ctx.ui.notify('Continuing after JSON parse error...', 'info'); + }, triggerCheckpointTerminate: (_category: string, _errorDetail: string) => { ctx.ui.notify(`${_category}: ${_errorDetail.substring(0, 100)}`, 'error'); }, @@ -398,6 +423,45 @@ async function triggerCompactionContinue(): Promise { void triggerInvisibleContinue(); } +/** + * Single-shot invisible continue for JSON parse errors. + * + * Waits for the agent to become idle, then sends exactly one plain + * "continue" prompt. No exponential backoff and no error-message removal — + * a deterministic parse error simply triggers another continue on the next + * agent_end (single-shot per occurrence). Guards against user abort and + * session switches; the mutex prevents concurrent continues. + */ +async function triggerParseErrorContinue(): Promise { + if (!_agent) return; + + // Guard: if user aborted, don't start + if (interruptibleState.userAborted) return; + + // Guard: mutex — only one continue at a time + if (_continueInProgress) return; + _continueInProgress = true; + + const sessionGenerationAtStart = interruptibleState.sessionGeneration; + + try { + await executeParseErrorContinue({ + waitForIdle: () => _agent?.waitForIdle?.(), + // Plain "continue" prompt per operator decision; text is + // settings-driven via the parseError category config. + sendContinue: () => _agent.prompt([DEFAULT_RECOVERY_CONFIG.parseError.continuationPrompt ?? 'continue']), + shouldAbort: () => + interruptibleState.userAborted || + interruptibleState.sessionGeneration !== sessionGenerationAtStart, + }); + } finally { + // Only reset mutex if session hasn't changed + if (interruptibleState.sessionGeneration === sessionGenerationAtStart) { + _continueInProgress = false; + } + } +} + /** Notify user about a retry attempt */ function _notifyRetryAttempt(attempt: number, duration: string): void { if (_notifyFn) { diff --git a/packages/tui/extensions/Worklog/lib/recovery/retry-command.test.ts b/packages/tui/extensions/Worklog/lib/recovery/retry-command.test.ts index eb018b7cd..8362f4eed 100644 --- a/packages/tui/extensions/Worklog/lib/recovery/retry-command.test.ts +++ b/packages/tui/extensions/Worklog/lib/recovery/retry-command.test.ts @@ -34,6 +34,7 @@ describe('formatRetryStatus', () => { expect(status).toContain('quotaExhausted'); expect(status).toContain('timeout'); expect(status).toContain('terminated'); + expect(status).toContain('parseError'); expect(status).toContain('Continuation'); }); @@ -151,6 +152,7 @@ describe('executeRetryCommand', () => { mockOptions = { triggerRetry: vi.fn(), triggerCompactContinue: vi.fn(), + triggerParseErrorContinue: vi.fn(), triggerCheckpointTerminate: vi.fn(), }; }); @@ -284,6 +286,29 @@ describe('executeRetryCommand', () => { expect(mockOptions.triggerCompactContinue).toHaveBeenCalled(); }); + // /retry with JSON parse error + it('/retry with JSON parse error triggers single-shot continue', async () => { + mockCtx.sessionManager.getEntries = vi.fn().mockReturnValue([ + { + type: 'message', + message: { + role: 'assistant', + stopReason: 'error', + errorMessage: "Unexpected end of JSON input", + }, + }, + ]); + + await executeRetryCommand('', mockCtx, mockOptions); + + expect(mockCtx.ui.notify).toHaveBeenCalledWith( + expect.stringContaining('JSON parse error'), + 'info', + ); + expect(mockOptions.triggerParseErrorContinue).toHaveBeenCalled(); + expect(mockOptions.triggerRetry).not.toHaveBeenCalled(); + }); + // /retry with unknown error it('/retry with unknown error shows warning', async () => { mockCtx.sessionManager.getEntries = vi.fn().mockReturnValue([ @@ -360,6 +385,12 @@ describe('getRetryStateForCategory', () => { expect(state?.getAttempt()).toBe(0); }); + it('returns RetryState for parseError category', () => { + const state = getRetryStateForCategory(ErrorCategory.PARSE_ERROR); + expect(state).toBeDefined(); + expect(state?.getAttempt()).toBe(0); + }); + it('returns undefined for unknown category', () => { const state = getRetryStateForCategory('unknown' as any); expect(state).toBeUndefined(); diff --git a/packages/tui/extensions/Worklog/lib/recovery/retry-command.ts b/packages/tui/extensions/Worklog/lib/recovery/retry-command.ts index 56939f887..012cdc8ab 100644 --- a/packages/tui/extensions/Worklog/lib/recovery/retry-command.ts +++ b/packages/tui/extensions/Worklog/lib/recovery/retry-command.ts @@ -30,6 +30,7 @@ export const retryStates: Record = { quotaExhausted: new RetryState(), timeout: new RetryState(), terminated: new RetryState(), + parseError: new RetryState(), }; export const continuationState = new ContinuationState(); @@ -57,6 +58,8 @@ export interface RetryCommandOptions { triggerCompactContinue: () => void; /** Called to trigger checkpoint-and-terminate */ triggerCheckpointTerminate: (category: string, errorDetail: string) => void; + /** Called to trigger a single-shot continue after a JSON parse error */ + triggerParseErrorContinue: () => void; } /** @@ -164,6 +167,11 @@ export async function executeRetryCommand( options.triggerCompactContinue(); break; + case ErrorCategory.PARSE_ERROR: + ctx.ui.notify('Manual continue after JSON parse error...', 'info'); + options.triggerParseErrorContinue(); + break; + case ErrorCategory.UNKNOWN: default: ctx.ui.notify( diff --git a/packages/tui/extensions/Worklog/lib/recovery/settings.test.ts b/packages/tui/extensions/Worklog/lib/recovery/settings.test.ts index de1ead63c..0b149010a 100644 --- a/packages/tui/extensions/Worklog/lib/recovery/settings.test.ts +++ b/packages/tui/extensions/Worklog/lib/recovery/settings.test.ts @@ -13,11 +13,12 @@ import { DEFAULT_RECOVERY_CONFIG, type RecoveryConfig, type RecoveryCategoryConf // ── RecoveryConfig defaults ─────────────────────────────────────────── describe('DEFAULT_RECOVERY_CONFIG', () => { - it('has all 7 error categories', () => { + it('has all 8 error categories', () => { const keys = Object.keys(DEFAULT_RECOVERY_CONFIG); expect(keys.sort()).toEqual([ 'authError', 'contextLength', + 'parseError', 'quotaExhausted', 'rateLimit', 'serverError', @@ -61,6 +62,15 @@ describe('DEFAULT_RECOVERY_CONFIG', () => { expect(DEFAULT_RECOVERY_CONFIG.terminated.enabled).toBe(false); }); + it('parse error defaults to enabled (single-shot continue)', () => { + expect(DEFAULT_RECOVERY_CONFIG.parseError.enabled).toBe(true); + expect(DEFAULT_RECOVERY_CONFIG.parseError.patterns.length).toBeGreaterThan(0); + expect(DEFAULT_RECOVERY_CONFIG.parseError.continuationPrompt).toBe('continue'); + // No backoff for the single-shot category + expect(DEFAULT_RECOVERY_CONFIG.parseError.baseDelayMs).toBe(0); + expect(DEFAULT_RECOVERY_CONFIG.parseError.maxDelayMs).toBe(0); + }); + it('each category has patterns array', () => { for (const [key, cat] of Object.entries(DEFAULT_RECOVERY_CONFIG)) { expect(Array.isArray((cat as RecoveryCategoryConfig).patterns)).toBe(true); @@ -70,10 +80,19 @@ describe('DEFAULT_RECOVERY_CONFIG', () => { it('each category has valid baseDelayMs and maxDelayMs', () => { for (const cat of Object.values(DEFAULT_RECOVERY_CONFIG)) { - expect(cat.baseDelayMs).toBeGreaterThanOrEqual(100); + expect(cat.baseDelayMs).toBeGreaterThanOrEqual(0); expect(cat.maxDelayMs).toBeGreaterThanOrEqual(cat.baseDelayMs); } }); + + it('backoff-based categories have a positive base delay', () => { + for (const cat of Object.values(DEFAULT_RECOVERY_CONFIG)) { + // Single-shot categories (e.g. parseError) legitimately use 0ms + // delays because they never back off. + if (cat.baseDelayMs === 0) continue; + expect(cat.baseDelayMs).toBeGreaterThanOrEqual(100); + } + }); }); // ── RecoveryConfig type validation ──────────────────────────────────── From 1e194fc1cb7744816b5e25021e2a19cfe172e79f Mon Sep 17 00:00:00 2001 From: pi Date: Fri, 14 Aug 2026 00:24:41 +0100 Subject: [PATCH 13/34] WL-0MSRG481O007QVEA: install-herdr.sh links plugin from main checkout (worktree-safe) Fixes the dangling herdr plugin link left after worktree cleanup: the postbuild hook now resolves the MAIN checkout via 'git worktree list --porcelain' and links the global herdr plugin from there, never from a transient worktree path. 'herdr plugin link' updates an existing link in place, so stale worktree links self-heal on the next build. - scripts/install-herdr.sh: main-checkout resolution + manifest path var - tests/scripts/install-herdr.test.ts: worktree-scenario test (real git worktree + fake herdr CLI, asserts main-checkout manifest is linked) - packages/herdr/README.md: document worktree-safe linking --- packages/herdr/README.md | 2 +- scripts/install-herdr.sh | 31 +++++++- tests/scripts/install-herdr.test.ts | 110 ++++++++++++++++++++++++++++ 3 files changed, 138 insertions(+), 5 deletions(-) diff --git a/packages/herdr/README.md b/packages/herdr/README.md index e1e0fb850..b074d7bc5 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -48,7 +48,7 @@ cd packages/herdr && npm run build `npm run build` runs `scripts/install-herdr.sh` via the root `postbuild` hook. The script: -- Links the plugin with `herdr plugin link packages/herdr/herdr-plugin.toml` (a no-op when already linked). +- Links the plugin from the **main checkout** with `herdr plugin link /packages/herdr/herdr-plugin.toml`. When the build runs inside a linked git worktree (as happens during `/skill:implement`), the main checkout is resolved via `git worktree list` so the global herdr plugin is never registered against a transient worktree path — a worktree-based link would dangle once the worktree is cleaned up and silently break `prefix+l` (WL-0MSRG481O007QVEA). `herdr plugin link` updates an existing link in place, so a stale link from an older build is corrected automatically. - Inserts the `prefix+l` → `worklog-selection-list.open-podcast-editor-tab` keybinding into your herdr config (`~/.config/herdr/config.toml`, or `$HERDR_CONFIG_PATH` when set) **only if** it is not already present — re-running the build never creates duplicate keybindings. - Migrates a legacy `prefix+l` → `worklog-selection-list.open-worklist` binding in-place to the new action, so the Podcast Editing tab name takes effect without a manual config edit. - Warns (without failing the build) when `herdr` is not on PATH or the config cannot be written, so `npm run build` succeeds in CI/offline environments. diff --git a/scripts/install-herdr.sh b/scripts/install-herdr.sh index 173146a90..9f57a793a 100755 --- a/scripts/install-herdr.sh +++ b/scripts/install-herdr.sh @@ -6,7 +6,12 @@ # the box after `npm run build` (invoked via the root `postbuild` hook). # # Safe to run on every build: -# - Re-linking the plugin is a no-op (herdr's own idempotence). +# - The plugin is always linked from the MAIN checkout, even when the +# build runs inside a linked git worktree (WL-0MSRG481O007QVEA): +# linking from a worktree leaves a dangling registry entry once the +# worktree is deleted, silently breaking the prefix+l keybinding. +# `herdr plugin link` updates an existing link in place, so a stale +# worktree link is corrected automatically on the next build. # - The keybinding block is inserted only when a binding for # `worklog-selection-list.open-podcast-editor-tab` is not already present. # - A legacy `open-worklist` prefix+l binding is migrated in-place to the @@ -18,6 +23,24 @@ set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" +# ── Main-checkout resolution (worktree-safe) ──────────────────────────── +# During /skill:implement the postbuild hook runs from a linked git +# worktree, so REPO_ROOT would resolve to the worktree path. Linking the +# global herdr plugin from a worktree leaves a dangling registry entry once +# implement.py finish deletes the worktree — the prefix+l keybinding then +# invokes a non-existent action and silently does nothing. Resolve the main +# checkout (always listed first by `git worktree list --porcelain`) and +# link from there instead. Falls back to REPO_ROOT when not inside a git +# repo or git is unavailable. +if command -v git >/dev/null 2>&1 && git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + MAIN_CHECKOUT="$(git worktree list --porcelain 2>/dev/null | awk '/^worktree /{sub(/^worktree /, ""); print; exit}')" + if [ -n "${MAIN_CHECKOUT}" ] && [ -d "${MAIN_CHECKOUT}" ]; then + REPO_ROOT="${MAIN_CHECKOUT}" + fi +fi + +MANIFEST_PATH="${REPO_ROOT}/packages/herdr/herdr-plugin.toml" + # Config resolution matches herdr itself: HERDR_CONFIG_PATH overrides, # otherwise ~/.config/herdr/config.toml. CONFIG_PATH="${HERDR_CONFIG_PATH:-${HOME}/.config/herdr/config.toml}" @@ -34,10 +57,10 @@ description = "Open the Podcast Editing tab (Worklog work item selection pane)." LEGACY_BINDING='herdr plugin action invoke worklog-selection-list.open-worklist' NEW_BINDING='herdr plugin action invoke worklog-selection-list.open-podcast-editor-tab' -# ── 1. Link the plugin (no-op when already linked) ────────────────────── +# ── 1. Link the plugin (updates an existing link in place) ─────────────── if command -v herdr >/dev/null 2>&1; then - if herdr plugin link "${REPO_ROOT}/packages/herdr/herdr-plugin.toml"; then - echo "Linked herdr plugin: worklog-selection-list" + if herdr plugin link "${MANIFEST_PATH}"; then + echo "Linked herdr plugin: worklog-selection-list (${MANIFEST_PATH})" else echo "Warning: 'herdr plugin link' failed (plugin may already be linked, or herdr is busy)." >&2 fi diff --git a/tests/scripts/install-herdr.test.ts b/tests/scripts/install-herdr.test.ts index 8510a3e19..8225e237e 100644 --- a/tests/scripts/install-herdr.test.ts +++ b/tests/scripts/install-herdr.test.ts @@ -47,6 +47,41 @@ function makeTempDir(prefix: string): string { return fs.mkdtempSync(path.join(os.tmpdir(), prefix)); } +/** + * Run a git command, asserting success. Identity and HOME are pinned so the + * temp repo is fully hermetic (no dependency on the developer's global git + * config). `gitBin` is the REAL git binary — see {@link realGitBin}. + */ +function git(gitBin: string, args: string[], cwd: string, tempHome: string) { + const result = spawnSync(gitBin, args, { + cwd, + encoding: 'utf8', + env: { ...process.env, HOME: tempHome }, + }); + expect(result.status, `git ${args.join(' ')} failed: ${result.stderr}`).toBe(0); + return result; +} + +/** + * The vitest setup (tests/setup-tests.ts) prepends tests/cli/mock-bin to + * PATH, which ships a git mock that does NOT implement `worktree list`. + * The worktree-scenario test needs REAL git worktree mechanics, so resolve + * the real git binary by searching PATH with the mock-bin entries removed. + */ +function realGitBin(): string { + const mockBinDir = path.resolve(__dirname, '..', 'cli', 'mock-bin'); + const candidates = (process.env.PATH ?? '') + .split(path.delimiter) + .filter((dir) => dir !== '' && path.resolve(dir) !== mockBinDir); + for (const dir of candidates) { + const candidate = path.join(dir, 'git'); + if (fs.existsSync(candidate)) { + return candidate; + } + } + throw new Error('real git binary not found on PATH (needed for worktree-scenario test)'); +} + describe('install-herdr script', () => { it('inserts the keybinding block exactly once when run twice (idempotent)', () => { const tempDir = makeTempDir('worklog-herdr-idempotent-'); @@ -154,4 +189,79 @@ describe('install-herdr script', () => { expect(countOccurrences(content, '[[keys.command]]')).toBe(1); expect(content).toContain('command = "herdr plugin action invoke ' + COMMAND_MARKER + '"'); }); + + it('links the plugin from the main checkout when run inside a linked worktree', () => { + // Reproduce the /skill:implement build: the postbuild hook runs + // scripts/install-herdr.sh from a linked git worktree. The script must + // register the global herdr plugin from the MAIN checkout, never the + // worktree — a worktree-based link dangles once implement.py finish + // deletes the worktree (WL-0MSRG481O007QVEA), silently breaking the + // prefix+l keybinding. + const tempBase = makeTempDir('worklog-herdr-wt-'); + const repoDir = path.join(tempBase, 'repo'); + const wtDir = path.join(tempBase, 'worktree'); + fs.mkdirSync(path.join(repoDir, 'scripts'), { recursive: true }); + fs.mkdirSync(path.join(repoDir, 'packages', 'herdr'), { recursive: true }); + + // Minimal repo layout mirroring this repository: the script plus the + // plugin manifest it links. + const repoRoot = path.resolve(__dirname, '..', '..'); + fs.writeFileSync( + path.join(repoDir, 'scripts', 'install-herdr.sh'), + fs.readFileSync(path.join(repoRoot, 'scripts', 'install-herdr.sh'), 'utf8'), + ); + fs.writeFileSync( + path.join(repoDir, 'packages', 'herdr', 'herdr-plugin.toml'), + 'id = "worklog-selection-list"\n', + ); + + const gitBin = realGitBin(); + git(gitBin, ['init', '-q', repoDir], tempBase, tempBase); + git(gitBin, ['-C', repoDir, 'config', 'user.email', 'test@example.com'], tempBase, tempBase); + git(gitBin, ['-C', repoDir, 'config', 'user.name', 'Worklog Test'], tempBase, tempBase); + git(gitBin, ['-C', repoDir, 'add', '-A'], tempBase, tempBase); + git(gitBin, ['-C', repoDir, 'commit', '-q', '-m', 'init'], tempBase, tempBase); + git(gitBin, ['-C', repoDir, 'worktree', 'add', '--detach', wtDir, 'HEAD'], tempBase, tempBase); + + // Fake herdr CLI that records the manifest path it is asked to link. + const fakeBinDir = path.join(tempBase, 'bin'); + fs.mkdirSync(fakeBinDir); + const linkLog = path.join(tempBase, 'herdr-link.log'); + fs.writeFileSync( + path.join(fakeBinDir, 'herdr'), + '#!/usr/bin/env bash\necho "$@" >> "' + linkLog + '"\nexit 0\n', + ); + fs.chmodSync(path.join(fakeBinDir, 'herdr'), 0o755); + + // The script must see the REAL git (not the test-suite mock, which does + // not implement `worktree list`) so its main-checkout resolution runs + // against genuine worktree metadata. Build a PATH that has the fake + // herdr first, then the real git's directory, with mock-bin removed. + const realGitDir = path.dirname(gitBin); + const mockBinDir = path.resolve(__dirname, '..', 'cli', 'mock-bin'); + const cleanPath = (process.env.PATH ?? '') + .split(path.delimiter) + .filter((dir) => dir !== '' && path.resolve(dir) !== mockBinDir); + const scriptPath = `${fakeBinDir}${path.delimiter}${realGitDir}${path.delimiter}${cleanPath.join(path.delimiter)}`; + + const configPath = path.join(tempBase, 'herdr', 'config.toml'); + const result = spawnSync('bash', [path.join(wtDir, 'scripts', 'install-herdr.sh')], { + cwd: wtDir, + encoding: 'utf8', + env: { + ...process.env, + PATH: scriptPath, + HERDR_CONFIG_PATH: configPath, + }, + }); + expect(result.status, `script failed: ${result.stderr}`).toBe(0); + + // The fake herdr must have been asked to link the MAIN checkout's + // manifest — never the worktree's (the worktree may be deleted at any + // time, which is exactly the regression this guards against). + const log = fs.readFileSync(linkLog, 'utf8').trim(); + const expectedManifest = path.join(repoDir, 'packages', 'herdr', 'herdr-plugin.toml'); + expect(log).toContain(expectedManifest); + expect(log).not.toContain(wtDir); + }); }); From 4468451f7db69649341fb967392518fc08ac74c9 Mon Sep 17 00:00:00 2001 From: pi Date: Fri, 14 Aug 2026 01:43:32 +0100 Subject: [PATCH 14/34] WL-0MSLWJ2KP0002SV0: audit-tier wl failure is a wl-error strike, never a null empty tier MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit getNextAuditCandidate now resolves through the DowntimeNextResult error channel like the plan/intake tiers: {ok:false} on wl/parse failure (a CLI-error strike), {ok:true, candidate:null} only for a genuinely empty audit tier. dispatchDowntimeWork fails closed to wl-error on audit failure — no silent fall-through to the implement/plan tiers — so a persistently broken audit query can no longer disable audit dispatch forever while the worker looks healthy; the three-strike rule pauses and logs it. Empty/success paths unchanged. --- packages/herdr/README.md | 13 ++- packages/herdr/src/downtime-worker.test.ts | 125 +++++++++++++++++---- packages/herdr/src/downtime-worker.ts | 45 ++++++-- packages/herdr/src/index.test.ts | 26 ++--- packages/herdr/src/index.ts | 23 ++-- 5 files changed, 180 insertions(+), 52 deletions(-) diff --git a/packages/herdr/README.md b/packages/herdr/README.md index b074d7bc5..93cbce34a 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -263,6 +263,17 @@ dispatch still governs (fresh → not a candidate). A missing or unreadable log is treated as empty (fail-safe), so audit dispatch keeps working on a fresh worklog. +> **Audit-tier error channel (WL-0MSLWJ2KP0002SV0):** the audit lookup +> resolves through the same `DowntimeNextResult` error channel as the +> plan/intake tiers. A `wl`/CLI failure or unparseable output is `{ok:false}` +> — a CLI-error strike, never a `null` that is indistinguishable from a +> genuinely empty audit tier — so a persistently broken audit query fails +> closed to busy (the three-strike rule pauses the worker and logs the +> error) instead of silently falling through to the plan tier looking +> healthy with audit dispatch disabled forever. A genuinely empty audit +> tier (`{ok:true, candidate:null}`) falls through to the implement tier +> exactly as before. + > **Audit-tier selection note (WL-0MSMAIP5F003WAGG):** the audit tier keeps > its `wl list --status completed --stage in_review --json` selection rather > than converting to `wl next --stage in_review`. `wl next` is strictly @@ -399,7 +410,7 @@ leaves behind (documented for WL-0MSKUG2WW0058A7W, audit gap AC2): | Genuine empty backlog (no-candidate) | **none** — intentionally silent | full cooldown pause (default 60 min); worker stops polling | | 1–2 transient wl CLI errors (strikes) | **none** — silent | one strike per `wl-error` outcome; retries on the next idle window | | 3rd consecutive wl CLI error | `recordError` JSONL entry | three-strike pause; the only failure path that logs | -| Audit-tier wl/parse failure | **none** — silent, and **no strike** | `getNextAuditCandidate` collapses the failure to `null` like an empty tier; worker falls through to the plan tier and looks healthy — known silent path, follow-up WL-0MSLWJ2KP0002SV0 | +| Audit-tier wl/parse failure | **none** — silent, but **counts as a `wl-error` strike** | `getNextAuditCandidate` resolves `{ok:false}` (never a `null` that looks like an empty tier, WL-0MSLWJ2KP0002SV0); the dispatch fails closed to busy — no fall-through to the implement/plan tiers — and the three-strike rule pauses + logs it after 3 consecutive failures | | Lost CAS claim race (`--if-status`/`--if-stage` stale) | **none** — and **no marker, no pane, no success record** | the dispatch ABORTS with reason `claim-failed` (neutral — another pane won); the failure is observable via the outcome and a stderr line, never silently discarded (WL-0MSLWJ310000ND0X absorbed) | | Claim wl CLI failure (non-stale) | **none** — counts as a `wl-error` strike | dispatch aborts; three consecutive such failures pause the worker | | Marker write failure | **none** — the item stays claimed (`in_progress`) | dispatch ABORTS **before** the pane spawns with reason `marker-write-failed` (fail-closed: an unmarked item is never dispatched; the claim still removes it from `wl next`, so no other pane selects it) | diff --git a/packages/herdr/src/downtime-worker.test.ts b/packages/herdr/src/downtime-worker.test.ts index 5de41e58e..bca123a3a 100644 --- a/packages/herdr/src/downtime-worker.test.ts +++ b/packages/herdr/src/downtime-worker.test.ts @@ -93,7 +93,9 @@ import { function makeDeps(overrides: Partial = {}): DowntimeWorkerDeps { return { getNextItem: vi.fn().mockResolvedValue({ ok: true, candidate: null }), - getNextAuditCandidate: vi.fn().mockResolvedValue(null), + // Audit tier answers a GENUINELY empty tier by default ({ok:true, + // candidate:null}); a wl/parse failure is {ok:false} (WL-0MSLWJ2KP0002SV0). + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextImplementCandidate: vi.fn().mockResolvedValue(null), claimItem: vi.fn().mockResolvedValue({ ok: true }), spawnAgentPane: vi.fn().mockResolvedValue(true), @@ -388,7 +390,7 @@ describe('dispatch audit tier', () => { it('dispatches /skill:audit on the audit candidate as the first tier', async () => { const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue(staleCandidate), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: staleCandidate }), }); const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); @@ -406,7 +408,7 @@ describe('dispatch audit tier', () => { it('records the audit dispatch with kind audit', async () => { const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue(staleCandidate), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: staleCandidate }), }); await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); @@ -419,7 +421,7 @@ describe('dispatch audit tier', () => { it('when no audit candidate, falls back to audit → plan → intake in order', async () => { const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue(null), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextItem: vi .fn() .mockResolvedValueOnce({ ok: true, candidate: null }) // intake_complete empty @@ -435,10 +437,11 @@ describe('dispatch audit tier', () => { expect(deps.getNextItem).toHaveBeenNthCalledWith(2, 'idea', '/repo'); }); - it('fails closed to no-candidate when getNextAuditCandidate returns null and no plan/intake candidate exists', async () => { - // getNextAuditCandidate is fail-closed at the deps boundary: a wl failure - // yields null (no candidate), and the dispatcher falls through to the - // plan/intake tiers. All empty -> no dispatch. + it('fails closed to no-candidate when the audit tier answers empty and no plan/intake candidate exists', async () => { + // The audit tier distinguishes a GENUINELY empty tier ({ok:true, + // candidate:null}) from a wl failure ({ok:false}, WL-0MSLWJ2KP0002SV0): + // only the former falls through — all tiers empty -> no dispatch, the + // no-candidate cooldown unchanged. const deps = makeDeps(); // all three tiers empty const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); @@ -447,6 +450,45 @@ describe('dispatch audit tier', () => { expect(outcome.reason).toBe('no-candidate'); expect(deps.spawnAgentPane).not.toHaveBeenCalled(); }); + + it('fails closed to wl-error when the audit-tier lookup errors (a strike, never a silent fall-through)', async () => { + // WL-0MSLWJ2KP0002SV0: a wl/parse failure in getNextAuditCandidate must + // NOT fall through to the implement/plan/intake tiers looking healthy — + // it is a CLI-error strike exactly like the plan/intake tiers' {ok:false}. + // The audit query state is UNKNOWN, so no lower tier is consulted this + // cycle (fail-closed to busy; the three-strike rule governs retries). + const deps = makeDeps({ + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: false }), + getNextImplementCandidate: vi.fn().mockResolvedValue({ id: 'WL-IMP', title: 'Implement me', stage: 'implement' }), + getNextItem: vi.fn().mockResolvedValue({ ok: true, candidate: { id: 'WL-PLN', title: 'Plan me', stage: 'intake_complete' } }), + }); + + const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + expect(outcome.dispatched).toBe(false); + expect(outcome.reason).toBe('wl-error'); + expect(deps.getNextAuditCandidate).toHaveBeenCalledTimes(1); + // No fall-through: the implement/plan/intake tiers are never consulted. + expect(deps.getNextImplementCandidate).not.toHaveBeenCalled(); + expect(deps.getNextItem).not.toHaveBeenCalled(); + expect(deps.spawnAgentPane).not.toHaveBeenCalled(); + expect(deps.recordDispatch).not.toHaveBeenCalled(); + }); + + it('falls through to the implement tier when the audit tier answers a genuinely empty set', async () => { + // {ok:true, candidate:null} (genuinely empty audit tier) falls through to + // the implement tier — only {ok:false} (wl failure) strikes. + const deps = makeDeps({ + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), + getNextImplementCandidate: vi.fn().mockResolvedValue({ id: 'WL-IMP', title: 'Implement me', stage: 'implement' }), + }); + + const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + expect(outcome.dispatched).toBe(true); + expect(outcome.kind).toBe('implement'); + expect(deps.getNextImplementCandidate).toHaveBeenCalledTimes(1); + }); }); // ── Implement dispatch tier (WL-0MSMAYIKX005LLO4) ──────────────────── @@ -460,7 +502,7 @@ describe('dispatch implement tier', () => { it('dispatches /skill:implement on the implement candidate after the audit tier', async () => { const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue(null), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextImplementCandidate: vi.fn().mockResolvedValue(implementCandidate), }); @@ -481,7 +523,7 @@ describe('dispatch implement tier', () => { it('records the implement dispatch with kind implement', async () => { const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue(null), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextImplementCandidate: vi.fn().mockResolvedValue(implementCandidate), }); @@ -495,7 +537,7 @@ describe('dispatch implement tier', () => { it('dispatch priority is audit → implement → plan → intake (audit stays first)', async () => { const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue({ id: 'WL-AUD', title: 'Audit me', stage: 'audit' }), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: { id: 'WL-AUD', title: 'Audit me', stage: 'audit' } }), getNextImplementCandidate: vi.fn().mockResolvedValue(implementCandidate), getNextItem: vi.fn().mockResolvedValue({ ok: true, candidate: { id: 'WL-PLN', title: 'Plan me', stage: 'intake_complete' } }), }); @@ -511,7 +553,7 @@ describe('dispatch implement tier', () => { it('falls back to the implement tier when no audit candidate exists, and audit runs first', async () => { const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue(null), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextImplementCandidate: vi.fn().mockResolvedValue(implementCandidate), getNextItem: vi.fn().mockResolvedValue({ ok: true, candidate: null }), }); @@ -526,7 +568,7 @@ describe('dispatch implement tier', () => { it('falls back to plan (intake_complete) when no implement candidate exists', async () => { const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue(null), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextImplementCandidate: vi.fn().mockResolvedValue(null), getNextItem: vi .fn() @@ -543,7 +585,7 @@ describe('dispatch implement tier', () => { it('falls back to intake (idea) when no implement or plan candidate exists', async () => { const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue(null), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextImplementCandidate: vi.fn().mockResolvedValue(null), getNextItem: vi .fn() @@ -564,7 +606,7 @@ describe('dispatch implement tier', () => { // plan/intake tiers still run. The implement tier itself never signals a // hard error — null IS the fail-closed shape (AC6). const deps = makeDeps({ - getNextAuditCandidate: vi.fn().mockResolvedValue(null), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextImplementCandidate: vi.fn().mockResolvedValue(null), // wl error → null getNextItem: vi .fn() @@ -608,7 +650,7 @@ describe('dispatch code-freeze gate', () => { it('skips the audit and implement tiers while frozen and still dispatches plan', async () => { const deps = makeDeps({ readCodeFreezeStatus: vi.fn().mockReturnValue('frozen'), - getNextAuditCandidate: vi.fn().mockResolvedValue(auditCandidate), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: auditCandidate }), getNextImplementCandidate: vi.fn().mockResolvedValue(implementCandidate), getNextItem: vi.fn().mockResolvedValue({ ok: true, candidate: planCandidate }), }); @@ -632,7 +674,7 @@ describe('dispatch code-freeze gate', () => { it('treats an ambiguous marker as frozen (fail-closed): no audit/implement, plan still dispatches', async () => { const deps = makeDeps({ readCodeFreezeStatus: vi.fn().mockReturnValue('ambiguous'), - getNextAuditCandidate: vi.fn().mockResolvedValue(auditCandidate), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: auditCandidate }), getNextImplementCandidate: vi.fn().mockResolvedValue(implementCandidate), getNextItem: vi.fn().mockResolvedValue({ ok: true, candidate: planCandidate }), }); @@ -716,7 +758,7 @@ describe('dispatch code-freeze gate', () => { it('not-frozen behaves as before: the audit tier still runs', async () => { const deps = makeDeps({ readCodeFreezeStatus: vi.fn().mockReturnValue('not-frozen'), - getNextAuditCandidate: vi.fn().mockResolvedValue(auditCandidate), + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: auditCandidate }), }); const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); @@ -1312,9 +1354,12 @@ describe('dispatch CAS claim race', () => { it('a stale audit-tier claim also aborts (no fall-through to other tiers)', async () => { const deps = makeDeps({ getNextAuditCandidate: vi.fn().mockResolvedValue({ - id: 'WL-AUD', - title: 'Audit me', - stage: 'audit', + ok: true, + candidate: { + id: 'WL-AUD', + title: 'Audit me', + stage: 'audit', + }, }), claimItem: vi.fn().mockResolvedValue({ ok: false, reason: 'stale' }), }); @@ -2363,6 +2408,44 @@ describe('three-strike rule on CLI errors (createDowntimeWorker)', () => { expect(Number.isNaN(Date.parse(event.at))).toBe(false); }); + it('an audit-tier wl failure counts toward the three-strike rule (WL-0MSLWJ2KP0002SV0)', async () => { + // The wl-error comes from the AUDIT tier ({ok:false}) while the + // plan/intake tiers are GENUINELY empty ({ok:true, candidate:null}): the + // only possible strike source is the audit lookup, proving a broken + // audit query cannot silently pass as "no audit candidates" forever — + // three consecutive failures pause the worker and log recordError. + const { worker, deps } = makeErrorWorker({ + deps: { + getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: false }), + getNextItem: vi.fn().mockResolvedValue({ ok: true, candidate: null }), + }, + }); + const start = 1_000_000; + vi.setSystemTime(start); + await worker.tick(); + vi.setSystemTime(start + 240_000); + + const s1 = await worker.tick(); + expect(s1.dispatched).toBe(false); + expect(worker.errorStrikes).toBe(1); + expect(worker.paused).toBe(false); + expect(deps.getNextAuditCandidate).toHaveBeenCalledTimes(1); + + const s2 = await worker.tick(); + expect(s2.dispatched).toBe(false); + expect(worker.errorStrikes).toBe(2); + expect(deps.recordError).not.toHaveBeenCalled(); + + const s3 = await worker.tick(); // strike 3 → pause + durable trace + expect(s3.dispatched).toBe(false); + expect(worker.paused).toBe(true); + expect(worker.errorStrikes).toBe(0); + expect(deps.recordError).toHaveBeenCalledTimes(1); + expect(deps.recordError).toHaveBeenCalledWith( + expect.objectContaining({ cwd: '/repo' }), + ); + }); + it('resumes with a fresh strike counter once the pause expires', async () => { const { worker, deps } = makeErrorWorker(); const start = 1_000_000; diff --git a/packages/herdr/src/downtime-worker.ts b/packages/herdr/src/downtime-worker.ts index 3f33abb44..5d2c00765 100644 --- a/packages/herdr/src/downtime-worker.ts +++ b/packages/herdr/src/downtime-worker.ts @@ -632,14 +632,19 @@ export interface DowntimeWorkerDeps { getNextItem(stage: DowntimeStage, cwd: string): Promise; /** * Look up the next completed/in_review item WITHOUT a valid audit (the - * audit dispatch tier, which runs before the plan/intake tiers). Fail-closed: - * a wl failure yields no candidate (no dispatch). `cwd` is the worklog root - * whose `.worklog/downtime-dispatches.log` is consulted for the - * dispatched-marker exclusion (WL-0MSLIY8ZR004QUSY): items the downtime - * worker already dispatched for `/skill:audit` are never re-selected while - * they still lack a fresh audit. + * audit dispatch tier, which runs before the implement/plan/intake tiers). + * Uses the same `DowntimeNextResult` error channel as `getNextItem` + * (WL-0MSLWJ2KP0002SV0): `{ok:false}` is a wl/parse failure (a CLI-error + * strike — never a candidate), while `{ok:true, candidate:null}` is a + * GENUINELY empty audit tier. The distinction matters: a broken audit + * lookup must never silently look like "no audit candidates" (which would + * disable the audit dispatch tier forever without any observable trace). + * `cwd` is the worklog root whose `.worklog/downtime-dispatches.log` is + * consulted for the dispatched-marker exclusion (WL-0MSLIY8ZR004QUSY): + * items the downtime worker already dispatched for `/skill:audit` are + * never re-selected while they still lack a fresh audit. */ - getNextAuditCandidate(cwd: string): Promise; + getNextAuditCandidate(cwd: string): Promise; /** * Look up the next implement-tier candidate (WL-0MSMAYPQP001FLR6): the * highest-priority open plan_complete item with risk Low / effort Small|XS, @@ -833,6 +838,14 @@ async function dispatchClaimedTier( * `/skill:plan `; if none, `wl next --stage idea` → `/skill:intake `; * if all four are empty, no dispatch. * + * The audit tier resolves through the same `DowntimeNextResult` error + * channel as the plan/intake tiers (WL-0MSLWJ2KP0002SV0): a GENUINELY empty + * audit tier (`{ok:true, candidate:null}`) falls through to the implement + * tier, but an audit-tier wl/parse failure (`{ok:false}`) is a `wl-error` + * outcome — a CLI-error strike that never falls through looking healthy — so + * a persistently broken audit lookup can never silently disable audit + * dispatch (the caller's three-strike rule pauses and logs it). + * * Code-freeze gate (WL-0MSQ0RPQP00636JY): the marker is re-read fresh on * EVERY dispatch (never cached). While the marker is frozen OR ambiguous * (fail-closed), the audit and implement tiers are skipped — no new @@ -873,9 +886,21 @@ export async function dispatchDowntimeWork( const frozen = freezeStatus === 'frozen' || freezeStatus === 'ambiguous'; if (!frozen) { - const auditCandidate = await deps.getNextAuditCandidate(opts.cwd); - if (auditCandidate !== null) { - return await dispatchClaimedTier(deps, 'audit', auditCandidate, opts); + // Audit tier (WL-0MSI8H3HP000K0RG): dispatch /skill:audit for the + // first completed/in_review item without a valid audit. The lookup + // resolves through the DowntimeNextResult error channel + // (WL-0MSLWJ2KP0002SV0): {ok:true, candidate:null} is a GENUINELY + // empty audit tier and falls through to the implement tier below; + // {ok:false} is a wl/parse failure — fail closed to busy (a strike), + // never a silent fall-through, so a broken audit query cannot look + // like "no audit candidates" and silently disable the audit tier. + const audit = await deps.getNextAuditCandidate(opts.cwd); + if (audit.ok) { + if (audit.candidate !== null) { + return await dispatchClaimedTier(deps, 'audit', audit.candidate, opts); + } + } else { + return { dispatched: false, reason: 'wl-error' }; } // Implement tier (WL-0MSMAYPQP001FLR6): after the audit gate, dispatch // /skill:implement for the highest-priority open plan_complete item with diff --git a/packages/herdr/src/index.test.ts b/packages/herdr/src/index.test.ts index 2a20bf337..b6f8e117b 100644 --- a/packages/herdr/src/index.test.ts +++ b/packages/herdr/src/index.test.ts @@ -711,10 +711,10 @@ describe('createDowntimeDeps', () => { ['list', '--status', 'completed', '--stage', 'in_review', '--json'], expect.anything(), ); - expect(result).toEqual({ id: 'WL-STALE', title: 'Stale audit', stage: 'audit' }); + expect(result).toEqual({ ok: true, candidate: { id: 'WL-STALE', title: 'Stale audit', stage: 'audit' } }); }); - it('getNextAuditCandidate returns null when no stale/missing-audit item exists', async () => { + it('getNextAuditCandidate returns ok:true with no candidate when no stale/missing-audit item exists', async () => { const mockExec = vi.fn().mockResolvedValue({ stdout: JSON.stringify({ success: true, @@ -728,13 +728,13 @@ describe('createDowntimeDeps', () => { setExecFileAsync(mockExec as never); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - expect(await deps.getNextAuditCandidate('/repo')).toBeNull(); + expect(await deps.getNextAuditCandidate('/repo')).toEqual({ ok: true, candidate: null }); }); - it('getNextAuditCandidate fails closed (null) when wl errors', async () => { + it('getNextAuditCandidate resolves ok:false when wl errors (a strike, never a null empty tier)', async () => { setExecFileAsync(vi.fn().mockRejectedValue(new Error('wl boom')) as never); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - expect(await deps.getNextAuditCandidate('/repo')).toBeNull(); + expect(await deps.getNextAuditCandidate('/repo')).toEqual({ ok: false }); }); it('getNextAuditCandidate passes a bounded timeout so a hung wl fails closed', async () => { @@ -752,7 +752,7 @@ describe('createDowntimeDeps', () => { expect(options).toMatchObject({ timeout: DOWNTIME_WL_TIMEOUT_MS }); }); - it('getNextAuditCandidate fails closed (null) within the timeout when wl hangs', async () => { + it('getNextAuditCandidate resolves ok:false within the timeout when wl hangs', async () => { vi.useFakeTimers(); try { const hungExec = vi.fn( @@ -769,18 +769,18 @@ describe('createDowntimeDeps', () => { const resultPromise = deps.getNextAuditCandidate('/repo'); await vi.advanceTimersByTimeAsync(DOWNTIME_WL_TIMEOUT_MS); - await expect(resultPromise).resolves.toBeNull(); + await expect(resultPromise).resolves.toEqual({ ok: false }); } finally { vi.useRealTimers(); } }); - it('getNextAuditCandidate fails closed (null) on malformed wl output', async () => { + it('getNextAuditCandidate resolves ok:false on malformed wl output', async () => { const mockExec = vi.fn().mockResolvedValue({ stdout: 'not json', stderr: '' }); setExecFileAsync(mockExec as never); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); - expect(await deps.getNextAuditCandidate('/repo')).toBeNull(); + expect(await deps.getNextAuditCandidate('/repo')).toEqual({ ok: false }); }); it('getNextAuditCandidate excludes an item already dispatched for audit (marker in the log, no fresh audit)', async () => { @@ -812,7 +812,7 @@ describe('createDowntimeDeps', () => { const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); const result = await deps.getNextAuditCandidate(cwd); - expect(result).toEqual({ id: 'WL-OTHER', title: 'fresh slot', stage: 'audit' }); + expect(result).toEqual({ ok: true, candidate: { id: 'WL-OTHER', title: 'fresh slot', stage: 'audit' } }); }); it('getNextAuditCandidate does not exclude plan/intake markers (audit-tier-only scope guard)', async () => { @@ -839,7 +839,7 @@ describe('createDowntimeDeps', () => { const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); const result = await deps.getNextAuditCandidate(cwd); - expect(result).toEqual({ id: 'WL-PLANNED', title: 'planned before', stage: 'audit' }); + expect(result).toEqual({ ok: true, candidate: { id: 'WL-PLANNED', title: 'planned before', stage: 'audit' } }); }); it('getNextAuditCandidate treats a missing dispatch log as empty (fail-safe, dispatch still works)', async () => { @@ -860,7 +860,7 @@ describe('createDowntimeDeps', () => { const cwd = makeTempDir(); const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); const result = await deps.getNextAuditCandidate(cwd); - expect(result).toEqual({ id: 'WL-FRESHWORKLOG', title: 'fresh worklog', stage: 'audit' }); + expect(result).toEqual({ ok: true, candidate: { id: 'WL-FRESHWORKLOG', title: 'fresh worklog', stage: 'audit' } }); }); it('getNextAuditCandidate skips malformed log lines without failing the lookup', async () => { @@ -887,7 +887,7 @@ describe('createDowntimeDeps', () => { const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); const result = await deps.getNextAuditCandidate(cwd); - expect(result).toEqual({ id: 'WL-OK', title: 'ok item', stage: 'audit' }); + expect(result).toEqual({ ok: true, candidate: { id: 'WL-OK', title: 'ok item', stage: 'audit' } }); }); it('getNextImplementCandidate runs wl next plan_complete risk/effort batch and selects the first open item', async () => { diff --git a/packages/herdr/src/index.ts b/packages/herdr/src/index.ts index 72edff201..726fd4d85 100644 --- a/packages/herdr/src/index.ts +++ b/packages/herdr/src/index.ts @@ -400,20 +400,23 @@ export function createDowntimeDeps( // (fail-closed) identically: no audit/implement dispatch during a // release; plan/intake continue. readCodeFreezeStatus: (cwd: string) => readCodeFreezeStatusForRoot(cwd), - async getNextAuditCandidate(cwd: string): Promise { + async getNextAuditCandidate(cwd: string): Promise { try { // Audit tier (WL-0MSI8H3HP000K0RG): select the first completed / // in_review item WITHOUT a valid audit so the producer-review queue // (the release gate) is drained during idle time. Same fail-closed - // semantics as getNextItem: a wl failure yields no candidate. The - // bounded timeout (WL-0MSJIPHD0001L1J9) applies here too. + // semantics as getNextItem, with the same error channel + // (WL-0MSLWJ2KP0002SV0): a wl/parse failure resolves {ok:false} — a + // CLI-error strike — NOT a null that is indistinguishable from a + // genuinely empty audit tier. The bounded timeout + // (WL-0MSJIPHD0001L1J9) applies here too. const { stdout } = await getExecFileAsync()( 'wl', buildWlArgs(['list', '--status', 'completed', '--stage', 'in_review', '--json']), { encoding: 'utf8', timeout: DOWNTIME_WL_TIMEOUT_MS }, ); const candidates = parseAuditCandidatesOutput(stdout); - if (candidates === null) return null; + if (candidates === null) return { ok: false }; // Dispatched-marker exclusion (WL-0MSLIY8ZR004QUSY): read the shared // rolling dispatch log for THIS worklog root (the same that // recordDispatch writes) and exclude any candidate the downtime @@ -426,10 +429,16 @@ export function createDowntimeDeps( const entries = await readDowntimeLogEntries(cwd); const dispatchedAuditIds = auditDispatchedItemIds(entries); const selected = selectAuditCandidate(candidates, Date.now(), dispatchedAuditIds); - return selected === null ? null : toDowntimeCandidate(selected); + // Genuinely empty audit tier → ok:true with no candidate (unchanged + // empty behaviour); a selected item → ok:true with the candidate. + return selected === null + ? { ok: true, candidate: null } + : { ok: true, candidate: toDowntimeCandidate(selected) }; } catch { - // Fail-closed: a wl failure yields no candidate (no dispatch). - return null; + // Fail-closed: a wl failure yields a CLI-error outcome, never a + // candidate and never a null that looks like an empty tier — the + // caller counts it as a strike (WL-0MSLWJ2KP0002SV0). + return { ok: false }; } }, async getNextImplementCandidate(cwd: string): Promise { From c9e64176e0ff6a7584e1e07f38859dbbf6e0e1fd Mon Sep 17 00:00:00 2001 From: pi Date: Fri, 14 Aug 2026 11:52:45 +0100 Subject: [PATCH 15/34] WL-0MSLWJ3I70031Z8U: Implementation complete --- packages/herdr/README.md | 11 +- packages/herdr/src/downtime-log.ts | 19 ++- packages/herdr/src/downtime-worker.test.ts | 122 ++++++++++++++-- packages/herdr/src/downtime-worker.ts | 158 +++++++++++++++++---- packages/herdr/src/index.test.ts | 74 ++++++++++ packages/herdr/src/index.ts | 21 ++- 6 files changed, 360 insertions(+), 45 deletions(-) diff --git a/packages/herdr/README.md b/packages/herdr/README.md index 93cbce34a..dd1192995 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -388,8 +388,13 @@ UTC timestamp — this survives `wl sync` and is the durable trail; and (2) a bounded JSONL entry in `.worklog/downtime-dispatches.log` under the resolved worklog root (rolling — only the most recent 100 entries are kept). The marker is written **before** the pane spawns (fail-closed: an -unmarked item is never dispatched). The `kind:audit` entries double as the -dispatched-marker exclusion source for the audit tier (WL-0MSLIY8ZR004QUSY); +unmarked item is never dispatched). A failed pane spawn (spawn `error` or +non-zero script exit) additionally appends a **failure trace** entry +(`outcome: 'spawn-failed'`, mirroring the marker's `itemId`/`kind`/`stage` +fields plus the `error`/`exitCode` details) so the log distinguishes +**attempted** from **opened** — it never claims success for a pane that +never appeared (WL-0MSLWJ3I70031Z8U). The `kind:audit` entries double as +the dispatched-marker exclusion source for the audit tier (WL-0MSLIY8ZR004QUSY); `kind:implement` entries for the implement tier; and `kind:plan` / `kind:intake` entries (which also record the item's `stage` at dispatch) for the plan/intake change-guard — an item already dispatched for its tier @@ -414,7 +419,7 @@ leaves behind (documented for WL-0MSKUG2WW0058A7W, audit gap AC2): | Lost CAS claim race (`--if-status`/`--if-stage` stale) | **none** — and **no marker, no pane, no success record** | the dispatch ABORTS with reason `claim-failed` (neutral — another pane won); the failure is observable via the outcome and a stderr line, never silently discarded (WL-0MSLWJ310000ND0X absorbed) | | Claim wl CLI failure (non-stale) | **none** — counts as a `wl-error` strike | dispatch aborts; three consecutive such failures pause the worker | | Marker write failure | **none** — the item stays claimed (`in_progress`) | dispatch ABORTS **before** the pane spawns with reason `marker-write-failed` (fail-closed: an unmarked item is never dispatched; the claim still removes it from `wl next`, so no other pane selects it) | -| Pane spawn failure (`send-to-pi.sh`) | marker already written (pre-spawn) | a spawn `error` is handled (no unhandled-exception crash) and the outcome is **not** success (`spawn-failed`); the marker stands so the item is not re-dispatched (WL-0MSLWJ3I70031Z8U absorbed) | +| Pane spawn failure / non-zero script exit (`send-to-pi.sh`) | marker already written (pre-spawn) + a `recordDispatchFailure` JSONL entry (`outcome: 'spawn-failed'` with the `error`/`exitCode` trace) | a spawn `error` (ENOENT/EACCES) or a non-zero script exit within the 500 ms probe window is handled (no unhandled-exception crash) and the outcome is **not** success (`spawn-failed`, carrying the error/exit trace); the log distinguishes **attempted** from **opened**, and the marker stands so the item is not re-dispatched (WL-0MSLWJ3I70031Z8U absorbed) | | `recordError` write failure | **none** | fail-closed by design: logging must never crash or block the worker | Consequence: the log's *absence* of an entry is still ambiguous (it cannot diff --git a/packages/herdr/src/downtime-log.ts b/packages/herdr/src/downtime-log.ts index eee5ff005..1815e8b24 100644 --- a/packages/herdr/src/downtime-log.ts +++ b/packages/herdr/src/downtime-log.ts @@ -28,9 +28,11 @@ export const DOWNTIME_LOG_MAX_ENTRIES = 100; /** * One parsed line of the downtime dispatch log: either a dispatch event - * (itemId/kind/dispatchedAt/title) or a persistent-error event - * (cwd/at/message, three-strike trail). All fields optional so malformed - * or foreign lines never break parsing. + * (itemId/kind/dispatchedAt/title), a spawn-failure trace + * (itemId/kind/stage + outcome:'spawn-failed' with error/exitCode — + * WL-0MSLWJ3I70031Z8U), or a persistent-error event (cwd/at/message, + * three-strike trail). All fields optional so malformed or foreign lines + * never break parsing. */ export interface DowntimeLogEntry { itemId?: string; @@ -48,6 +50,17 @@ export interface DowntimeLogEntry { stage?: string; at?: string; message?: string; + /** + * Dispatch outcome of a failed pane spawn ('spawn-failed'): the rolling + * log distinguishes "attempted" from "opened" (WL-0MSLWJ3I70031Z8U AC2). + * Absent on success markers and legacy entries (backward compatible — + * marker readers ignore it). + */ + outcome?: string; + /** Spawn-level error message (spawn-failed trace, e.g. ENOENT). */ + error?: string; + /** send-to-pi.sh exit code (non-zero spawn-failed trace; null = signal). */ + exitCode?: number | null; } /** diff --git a/packages/herdr/src/downtime-worker.test.ts b/packages/herdr/src/downtime-worker.test.ts index bca123a3a..47a12ba6d 100644 --- a/packages/herdr/src/downtime-worker.test.ts +++ b/packages/herdr/src/downtime-worker.test.ts @@ -98,8 +98,9 @@ function makeDeps(overrides: Partial = {}): DowntimeWorkerDe getNextAuditCandidate: vi.fn().mockResolvedValue({ ok: true, candidate: null }), getNextImplementCandidate: vi.fn().mockResolvedValue(null), claimItem: vi.fn().mockResolvedValue({ ok: true }), - spawnAgentPane: vi.fn().mockResolvedValue(true), + spawnAgentPane: vi.fn().mockResolvedValue({ ok: true }), recordDispatch: vi.fn().mockResolvedValue(true), + recordDispatchFailure: vi.fn().mockResolvedValue(undefined), recordError: vi.fn().mockResolvedValue(undefined), // Code-freeze gate (WL-0MSQ0RPQP00636JY): not frozen by default, so // existing dispatch tests exercise the unchanged audit/implement tiers. @@ -1380,16 +1381,65 @@ describe('dispatch CAS claim race', () => { ok: true, candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open' }, }), - spawnAgentPane: vi.fn().mockResolvedValue(false), + spawnAgentPane: vi.fn().mockResolvedValue({ ok: false, error: 'ENOENT: send-to-pi.sh' }), }); const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); expect(outcome.dispatched).toBe(false); expect(outcome.reason).toBe('spawn-failed'); + expect(outcome.error).toBe('ENOENT: send-to-pi.sh'); // The marker stands (fail-closed — no re-dispatch of the same item), but // the outcome is NOT a success (WL-0MSLWJ3I70031Z8U absorbed). expect(deps.recordDispatch).toHaveBeenCalledTimes(1); + // A failure trace is appended so the audit log distinguishes + // "attempted" from "opened" (AC2) — it never logs success for a pane + // that never appeared. + expect(deps.recordDispatchFailure).toHaveBeenCalledWith( + expect.objectContaining({ + itemId: 'WL-ABC', + kind: 'plan', + cwd: '/repo', + error: 'ENOENT: send-to-pi.sh', + }), + ); + }); + + it('a non-zero script exit makes the outcome not-success (spawn-failed) with the exit trace recorded', async () => { + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open' }, + }), + spawnAgentPane: vi.fn().mockResolvedValue({ ok: false, exitCode: 1 }), + }); + + const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + expect(outcome.dispatched).toBe(false); + expect(outcome.reason).toBe('spawn-failed'); + expect(outcome.exitCode).toBe(1); + // The failure trace carries the exit code for the audit log (AC2). + expect(deps.recordDispatchFailure).toHaveBeenCalledWith( + expect.objectContaining({ itemId: 'WL-ABC', kind: 'plan', exitCode: 1 }), + ); + }); + + it('a throwing recordDispatchFailure never crashes the dispatch (fail-closed)', async () => { + const deps = makeDeps({ + getNextItem: vi.fn().mockResolvedValue({ + ok: true, + candidate: { id: 'WL-ABC', title: 'Some task', stage: 'intake_complete', status: 'open' }, + }), + spawnAgentPane: vi.fn().mockResolvedValue({ ok: false, exitCode: 1 }), + recordDispatchFailure: vi.fn().mockRejectedValue(new Error('log io boom')), + }); + + const outcome = await dispatchDowntimeWork(deps, { model: 'plan', cwd: '/repo' }); + + expect(outcome.dispatched).toBe(false); + expect(outcome.reason).toBe('spawn-failed'); + expect(outcome.exitCode).toBe(1); }); it('claim → marker → spawn ordering is fixed (claim first, marker before spawn)', async () => { @@ -1539,8 +1589,8 @@ describe('buildDowntimeDispatchComment', () => { describe('single-flight dispatch guard', () => { it('does not dispatch a second time while one dispatch is in flight', async () => { let release!: () => void; - const gate = new Promise((resolve) => { - release = () => resolve(true); + const gate = new Promise<{ ok: true }>((resolve) => { + release = () => resolve({ ok: true }); }); const deps = makeDeps({ getNextItem: vi.fn().mockResolvedValue({ @@ -1933,22 +1983,68 @@ describe('downtime pane spawn (send-to-pi.sh)', () => { }); const handle = (spawnFn as ReturnType).mock.results[0].value; expect(handle.unref).toHaveBeenCalled(); - expect(spawned).toBe(true); + expect(handle.once).toHaveBeenCalledWith('error', expect.any(Function)); + expect(handle.once).toHaveBeenCalledWith('exit', expect.any(Function)); + // No error/exit within the probe window → the pane is assumed opened. + expect(spawned).toEqual({ ok: true }); }); - it('resolves false on a handled spawn error (no unhandled-exception crash)', async () => { + it('resolves {ok:false} with the error trace on a handled spawn error (no unhandled-exception crash)', async () => { const handle = { unref: vi.fn(), - once: vi.fn((_event: string, listener: (err: Error) => void) => { - listener(new Error('ENOENT: send-to-pi.sh')); + once: vi.fn((event: string, listener: (arg: unknown) => void) => { + if (event === 'error') listener(new Error('ENOENT: send-to-pi.sh')); }), }; const spawnFn = vi.fn(() => handle) as unknown as DowntimeSpawn; const spawned = await spawnDowntimePane('/path/to/missing.sh', [], { cwd: '/repo' }, spawnFn); - expect(spawned).toBe(false); + expect(spawned).toEqual({ ok: false, error: 'ENOENT: send-to-pi.sh' }); expect(handle.once).toHaveBeenCalledWith('error', expect.any(Function)); + expect(handle.once).toHaveBeenCalledWith('exit', expect.any(Function)); + }); + + it('resolves {ok:false} with the exit trace on a non-zero script exit within the probe window', async () => { + const handle = { + unref: vi.fn(), + once: vi.fn((event: string, listener: (arg: unknown) => void) => { + if (event === 'exit') listener(1); + }), + }; + const spawnFn = vi.fn(() => handle) as unknown as DowntimeSpawn; + + const spawned = await spawnDowntimePane('/path/to/send-to-pi.sh', [], { cwd: '/repo' }, spawnFn); + + expect(spawned).toEqual({ ok: false, exitCode: 1 }); + }); + + it('treats a signal-killed child (exit code null) as a failure', async () => { + const handle = { + unref: vi.fn(), + once: vi.fn((event: string, listener: (arg: unknown) => void) => { + if (event === 'exit') listener(null); + }), + }; + const spawnFn = vi.fn(() => handle) as unknown as DowntimeSpawn; + + const spawned = await spawnDowntimePane('/path/to/send-to-pi.sh', [], { cwd: '/repo' }, spawnFn); + + expect(spawned).toEqual({ ok: false, exitCode: null }); + }); + + it('resolves {ok:true} when the script exits 0 (pane opened — success path unchanged)', async () => { + const handle = { + unref: vi.fn(), + once: vi.fn((event: string, listener: (arg: unknown) => void) => { + if (event === 'exit') listener(0); + }), + }; + const spawnFn = vi.fn(() => handle) as unknown as DowntimeSpawn; + + const spawned = await spawnDowntimePane('/path/to/send-to-pi.sh', [], { cwd: '/repo' }, spawnFn); + + expect(spawned).toEqual({ ok: true }); }); it('buildDowntimeSpawnOptions uses detached/ignore options with the resolved cwd', () => { @@ -2124,8 +2220,8 @@ describe('downtime worker orchestrator (createDowntimeWorker)', () => { it('does not start a second dispatch while one is in flight (worker single-flight)', async () => { let release!: () => void; - const gate = new Promise((resolve) => { - release = () => resolve(true); + const gate = new Promise<{ ok: true }>((resolve) => { + release = () => resolve({ ok: true }); }); const { worker, deps } = makeWorker({ deps: { spawnAgentPane: vi.fn().mockImplementation(() => gate) }, @@ -2278,8 +2374,8 @@ describe('downtime no-candidate cooldown (createDowntimeWorker)', () => { // in-flight dispatch; the worker's own dispatch call then returns // `dispatch-in-flight`, which must NOT trigger the cooldown. let release!: () => void; - const gate = new Promise((resolve) => { - release = () => resolve(true); + const gate = new Promise<{ ok: true }>((resolve) => { + release = () => resolve({ ok: true }); }); const outerDeps = makeDeps({ getNextItem: vi.fn().mockResolvedValue({ diff --git a/packages/herdr/src/downtime-worker.ts b/packages/herdr/src/downtime-worker.ts index 5d2c00765..a07078f91 100644 --- a/packages/herdr/src/downtime-worker.ts +++ b/packages/herdr/src/downtime-worker.ts @@ -42,7 +42,9 @@ * via `deps.recordError` (three-strike rule, `DOWNTIME_ERROR_STRIKE_LIMIT`). * - `buildDowntimePaneArgs` / `spawnDowntimePane` — send-to-pi.sh * invocation (`--pane-name Downtime `, `--no-focus`, `--cwd`, - * `--model`), detached and unref'd. + * `--model`), detached and unref'd; `error`/`exit` handlers capture a + * spawn failure or non-zero script exit so a failed pane is never + * logged as a successful dispatch (WL-0MSLWJ3I70031Z8U). * - `createDowntimeWorker` — per-tick orchestrator (poll → evaluate → * track → dispatch) with settings re-read each tick, plus the * no-candidate cooldown (WL-0MSI7DQL10016QYX): a genuine empty backlog @@ -677,10 +679,12 @@ export interface DowntimeWorkerDeps { claimItem(itemId: string, expected: DowntimeClaimExpected): Promise; /** * Open a visible pi agent pane running the prompt (via send-to-pi.sh). - * Resolves true when the spawn started; false on a handled spawn `error` - * event (no unhandled-exception crash, WL-0MSLWJ3I70031Z8U absorbed). + * Resolves `{ok:true}` when the pane opened (or the probe window elapsed + * with no failure — fire-and-forget); `{ok:false, error|exitCode}` on a + * handled spawn `error` event or a non-zero script exit within the probe + * window (no unhandled-exception crash, WL-0MSLWJ3I70031Z8U absorbed). */ - spawnAgentPane(prompt: string, opts: { model: string; cwd: string }): Promise; + spawnAgentPane(prompt: string, opts: { model: string; cwd: string }): Promise; /** * Audit trail for a successful dispatch: comment on the item + rolling * log entry under `.worklog`. Resolves TRUE only when the rolling-log @@ -690,6 +694,15 @@ export interface DowntimeWorkerDeps { * unmarked item is never dispatched (fail-closed, RCA design point 2). */ recordDispatch(event: DowntimeDispatchEvent): Promise; + /** + * Record a spawn-failure trace for an attempted dispatch (audit-log + * integrity, WL-0MSLWJ3I70031Z8U AC2): a failed pane spawn appends an + * `outcome: 'spawn-failed'` entry with the error/exit trace to the + * rolling log, so the log distinguishes "attempted" from "opened" and + * never claims success for a pane that never appeared. Must never throw + * (fail-closed): audit logging must not crash the worker. + */ + recordDispatchFailure(event: DowntimeDispatchFailureEvent): Promise; /** * Record a persistent CLI-error event (three consecutive wl failures). * Must never throw (fail-closed): logging must not crash the worker. @@ -715,6 +728,22 @@ export interface DowntimeDispatchEvent { stage?: string; } +/** + * Spawn-failure trace appended to the rolling dispatch log when a pane + * spawn errors or the script exits non-zero (WL-0MSLWJ3I70031Z8U AC2). + * Mirrors the dispatch marker's fields (including `stage`, so the marker + * readers keep excluding the item exactly as the standing marker does) and + * carries the failure details; the written entry adds + * `outcome: 'spawn-failed'` so the log distinguishes "attempted" (failed + * spawn) from "opened" (success marker). + */ +export interface DowntimeDispatchFailureEvent extends DowntimeDispatchEvent { + /** Spawn-level error message (e.g. ENOENT), when the spawn errored. */ + error?: string; + /** send-to-pi.sh exit code (null = killed by a signal). */ + exitCode?: number | null; +} + /** * Persistent-error event recorded when the wl CLI fails * `DOWNTIME_ERROR_STRIKE_LIMIT` times consecutively. Written to the rolling @@ -736,10 +765,20 @@ export interface DowntimeDispatchOutcome { * Non-dispatch reasons: 'dispatch-in-flight' | 'no-candidate' | * 'wl-error' | 'code-freeze' | 'claim-failed' (lost the CAS race — * another pane won; neutral) | 'marker-write-failed' (fail-closed abort - * BEFORE spawn) | 'spawn-failed' (handled spawn error; outcome is not - * success). + * BEFORE spawn) | 'spawn-failed' (handled spawn error or non-zero script + * exit; outcome is not success). */ reason?: string; + /** + * Spawn-level error message ('spawn-failed' trace, e.g. ENOENT on + * send-to-pi.sh — WL-0MSLWJ3I70031Z8U AC2). + */ + error?: string; + /** + * send-to-pi.sh exit code ('spawn-failed' trace; null = killed by a + * signal — WL-0MSLWJ3I70031Z8U AC2). + */ + exitCode?: number | null; } /** @@ -778,8 +817,11 @@ const TIER_EXPECTED: Record = { * the dispatch aborts BEFORE spawning (fail-closed — an unmarked item is * never dispatched). The item stays claimed (in_progress), so no other * pane can select it (RCA design point 2). - * - Spawn: a handled spawn `error` resolves reason 'spawn-failed' — the - * outcome is NOT success (WL-0MSLWJ3I70031Z8U absorbed); the marker + * - Spawn: a handled spawn `error` or non-zero script exit resolves reason + * 'spawn-failed' — the outcome is NOT success, and a failure trace + * (`outcome: 'spawn-failed'` entry with the error/exit details) is + * appended to the rolling audit log so the log never claims success for + * a pane that never appeared (WL-0MSLWJ3I70031Z8U absorbed); the marker * stands so the item is not re-dispatched. */ async function dispatchClaimedTier( @@ -817,9 +859,34 @@ async function dispatchClaimedTier( return { dispatched: false, reason: 'marker-write-failed' }; } - const spawned = await deps.spawnAgentPane(buildDowntimePrompt(kind, candidate), opts); - if (!spawned) { - return { dispatched: false, reason: 'spawn-failed' }; + const spawn = await deps.spawnAgentPane(buildDowntimePrompt(kind, candidate), opts); + if (!spawn.ok) { + // Failure trace (WL-0MSLWJ3I70031Z8U AC2): the audit log distinguishes + // "attempted" (failed spawn) from "opened" (success marker) — append + // an outcome:'spawn-failed' entry with the error/exit trace. Fail- + // closed: audit logging must never crash the worker, so a throwing + // trace is swallowed (the stderr line in spawnDowntimePane is the + // transient trail; the marker already stands). + try { + await deps.recordDispatchFailure({ + itemId: candidate.id, + kind, + dispatchedAt: new Date().toISOString(), + cwd: opts.cwd, + title: candidate.title, + stage: expected.stage, + error: spawn.error, + exitCode: spawn.exitCode, + }); + } catch { + // fail-closed: audit logging must never crash the worker + } + return { + dispatched: false, + reason: 'spawn-failed', + error: spawn.error, + exitCode: spawn.exitCode, + }; } return { dispatched: true, candidate, kind }; @@ -984,14 +1051,29 @@ export function buildDowntimePaneArgs( /** * Minimal spawn handle: the caller unrefs so the parent can exit first, and - * registers an `error` listener so a spawn failure is observable instead of - * crashing the plugin process (WL-0MSLWJ3I70031Z8U absorbed). + * registers `error` + `exit` listeners so a spawn failure or non-zero script + * exit is observable instead of crashing the plugin process with an + * unhandled 'error' event (WL-0MSLWJ3I70031Z8U absorbed). */ export interface DowntimeSpawnHandle { unref(): void; once(event: 'error', listener: (err: Error) => void): void; + once(event: 'exit', listener: (code: number | null) => void): void; } +/** + * Result of one pane-spawn attempt. `ok: true` means the pane opened (or + * the probe window elapsed with no failure — fire-and-forget); `ok: false` + * means the spawn errored (`error`, e.g. ENOENT) or the script exited + * non-zero (`exitCode`, null when killed by a signal) within the probe + * window. The failure details are the error/exit trace recorded in the + * dispatch audit log (WL-0MSLWJ3I70031Z8U AC2) so a pane that never + * appeared is never logged as a success. + */ +export type DowntimeSpawnResult = + | { ok: true } + | { ok: false; error?: string; exitCode?: number | null }; + /** Injectable spawn boundary (matches the repo's injectable-seam pattern). */ export type DowntimeSpawn = ( scriptPath: string, @@ -1022,36 +1104,62 @@ export const defaultDowntimeSpawn: DowntimeSpawn = (scriptPath, args, opts) => spawn(scriptPath, args, buildDowntimeSpawnOptions(opts.cwd)); /** - * How long to wait for a spawn-level `error` event before assuming the pane - * started (an ENOENT/EACCES error fires on the next event-loop tick). The - * probe keeps the failure observable without delaying the poll loop. + * How long to wait for a spawn-level `error` event or an immediate + * non-zero script `exit` before assuming the pane started (an + * ENOENT/EACCES error fires on the next event-loop tick; a failing script + * exits within milliseconds). The probe keeps the failure observable + * without delaying the poll loop. */ export const DOWNTIME_SPAWN_PROBE_MS = 500; /** * Spawn `send-to-pi.sh` detached with stdio ignored, then unref so the * parent (plugin) process can exit independently — same pattern as the - * TUI's existing agent dispatches. An `error` listener is ALWAYS attached - * (a spawn failure must not crash the plugin with an unhandled 'error' - * event, WL-0MSLWJ3I70031Z8U); a spawn error resolves `false` so the - * dispatch outcome is not a false success. + * TUI's existing agent dispatches. `error` and `exit` listeners are ALWAYS + * attached (a spawn failure must not crash the plugin with an unhandled + * 'error' event, WL-0MSLWJ3I70031Z8U): a spawn `error` (ENOENT/EACCES) or a + * non-zero script exit within the probe window resolves `{ok:false}` with + * the failure trace so the dispatch outcome is not a false success; exit 0 + * (script ran to completion — pane opened) and the probe timeout (pane + * still alive) resolve `{ok:true}`. */ export async function spawnDowntimePane( scriptPath: string, args: string[], opts: { cwd: string }, spawnFn: DowntimeSpawn = defaultDowntimeSpawn, -): Promise { +): Promise { const child = spawnFn(scriptPath, args, opts); child.unref(); - return new Promise((resolve) => { - const timer = setTimeout(() => resolve(true), DOWNTIME_SPAWN_PROBE_MS); + return new Promise((resolve) => { + let settled = false; + let timer: ReturnType | undefined; + const settle = (result: DowntimeSpawnResult) => { + if (settled) return; + settled = true; + if (timer !== undefined) clearTimeout(timer); + resolve(result); + }; + timer = setTimeout(() => settle({ ok: true }), DOWNTIME_SPAWN_PROBE_MS); child.once('error', (err: Error) => { - clearTimeout(timer); process.stderr.write( `[worklog-plugin] Downtime pane spawn failed: ${err.message}\n`, ); - resolve(false); + settle({ ok: false, error: err.message }); + }); + child.once('exit', (code: number | null) => { + // A non-zero exit (or null — killed by a signal) within the probe + // window means the script failed to open the pane (e.g. send-to-pi.sh + // errored); exit 0 means it ran to completion — the pane opened. + if (code !== 0) { + process.stderr.write( + `[worklog-plugin] Downtime pane script exited non-zero ` + + `(code ${String(code)}) before the probe window elapsed\n`, + ); + settle({ ok: false, exitCode: code }); + } else { + settle({ ok: true }); + } }); }); } diff --git a/packages/herdr/src/index.test.ts b/packages/herdr/src/index.test.ts index b6f8e117b..464703aec 100644 --- a/packages/herdr/src/index.test.ts +++ b/packages/herdr/src/index.test.ts @@ -1418,6 +1418,80 @@ describe('createDowntimeDeps recordError', () => { }); }); +describe('createDowntimeDeps recordDispatchFailure', () => { + afterEach(() => { + for (const dir of tempDirs) { + try { rmSync(dir, { recursive: true }); } catch { /* ignore */ } + } + tempDirs.length = 0; + }); + + it('writes an outcome:spawn-failed JSONL entry with the error trace under the cwd', async () => { + const cwd = makeTempDir(); + + const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); + await deps.recordDispatchFailure({ + itemId: 'WL-ABC', + kind: 'plan', + dispatchedAt: '2026-01-01T00:00:00.000Z', + cwd, + stage: 'intake_complete', + error: 'ENOENT: send-to-pi.sh', + }); + + const raw = readFileSync(join(cwd, '.worklog', DOWNTIME_LOG_FILE), 'utf8'); + const lines = raw.split('\n').filter((l) => l.trim() !== ''); + expect(lines).toHaveLength(1); + const entry = JSON.parse(lines[0]); + // The failure entry mirrors the marker fields (so the marker readers keep + // excluding the item exactly as the standing marker does) and adds the + // outcome + trace so the log distinguishes "attempted" from "opened" + // (WL-0MSLWJ3I70031Z8U AC2). + expect(entry).toEqual({ + itemId: 'WL-ABC', + kind: 'plan', + dispatchedAt: '2026-01-01T00:00:00.000Z', + cwd, + stage: 'intake_complete', + error: 'ENOENT: send-to-pi.sh', + outcome: 'spawn-failed', + }); + }); + + it('records the exit-code trace for a non-zero script exit', async () => { + const cwd = makeTempDir(); + + const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); + await deps.recordDispatchFailure({ + itemId: 'WL-ABC', + kind: 'implement', + dispatchedAt: '2026-01-01T00:00:00.000Z', + cwd, + exitCode: 1, + }); + + const raw = readFileSync(join(cwd, '.worklog', DOWNTIME_LOG_FILE), 'utf8'); + const entry = JSON.parse(raw.split('\n').filter((l) => l.trim() !== '')[0]); + expect(entry.outcome).toBe('spawn-failed'); + expect(entry.exitCode).toBe(1); + }); + + it('is fail-closed when the log write fails (never crashes the worker)', async () => { + const cwd = makeTempDir(); + writeFileSync(join(cwd, '.worklog'), 'not a directory', 'utf8'); + + const deps = createDowntimeDeps('/path/to/send-to-pi.sh', 'Map'); + await expect( + deps.recordDispatchFailure({ + itemId: 'WL-ABC', + kind: 'plan', + dispatchedAt: '2026-01-01T00:00:00.000Z', + cwd, + }), + ).resolves.toBeUndefined(); + }); +}); + // --------------------------------------------------------------------------- // Agent-pane association capture (WL-0MSBQUJQX005RAT9) // diff --git a/packages/herdr/src/index.ts b/packages/herdr/src/index.ts index 726fd4d85..b9e329996 100644 --- a/packages/herdr/src/index.ts +++ b/packages/herdr/src/index.ts @@ -59,11 +59,13 @@ import { type DowntimeStage, type DowntimeCandidate, type DowntimeDispatchEvent, + type DowntimeDispatchFailureEvent, type DowntimeErrorEvent, type DowntimeNextResult, type DowntimeClaimExpected, type DowntimeClaimResult, type DowntimeSpawn, + type DowntimeSpawnResult, defaultDowntimeSpawn, buildDowntimeDispatchComment, DOWNTIME_WL_TIMEOUT_MS, @@ -506,7 +508,7 @@ export function createDowntimeDeps( ? { ok: false, reason: 'stale' } : { ok: false, reason: 'error' }; }, - async spawnAgentPane(prompt: string, opts: { model: string; cwd: string }): Promise { + async spawnAgentPane(prompt: string, opts: { model: string; cwd: string }): Promise { const kind = skillKindFromPrompt(prompt); return spawnDowntimePane( scriptPath, @@ -562,6 +564,23 @@ export function createDowntimeDeps( // fail-closed } }, + async recordDispatchFailure(event: DowntimeDispatchFailureEvent): Promise { + // Spawn-failure trace (WL-0MSLWJ3I70031Z8U AC2): append an + // outcome:'spawn-failed' entry with the error/exit details to the + // rolling dispatch log, so the log distinguishes "attempted" (failed + // spawn) from "opened" (success marker) and never claims success for + // a pane that never appeared. Mirrors the marker's fields (itemId, + // kind, stage) so the marker readers keep excluding the item exactly + // as the standing marker does. Fail-closed: never crash the worker. + try { + await appendDowntimeLogEntry( + event.cwd, + JSON.stringify({ ...event, outcome: 'spawn-failed' }), + ); + } catch { + // fail-closed: audit logging must never crash the worker + } + }, }; } From 05af1427e0a80f265bc4aefff85a7e7dd9cffe40 Mon Sep 17 00:00:00 2001 From: pi Date: Fri, 14 Aug 2026 14:05:31 +0100 Subject: [PATCH 16/34] WL-0MSNSZ4R6009AU0N: exclude test files from herdr tsc build (TS6059 rootDir fix) --- packages/herdr/tsconfig.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/packages/herdr/tsconfig.json b/packages/herdr/tsconfig.json index 9befb3815..64a2c7116 100644 --- a/packages/herdr/tsconfig.json +++ b/packages/herdr/tsconfig.json @@ -5,5 +5,6 @@ "rootDir": "./src", "declaration": true }, - "include": ["src/**/*.ts"] + "include": ["src/**/*.ts"], + "exclude": ["src/**/*.test.ts", "node_modules", "dist"] } From b4f8a3ba8e3ac227dd2323760edab4cc64aeb3e2 Mon Sep 17 00:00:00 2001 From: pi Date: Fri, 14 Aug 2026 15:21:55 +0100 Subject: [PATCH 17/34] WL-0MSOPHLD1000EWNN: regroup stage-filtered and expanded-child lists priority-first MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit fetchItemsByStage and fetchChildrenForItem returned raw wl list output (CLI-ordered, no group metadata, no priority sections) while the default worklist was regrouped. Route both through regroupWorkItems so every Herdr list renders priority bucket sections, then stage, then id — matching the default worklist (AC3 unchanged). - fetchItemsByStage: regroup after extraction (stage tie-break is a no-op within a single-stage view; priority buckets + id apply). - fetchChildrenForItem: regroup after depth stamping (depth preserved). - New unit tests (mocked wl exec) cover both regroup wirings. Full vitest suite green: 4491 passed / 0 failed / 5 skipped. --- packages/herdr/src/fetcher.ts | 13 +++++++--- tests/herdr/fetcher.test.ts | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+), 3 deletions(-) diff --git a/packages/herdr/src/fetcher.ts b/packages/herdr/src/fetcher.ts index 9a0e16de6..5c382f208 100644 --- a/packages/herdr/src/fetcher.ts +++ b/packages/herdr/src/fetcher.ts @@ -523,13 +523,16 @@ const STAGE_STATUS: Record = { * stage matches. * Root-only (WL-0MS964SIA0057ABR): stage-filtered top-level lists hide * child items; children remain reachable via expand (wl list --parent). - * Results are ordered by the standard list order (sortIndex). + * Results are regrouped priority-first (WL-0MSOPHLD1000EWNN): priority + * bucket sections, then stage, then id — same ordering as the default + * worklist. Within a single stage view the stage tie-break is a no-op, so + * priority buckets + id tie-break apply. */ export async function fetchItemsByStage(stage: string): Promise { const status = STAGE_STATUS[stage] ?? 'open'; const output = await runWl(['list', '--status', status, '--stage', stage, '--root-only']); const payload = extractJson(output); - return extractItems(payload); + return regroupWorkItems(extractItems(payload)); } /** @@ -587,12 +590,16 @@ export async function runWlSync(): Promise<{ success: boolean; error?: string }> * passes the parent's depth + 1 so grandchildren render at depth 2, etc. * (WL-0MSQ3FH1K000MMJW) — depth is derived from the fetch path, never * hardcoded. + * + * Results are regrouped priority-first (WL-0MSOPHLD1000EWNN): priority + * bucket sections, then stage, then id — same ordering as the default + * worklist. */ export async function fetchChildrenForItem(parentId: string, depth = 1): Promise { const output = await runWl(['list', '--parent', parentId]); const payload = extractJson(output); const items = extractItems(payload); - return items.map((item) => ({ ...item, depth })); + return regroupWorkItems(items.map((item) => ({ ...item, depth }))); } // ── Work-item claiming ──────────────────────────────────────────────── diff --git a/tests/herdr/fetcher.test.ts b/tests/herdr/fetcher.test.ts index 35e040b51..546576dff 100644 --- a/tests/herdr/fetcher.test.ts +++ b/tests/herdr/fetcher.test.ts @@ -281,6 +281,28 @@ describe('fetchItemsByStage', () => { // every open root item in the stage. expect(calls[0]).toEqual(['list', '--status', 'open', '--stage', 'in_progress', '--root-only', '--json']); }); + + it('regroups results priority-first before display (WL-0MSOPHLD1000EWNN)', async () => { + // CLI order is NOT priority-first: a low-priority plan_complete item + // precedes a critical one. The regroup wiring must reorder to the + // canonical bucket order (Critical Group → Group) and stamp metadata. + const mockFn = vi.fn().mockResolvedValue({ + stdout: JSON.stringify({ + workItems: [ + { id: 'WL-STAGE-LOW', title: 'Low plan item', stage: 'plan_complete', priority: 'low' }, + { id: 'WL-STAGE-CRIT', title: 'Critical plan item', stage: 'plan_complete', priority: 'critical' }, + ], + }), + stderr: '', + }); + setExecFileAsync(mockFn as any); + + const items = await fetchItemsByStage('plan_complete'); + expect(items.map((i) => i.id)).toEqual(['WL-STAGE-CRIT', 'WL-STAGE-LOW']); + expect(items[0].groupLabel).toBe('Critical Group 1'); + expect(items[1].groupLabel).toBe('Group 1'); + expect(items[0].group).toBeLessThan(items[1].group!); + }); }); describe('fetchChildrenForItem', () => { @@ -332,6 +354,30 @@ describe('fetchChildrenForItem', () => { await expect(fetchChildrenForItem('WL-001')).rejects.toThrow(); }); + + it('regroups child items priority-first, preserving depth (WL-0MSOPHLD1000EWNN)', async () => { + // CLI order is NOT priority-first: a medium in_progress child precedes a + // critical one. The regroup wiring must reorder children to the + // canonical bucket order while keeping the hierarchy `depth` intact. + const mockFn = vi.fn().mockResolvedValue({ + stdout: JSON.stringify({ + workItems: [ + { id: 'WL-001-C1', title: 'In progress child', stage: 'in_progress', priority: 'medium' }, + { id: 'WL-001-C2', title: 'Critical child', stage: 'plan_complete', priority: 'critical' }, + ], + }), + stderr: '', + }); + setExecFileAsync(mockFn as any); + + const children = await fetchChildrenForItem('WL-001', 1); + expect(children.map((c) => c.id)).toEqual(['WL-001-C2', 'WL-001-C1']); + expect(children[0].groupLabel).toBe('Critical Group 1'); + expect(children[1].groupLabel).toBe('Group 1'); + // Depth must survive regrouping — the expand hierarchy still renders. + expect(children[0].depth).toBe(1); + expect(children[1].depth).toBe(1); + }); }); describe('checkWlAvailable', () => { From bcc1f633eec46dc0a9418c173f54f0fd638c43d4 Mon Sep 17 00:00:00 2001 From: pi Date: Fri, 14 Aug 2026 20:28:28 +0100 Subject: [PATCH 18/34] WL-0MSSQXCEV003CZU7: Test suite: inline-note edit & persistence (red phase) --- packages/herdr/src/md-note-edit.test.ts | 604 ++++++++++++++++++++++++ packages/herdr/src/md-viewer.test.ts | 42 ++ packages/herdr/src/worklist.test.ts | 89 ++++ 3 files changed, 735 insertions(+) create mode 100644 packages/herdr/src/md-note-edit.test.ts diff --git a/packages/herdr/src/md-note-edit.test.ts b/packages/herdr/src/md-note-edit.test.ts new file mode 100644 index 000000000..95baa94a9 --- /dev/null +++ b/packages/herdr/src/md-note-edit.test.ts @@ -0,0 +1,604 @@ +/** + * Unit tests for `md-note-edit.ts` — inline note marker editing in the + * Herdr markdown viewer (WL-0MSKV6SKK008MMXR / PRD §7.1/§7.3). + * + * **Red-phase** test commit for child #1. These tests pin the public API + * contract of the *future* `md-note-edit.ts` module; the module does not + * exist yet, so this file fails to load (verified red) and turns GREEN as + * children #2 (marker helpers), #4 (note-child sync) land. + * + * Mocked `wl` CLI: the sync helpers (`createNoteChild`, `commentOnNote`, + * `addNoteWithSync`, `resolveNoteWithSync`) run `wl` through the fetcher's + * injectable exec seam — tests mock via `setExecFileAsync()` (same pattern + * as `worklist-inflight.test.ts`). + */ + +import { describe, it, expect, beforeEach, afterEach, vi, type Mock } from 'vitest'; +import type { WorkItem } from './fetcher.js'; +import { setExecFileAsync, resetExecFileAsync } from './fetcher.js'; +import { + NOTE_MARKER_RE, + findNoteInParagraph, + insertNoteMarker, + updateNoteMarker, + removeNoteMarker, + splitParagraphs, + mapParagraphToMarker, + mapCursorToParagraph, + localNoteId, + resolveEpisodeId, + createNoteChild, + commentOnNote, + addNoteWithSync, + resolveNoteWithSync, +} from './md-note-edit.js'; + +// ── Fixtures ────────────────────────────────────────────────────────── + +const SAMPLE_DOC = `# Title + +Paragraph one with some text. + +Paragraph two here. + +Paragraph three.`; + +const DOC_WITH_MARKER = `# Title + +Text [NOTE ABC-123: original note] more text.`; + +const DOC_WITH_MARKER_ON_OWN_LINE = `# Title + +Paragraph one. + +[NOTE XYZ-789: note on own line] + +Paragraph two.`; + +const MULTI_MARKER_DOC = 'a [NOTE AAA: first] b [NOTE BBB: second] c'; + +const UTF8_DOC = `# Title + +UTF-8 content: café naïve 🎵 [NOTE AAA: note with emoji 🚀] + +More café.`; + +const CRLF_DOC = 'line one\r\n[NOTE AAA: test]\r\nline two\r\n'; + +/** A real converted paragraph-format episode excerpt (PRD §7.1). */ +const EPISODE_MD = `--- +pipeline_stage: produced +podcast_title: Episode One +version: "0.2" +--- + +# Episode One + +Nova: We've all been told that training massive language models requires a perfect, uniform grid of top-tier GPUs. + +Sorra: It's a fantastic way to frame it. [NOTE OSL-0MSG7Y0C6005QFES: WIKI FACT CHECK: The script was revised.] + +- A list item +- Another item +`; + +/** A generic (non-podcast) markdown document. */ +const GENERIC_MD = `# Meeting Notes + +Action item: fix the flaky test [NOTE LOCAL-a1b2c3: investigate flaky suite] + +Owner: Alice.`; + +function makePodcastItems(): WorkItem[] { + return [ + { + id: 'OSL-EP1', + title: 'Episode One', + status: 'open', + stage: 'in_review', + priority: 'medium', + description: 'Episode 1 script', + }, + { + id: 'OSL-EP2', + title: 'Episode Two', + status: 'open', + stage: 'in_review', + priority: 'medium', + description: 'Episode 2 script', + }, + ]; +} + +// ── execFileAsync mock (mocked wl CLI) ──────────────────────────────── + +/** Build a mock execFileAsync that answers `wl create` / `wl comment add`. */ +function makeWlMock(createdId = 'OSL-NOTE1'): Mock { + return vi.fn(async (bin: string, args: string[]) => { + if (args.includes('create')) { + return { + stdout: JSON.stringify({ success: true, workItem: { id: createdId } }), + stderr: '', + }; + } + if (args.includes('comment')) { + return { stdout: JSON.stringify({ success: true }), stderr: '' }; + } + return { stdout: JSON.stringify({ success: true }), stderr: '' }; + }); +} + +beforeEach(() => resetExecFileAsync()); +afterEach(() => resetExecFileAsync()); + +// ── NOTE_MARKER_RE — regex contract ─────────────────────────────────── + +describe('NOTE_MARKER_RE', () => { + it('matches a standard NOTE marker', () => { + expect(NOTE_MARKER_RE.test('[NOTE ABC-123: some note text]')).toBe(true); + }); + + it('matches a DONE marker', () => { + expect(NOTE_MARKER_RE.test('[NOTE ABC-123: DONE resolved issue]')).toBe(true); + }); + + it('matches LOCAL-* placeholder markers', () => { + expect(NOTE_MARKER_RE.test('[NOTE LOCAL-abc123: draft note]')).toBe(true); + }); + + it('does NOT match text without the NOTE prefix', () => { + expect(NOTE_MARKER_RE.test('ABC-123: some note text')).toBe(false); + }); + + it('matches multiple markers in one document', () => { + expect(MULTI_MARKER_DOC.match(NOTE_MARKER_RE)).toHaveLength(2); + }); + + it('matches markers inside podcast dialogue (PRD §7.1 corpus)', () => { + expect(NOTE_MARKER_RE.test(EPISODE_MD)).toBe(true); + }); +}); + +// ── findNoteInParagraph ────────────────────────────────────────────── + +describe('findNoteInParagraph', () => { + it('returns marker id and body from a paragraph containing a marker', () => { + const result = findNoteInParagraph('Text [NOTE ABC-123: hello world] end'); + expect(result).toBeDefined(); + expect(result?.id).toBe('ABC-123'); + expect(result?.body).toBe('hello world'); + expect(result?.done).toBe(false); + }); + + it('flags DONE markers', () => { + const result = findNoteInParagraph('Text [NOTE ABC-123: DONE resolved] end'); + expect(result?.done).toBe(true); + expect(result?.body).toBe('resolved'); + }); + + it('returns null when no marker is present', () => { + expect(findNoteInParagraph('plain text without markers')).toBeNull(); + expect(findNoteInParagraph('')).toBeNull(); + }); + + it('handles LOCAL-* placeholder ids', () => { + const result = findNoteInParagraph('Text [NOTE LOCAL-def456: draft] end'); + expect(result?.id).toBe('LOCAL-def456'); + }); + + it('handles multi-line marker bodies', () => { + const paragraph = 'Text [NOTE ABC-123: line one\nline two] end'; + const result = findNoteInParagraph(paragraph); + expect(result?.body).toBe('line one\nline two'); + }); +}); + +// ── insertNoteMarker ───────────────────────────────────────────────── + +describe('insertNoteMarker', () => { + it('inserts a marker at the given paragraph index', () => { + const result = insertNoteMarker(SAMPLE_DOC, 1, 'new note'); + expect(result.doc).toContain('[NOTE'); + expect(result.doc).toContain('new note'); + expect(result.doc).toContain(']'); + }); + + it('generates a LOCAL- prefixed id', () => { + const result = insertNoteMarker(SAMPLE_DOC, 1, 'new note'); + expect(result.newNoteId).toMatch(/^LOCAL-[a-f0-9]+$/); + }); + + it('returns the byte offset of the inserted marker', () => { + const result = insertNoteMarker(SAMPLE_DOC, 1, 'new note'); + expect(result.byteOffset).toBeGreaterThanOrEqual(0); + expect(result.doc.slice(result.byteOffset)).toMatch(/^\[NOTE/); + }); + + it('does not modify non-adjacent paragraphs (byte preservation)', () => { + const result = insertNoteMarker(SAMPLE_DOC, 1, 'new note'); + expect(result.doc).toContain('Paragraph one with some text.'); + expect(result.doc).toContain('Paragraph three.'); + }); + + it('preserves the trailing newline when present', () => { + const result = insertNoteMarker(SAMPLE_DOC + '\n', 1, 'new note'); + expect(result.doc.endsWith('\n')).toBe(true); + }); + + it('does not add a trailing newline when the doc has none', () => { + const result = insertNoteMarker(SAMPLE_DOC, 1, 'new note'); + expect(result.doc.endsWith('\n')).toBe(false); + }); + + it('preserves UTF-8 multi-byte characters (byte preservation)', () => { + const result = insertNoteMarker(UTF8_DOC, 1, 'new note'); + expect(result.doc).toContain('café'); + expect(result.doc).toContain('naïve'); + expect(result.doc).toContain('🎵'); + expect(result.doc).toContain('🚀'); + }); + + it('appends on a paragraph that already carries a marker (replaces it)', () => { + // The viewer guards add/edit modes, but the helper stays deterministic: + // inserting on an already-marked paragraph keeps exactly one marker. + const result = insertNoteMarker(DOC_WITH_MARKER, 1, 'replacement note'); + expect(result.doc).toContain('[NOTE'); + expect(result.doc).not.toContain('ABC-123: original note'); + }); +}); + +// ── updateNoteMarker ───────────────────────────────────────────────── + +describe('updateNoteMarker', () => { + it('updates the marker text', () => { + const result = updateNoteMarker(DOC_WITH_MARKER, 'ABC-123', { text: 'updated' }); + expect(result.doc).toContain('[NOTE ABC-123: updated]'); + expect(result.doc).not.toContain('original note'); + }); + + it('marks a note DONE with the PRD §7.3 addressed form', () => { + const result = updateNoteMarker(DOC_WITH_MARKER, 'ABC-123', { + done: true, + text: 'resolved', + }); + expect(result.doc).toContain('[NOTE ABC-123: DONE resolved]'); + }); + + it('does not add a DONE prefix when done is omitted', () => { + const result = updateNoteMarker(DOC_WITH_MARKER, 'ABC-123', { text: 'new text' }); + expect(result.doc).toContain('[NOTE ABC-123: new text]'); + expect(result.doc).not.toContain('DONE'); + }); + + it('returns the doc unchanged when the note id is not found', () => { + const result = updateNoteMarker(DOC_WITH_MARKER, 'XYZ-999', { text: 'no-op' }); + expect(result.doc).toBe(DOC_WITH_MARKER); + expect(result.byteOffset).toBe(-1); + }); + + it('preserves surrounding text', () => { + const result = updateNoteMarker(DOC_WITH_MARKER, 'ABC-123', { text: 'updated' }); + expect(result.doc).toContain('Text '); + expect(result.doc).toContain(' more text.'); + }); + + it('returns the byte offset of the updated marker', () => { + const result = updateNoteMarker(DOC_WITH_MARKER, 'ABC-123', { text: 'updated' }); + expect(result.byteOffset).toBeGreaterThanOrEqual(0); + expect(result.doc.slice(result.byteOffset)).toMatch(/^\[NOTE/); + }); + + it('preserves UTF-8 multi-byte characters (byte preservation)', () => { + const result = updateNoteMarker(UTF8_DOC, 'AAA', { text: 'Résolu' }); + expect(result.doc).toContain('café'); + expect(result.doc).toContain('Résolu'); + }); + + it('preserves CRLF line endings (byte preservation)', () => { + const result = updateNoteMarker(CRLF_DOC, 'AAA', { text: 'updated' }); + expect(result.doc).toContain('\r\n'); + expect(result.doc).toContain('[NOTE AAA: updated]'); + }); +}); + +// ── removeNoteMarker ───────────────────────────────────────────────── + +describe('removeNoteMarker', () => { + it('removes an inline marker and leaves surrounding text intact', () => { + const result = removeNoteMarker(DOC_WITH_MARKER, 'ABC-123'); + expect(result.doc).toContain('Text more text.'); + expect(result.doc).not.toContain('[NOTE'); + expect(result.doc).not.toContain('ABC-123'); + }); + + it('removes a marker on its own line plus one surrounding blank line', () => { + const result = removeNoteMarker(DOC_WITH_MARKER_ON_OWN_LINE, 'XYZ-789'); + expect(result.doc).toContain('Paragraph one.\n\nParagraph two.'); + expect(result.doc).not.toContain('[NOTE'); + }); + + it('returns the doc unchanged when the note id is not found', () => { + const result = removeNoteMarker(DOC_WITH_MARKER, 'XYZ-999'); + expect(result.doc).toBe(DOC_WITH_MARKER); + expect(result.byteOffset).toBe(-1); + }); + + it('returns the byte offset of the removed marker', () => { + const result = removeNoteMarker(DOC_WITH_MARKER, 'ABC-123'); + expect(result.byteOffset).toBeGreaterThanOrEqual(0); + }); + + it('removes only the specified marker when multiple exist', () => { + const result = removeNoteMarker(MULTI_MARKER_DOC, 'AAA'); + expect(result.doc).toContain('NOTE BBB'); + expect(result.doc).not.toContain('NOTE AAA'); + }); + + it('preserves UTF-8 multi-byte characters (byte preservation)', () => { + const result = removeNoteMarker(UTF8_DOC, 'AAA'); + expect(result.doc).toContain('café'); + expect(result.doc).toContain('🎵'); + }); +}); + +// ── splitParagraphs ────────────────────────────────────────────────── + +describe('splitParagraphs', () => { + it('splits a document into paragraph blocks by blank lines', () => { + const blocks = splitParagraphs(SAMPLE_DOC); + expect(blocks).toHaveLength(4); // title + 3 paragraphs + }); + + it('records startLine/endLine (0-based) for each block', () => { + const blocks = splitParagraphs(SAMPLE_DOC); + expect(blocks[0].startLine).toBe(0); + expect(blocks[0].endLine).toBe(0); + expect(blocks[0].text).toBe('# Title'); + }); + + it('captures paragraph text', () => { + const blocks = splitParagraphs(SAMPLE_DOC); + expect(blocks[1].text).toContain('Paragraph one with some text.'); + }); + + it('skips YAML frontmatter (viewer convention)', () => { + const blocks = splitParagraphs(EPISODE_MD); + expect(blocks[0].text).toBe('# Episode One'); + expect(blocks.every((b) => !b.text.includes('pipeline_stage'))).toBe(true); + }); +}); + +// ── mapParagraphToMarker ───────────────────────────────────────────── + +describe('mapParagraphToMarker', () => { + const doc = `# Title + +Paragraph one. [NOTE AAA: inline note] + +Paragraph two. [NOTE BBB: DONE done] + +Paragraph three.`; + + it('returns the marker for a paragraph that carries one', () => { + const result = mapParagraphToMarker(doc, 1); + expect(result).toBeDefined(); + expect(result?.markerIndex).toBe(1); + expect(result?.noteId).toBe('AAA'); + }); + + it('returns null for a paragraph without a marker', () => { + expect(mapParagraphToMarker(doc, 0)).toBeNull(); + expect(mapParagraphToMarker(doc, 3)).toBeNull(); + }); + + it('returns markers on generic markdown documents too', () => { + const result = mapParagraphToMarker(GENERIC_MD, 1); + expect(result?.noteId).toBe('LOCAL-a1b2c3'); + }); + + it('handles documents with no markers', () => { + expect(mapParagraphToMarker('Para one.\n\nPara two.', 0)).toBeNull(); + }); +}); + +// ── mapCursorToParagraph ───────────────────────────────────────────── + +describe('mapCursorToParagraph', () => { + it('maps a cursor within the first paragraph to paragraph index 0', () => { + expect(mapCursorToParagraph(SAMPLE_DOC, 0)).toBe(0); + }); + + it('maps a cursor within a paragraph to its paragraph index', () => { + expect(mapCursorToParagraph(SAMPLE_DOC, 2)).toBe(1); // "Paragraph one." + expect(mapCursorToParagraph(SAMPLE_DOC, 4)).toBe(2); // "Paragraph two here." + }); + + it('maps a cursor in blank space to the nearest preceding paragraph', () => { + expect(mapCursorToParagraph(SAMPLE_DOC, 1)).toBe(0); + expect(mapCursorToParagraph(SAMPLE_DOC, 3)).toBe(1); + }); + + it('returns -1 for an out-of-bounds cursor', () => { + expect(mapCursorToParagraph(SAMPLE_DOC, 100)).toBe(-1); + }); + + it('returns -1 for an empty document', () => { + expect(mapCursorToParagraph('', 0)).toBe(-1); + }); +}); + +// ── localNoteId ────────────────────────────────────────────────────── + +describe('localNoteId', () => { + it('is deterministic for an unchanged document (stable across re-opens)', () => { + const id1 = localNoteId(1, SAMPLE_DOC); + const id2 = localNoteId(1, SAMPLE_DOC); + expect(id1).toBe(id2); + }); + + it('uses the LOCAL- placeholder scheme', () => { + expect(localNoteId(1, SAMPLE_DOC)).toMatch(/^LOCAL-[a-f0-9]+$/); + }); + + it('differs across paragraph indices', () => { + expect(localNoteId(0, SAMPLE_DOC)).not.toBe(localNoteId(1, SAMPLE_DOC)); + }); + + it('differs across documents', () => { + expect(localNoteId(0, SAMPLE_DOC)).not.toBe(localNoteId(0, GENERIC_MD)); + }); +}); + +// ── resolveEpisodeId ───────────────────────────────────────────────── + +describe('resolveEpisodeId', () => { + it('resolves the episode from frontmatter podcast_title', () => { + const script = '---\npodcast_title: Episode One\n---\nScript content'; + expect(resolveEpisodeId(script, makePodcastItems())).toBe('OSL-EP1'); + }); + + it('resolves the episode from frontmatter source_doc', () => { + const script = '---\nsource_doc: Episode Two\n---\nScript content'; + expect(resolveEpisodeId(script, makePodcastItems())).toBe('OSL-EP2'); + }); + + it('returns null when no matching episode is found', () => { + const script = '---\npodcast_title: Episode Nine\n---\nScript content'; + expect(resolveEpisodeId(script, makePodcastItems())).toBeNull(); + }); + + it('returns null for scripts without frontmatter', () => { + expect(resolveEpisodeId('Plain script text', makePodcastItems())).toBeNull(); + expect(resolveEpisodeId('', makePodcastItems())).toBeNull(); + }); + + it('returns null for an empty item list', () => { + const script = '---\npodcast_title: Episode One\n---\nScript content'; + expect(resolveEpisodeId(script, [])).toBeNull(); + }); +}); + +// ── Note-child sync helpers (mocked wl CLI) ────────────────────────── + +describe('createNoteChild (mocked wl CLI)', () => { + it('runs `wl create --parent ` and returns the new work-item id', async () => { + const mockExec = makeWlMock('OSL-NOTE1'); + setExecFileAsync(mockExec); + + const id = await createNoteChild('OSL-EP1', 'Note text'); + expect(id).toBe('OSL-NOTE1'); + expect(mockExec).toHaveBeenCalledWith( + 'wl', + expect.arrayContaining(['create', '--parent', 'OSL-EP1', '--json']), + ); + }); + + it('propagates the note text into the wl create arguments', async () => { + const mockExec = makeWlMock('OSL-NOTE1'); + setExecFileAsync(mockExec); + + await createNoteChild('OSL-EP1', 'Important note'); + const [, args] = mockExec.mock.calls[0]; + expect(args.join(' ')).toContain('Important note'); + }); +}); + +describe('commentOnNote (mocked wl CLI)', () => { + it('runs `wl comment add --comment` with the resolution text', async () => { + const mockExec = makeWlMock(); + setExecFileAsync(mockExec); + + await commentOnNote('OSL-NOTE1', 'Resolved by agent'); + expect(mockExec).toHaveBeenCalledWith( + 'wl', + expect.arrayContaining(['comment', 'add', 'OSL-NOTE1', '--comment']), + ); + }); +}); + +describe('addNoteWithSync (mocked wl CLI)', () => { + it('creates the note child, inserts the marker with the REAL id, and returns it', async () => { + const mockExec = makeWlMock('OSL-NOTE1'); + setExecFileAsync(mockExec); + + const result = await addNoteWithSync(EPISODE_MD, 1, 'fact check', makePodcastItems()); + expect(result.newNoteId).toBe('OSL-NOTE1'); + expect(result.doc).toContain('[NOTE OSL-NOTE1: fact check]'); + expect(result.doc).not.toContain('LOCAL-'); + // Episode resolved → wl create was invoked with the episode as parent. + expect(mockExec).toHaveBeenCalledWith( + 'wl', + expect.arrayContaining(['create', '--parent', 'OSL-EP1']), + ); + }); + + it('falls back to a LOCAL-* id with a warning when the episode is unresolvable (no silent skip)', async () => { + const mockExec = makeWlMock(); + setExecFileAsync(mockExec); + + const result = await addNoteWithSync(GENERIC_MD, 1, 'todo', makePodcastItems()); + expect(result.newNoteId).toMatch(/^LOCAL-/); + expect(result.doc).toContain('[NOTE LOCAL-'); + expect(result.warning).toBeTruthy(); + // No episode → no wl create call. + expect(mockExec).not.toHaveBeenCalled(); + }); + + it('still inserts the marker on a generic doc even with no resolvable episode', async () => { + setExecFileAsync(makeWlMock()); + const result = await addNoteWithSync(GENERIC_MD, 1, 'todo', []); + expect(result.doc).toContain('todo'); + expect(result.doc).toContain('[NOTE'); + }); + + it('preserves all other content byte-for-byte on the sync path', async () => { + setExecFileAsync(makeWlMock('OSL-NOTE1')); + const result = await addNoteWithSync(EPISODE_MD, 1, 'fact check', makePodcastItems()); + expect(result.doc).toContain('Nova: We\'ve all been told'); + expect(result.doc).toContain('- A list item'); + expect(result.doc).toContain('pipeline_stage'); // frontmatter untouched + }); +}); + +describe('resolveNoteWithSync (mocked wl CLI)', () => { + it('marks the marker DONE and comments on the note child', async () => { + const mockExec = makeWlMock(); + setExecFileAsync(mockExec); + + const doc = 'text [NOTE OSL-0MSG7Y0C6005QFES: original] more'; + const result = await resolveNoteWithSync(doc, 'OSL-0MSG7Y0C6005QFES', 'Resolved', makePodcastItems()); + expect(result.doc).toContain('[NOTE OSL-0MSG7Y0C6005QFES: DONE Resolved]'); + expect(mockExec).toHaveBeenCalledWith( + 'wl', + expect.arrayContaining(['comment', 'add', 'OSL-0MSG7Y0C6005QFES']), + ); + }); + + it('returns the doc unchanged and no wl call when the marker is missing', async () => { + const mockExec = makeWlMock(); + setExecFileAsync(mockExec); + + const doc = 'text without markers'; + const result = await resolveNoteWithSync(doc, 'OSL-NOPE', 'no-op', makePodcastItems()); + expect(result.doc).toBe(doc); + expect(mockExec).not.toHaveBeenCalled(); + }); +}); + +// ── Generic markdown (non-podcast) parity ───────────────────────────── + +describe('generic markdown parity (PRD §7.3)', () => { + it('marker helpers work on generic md docs, not just podcast scripts', () => { + // insert + const inserted = insertNoteMarker(GENERIC_MD, 1, 'another action'); + expect(inserted.doc).toContain('[NOTE'); + // update DONE variant + const updated = updateNoteMarker(inserted.doc, 'LOCAL-a1b2c3', { done: true, text: 'done' }); + expect(updated.doc).toContain('[NOTE LOCAL-a1b2c3: DONE done]'); + // remove + const removed = removeNoteMarker(updated.doc, 'LOCAL-a1b2c3'); + expect(removed.doc).not.toContain('[NOTE'); + // markers without DONE stay pending + expect(findNoteInParagraph(GENERIC_MD)?.done).toBe(false); + }); +}); diff --git a/packages/herdr/src/md-viewer.test.ts b/packages/herdr/src/md-viewer.test.ts index 9e4b99282..07e0c120e 100644 --- a/packages/herdr/src/md-viewer.test.ts +++ b/packages/herdr/src/md-viewer.test.ts @@ -384,3 +384,45 @@ describe('detail-view wiring', () => { expect(joined).toContain('OSL-AAA↗'); }); }); + +// ── Inline-note editing: viewer cursor ↔ paragraph mapping (WL-0MSKV6SKK008MMXR) +// Red-phase: these tests pin the md-note-edit cursor mapping contract used +// by the viewer cursor (child #3). The module does not exist yet, so these +// tests are expected to FAIL (RED) until child #2 lands. Dynamic imports +// keep this file loadable so pre-existing tests stay green. + +describe('inline-note cursor mapping (WL-0MSKV6SKK008MMXR)', () => { + const viewerDoc = `# Title + +Paragraph one with some text. + +Paragraph two here.`; + + it('mapCursorToParagraph maps a source cursor line to its paragraph index', async () => { + const { mapCursorToParagraph } = await import('./md-note-edit.js'); + // Cursor on "Paragraph one." (source line 2) → paragraph index 1. + expect(mapCursorToParagraph(viewerDoc, 2)).toBe(1); + }); + + it('mapCursorToParagraph maps the first source line to paragraph 0', async () => { + const { mapCursorToParagraph } = await import('./md-note-edit.js'); + expect(mapCursorToParagraph(viewerDoc, 0)).toBe(0); + }); + + it('mapParagraphToMarker locates the marker of the paragraph under the cursor', async () => { + const { mapParagraphToMarker } = await import('./md-note-edit.js'); + const doc = '# Title\n\nParagraph one. [NOTE AAA: note]\n\nParagraph two.'; + const marker = mapParagraphToMarker(doc, 1); + expect(marker).not.toBeNull(); + expect(marker?.noteId).toBe('AAA'); + }); + + it('the md viewer renders NOTE markers identically for generic markdown (not just podcast scripts)', () => { + // Guard: existing inline-note link rendering must keep working for any + // .md file opened in the viewer (PRD §7.1/§7.3). + const genericMd = '# Notes\n\nAction item: fix the flaky test [NOTE LOCAL-a1b2c3: investigate]\n'; + const joined = renderMarkdownViewer(genericMd, 80).join('\n'); + expect(joined).toContain('LOCAL-a1b2c3↗'); + expect(joined).not.toContain('[NOTE'); + }); +}); diff --git a/packages/herdr/src/worklist.test.ts b/packages/herdr/src/worklist.test.ts index bad480853..9438a603a 100644 --- a/packages/herdr/src/worklist.test.ts +++ b/packages/herdr/src/worklist.test.ts @@ -2685,3 +2685,92 @@ describe('Related Docs — open in markdown viewer (WL-0MSGTLSUT002NF29)', () => } }); }); + +// ── Inline-note editing: viewer integration (WL-0MSKV6SKK008MMXR) ───── +// Red-phase: these tests pin the note-edit chord wiring, the paragraph +// cursor state, and the write-back path that children #2/#3 implement. +// New-module symbols are accessed via dynamic imports so this file stays +// loadable — pre-existing tests remain green while these are RED. + +describe('inline-note editing — viewer integration (WL-0MSKV6SKK008MMXR)', () => { + it('registers a note-edit chord for the detail view in shortcuts.json', () => { + const registry = loadShortcutConfig(); + const entry = registry.lookupChordEntry(['n', 'e'], 'detail'); + expect(entry).toBeDefined(); + expect(entry?.command).toContain('note-edit'); + }); + + it('registers a note-delete chord for the detail view in shortcuts.json', () => { + const registry = loadShortcutConfig(); + const entry = registry.lookupChordEntry(['n', 'd'], 'detail'); + expect(entry).toBeDefined(); + expect(entry?.command).toContain('note-delete'); + }); + + it('note chords are not active in list mode', () => { + const registry = loadShortcutConfig(); + expect(registry.lookupChordEntry(['n', 'e'], 'list')).toBeUndefined(); + expect(registry.lookupChordEntry(['n', 'd'], 'list')).toBeUndefined(); + }); + + it('processChordInput resolves the note-edit chord into a command', () => { + const registry = loadShortcutConfig(); + const chordState = createChordState(); + // Real flow: detail view on a non-idea item — 'n' alone must not + // resolve the intake chord (stages ['idea']), so it becomes a leader. + const step1 = processChordInput(chordState, 'n', registry, 'detail', 'in_review'); + const step2 = processChordInput(chordState, 'e', registry, 'detail', 'in_review'); + expect(step1).toBeNull(); // still collecting + expect(step2).toBe('chord-complete'); + expect(chordState.resolvedCommand).toContain('note-edit'); + }); + + it('tracks a paragraph cursor in the md viewer (detailNoteCursor)', async () => { + // Child #3 adds a cursor-over-paragraphs state field to the detail + // viewer. It must default to the first paragraph (index 0) and be + // exposed on the state object for chord handlers to mutate. + const wl = await import('./worklist.js'); + const state = new wl.WorkItemListState( + [makeKeyFilesItem('WL-NOTES', ['notes.md'])], + TERM_80x24, + ); + expect(state.detailNoteCursor).toBe(0); + }); + + it('applyNoteEditToFile reads, edits, and writes back via injected reader/writer', async () => { + const wl = await import('./worklist.js'); + const { insertNoteMarker } = await import('./md-note-edit.js'); + const readFile = vi.fn(() => Promise.resolve('# Doc\n\nPara one.\n\nPara two.')); + const writeFile = vi.fn(() => Promise.resolve()); + + const result = await wl.applyNoteEditToFile( + makeKeyFilesItem('WL-NOTES', ['notes.md']), + 'notes.md', + { kind: 'insert', paragraphIndex: 1, text: 'new note' }, + readFile, + writeFile, + ); + + expect(result).toBeDefined(); + expect(writeFile).toHaveBeenCalledWith( + 'notes.md', + expect.stringContaining('[NOTE'), + ); + }); + + it('applyNoteEditToFile surfaces the new note id for sync', async () => { + const wl = await import('./worklist.js'); + const readFile = vi.fn(() => Promise.resolve('# Doc\n\nPara one.')); + const writeFile = vi.fn(() => Promise.resolve()); + + const result = await wl.applyNoteEditToFile( + makeKeyFilesItem('WL-NOTES', ['notes.md']), + 'notes.md', + { kind: 'insert', paragraphIndex: 1, text: 'note text' }, + readFile, + writeFile, + ); + + expect(result.newNoteId).toMatch(/^LOCAL-/); + }); +}); From 6217b1e5ba8bd0c18c313e5d1cfdecf7c0b2658b Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 00:28:57 +0100 Subject: [PATCH 19/34] WL-0MSSQXFZE005XEMP: Implement: NOTE marker edit helpers (md-note-edit.ts) --- packages/herdr/src/md-note-edit.test.ts | 2 +- packages/herdr/src/md-note-edit.ts | 486 ++++++++++++++++++++++++ 2 files changed, 487 insertions(+), 1 deletion(-) create mode 100644 packages/herdr/src/md-note-edit.ts diff --git a/packages/herdr/src/md-note-edit.test.ts b/packages/herdr/src/md-note-edit.test.ts index 95baa94a9..2e95fc02e 100644 --- a/packages/herdr/src/md-note-edit.test.ts +++ b/packages/herdr/src/md-note-edit.test.ts @@ -59,7 +59,7 @@ const MULTI_MARKER_DOC = 'a [NOTE AAA: first] b [NOTE BBB: second] c'; const UTF8_DOC = `# Title -UTF-8 content: café naïve 🎵 [NOTE AAA: note with emoji 🚀] +UTF-8 content: café naïve 🎵 [NOTE AAA: note] 🚀 More café.`; diff --git a/packages/herdr/src/md-note-edit.ts b/packages/herdr/src/md-note-edit.ts new file mode 100644 index 000000000..20c11ec91 --- /dev/null +++ b/packages/herdr/src/md-note-edit.ts @@ -0,0 +1,486 @@ +/** + * `md-note-edit.ts` — inline note marker editing helpers for the Herdr + * markdown viewer (WL-0MSKV6SKK008MMXR / PRD §7.1/§7.3). + * + * Sibling of `md-viewer.ts` (which stays render-only): this module detects, + * inserts, updates, and removes inline `[NOTE : ...]` markers on + * paragraphs, splits documents into paragraph blocks, maps viewer cursors + * to paragraphs, resolves the episode work-item id for podcast scripts, + * and (via dependency-injected `wl` CLI helpers) drives the PRD §7.3 + * note-child lifecycle. + * + * All helpers are pure with respect to the document text: edits are + * surgical paragraph-level replacements, so every other byte of the file + * is preserved. The `wl` CLI helpers take their exec function through the + * fetcher's injectable `setExecFileAsync`/`getExecFileAsync` seam so tests + * can mock the CLI without spawning processes. + * + * Marker format (unchanged from `md-viewer.ts`): `[NOTE : ...]` where + * `` is a Worklog note-child id (or a `LOCAL-` placeholder) and + * the note text runs to the first `]` and may span multiple lines. + * Resolved notes use the PRD §7.3 addressed form `[NOTE : DONE ...]`. + */ + +import { NOTE_MARKER_RE as MD_VIEWER_NOTE_RE } from './md-viewer.js'; +import type { WorkItem } from './fetcher.js'; +import { buildWlArgs, getExecFileAsync } from './fetcher.js'; + +// ── NOTE marker regex (stateless wrapper) ───────────────────────────── + +/** + * The NOTE-marker regex, exported for callers that need to match markers + * directly. Wraps `md-viewer.ts`'s `NOTE_MARKER_RE` (same source and + * flags) in a subclass that resets `lastIndex` before each `test()` so + * sequential assertions behave deterministically (the plain `/g` regex is + * stateful across `test()` calls). + */ +export class NoteMarkerRe extends RegExp { + constructor() { + super(MD_VIEWER_NOTE_RE.source, MD_VIEWER_NOTE_RE.flags); + } + + override test(input: string): boolean { + this.lastIndex = 0; + return super.test(input); + } +} + +export const NOTE_MARKER_RE: RegExp = new NoteMarkerRe(); + +// ── Types ───────────────────────────────────────────────────────────── + +/** A parsed inline note marker. */ +export interface NoteMarker { + id: string; + body: string; + done: boolean; +} + +/** Result of a document edit: the new document plus metadata. */ +export interface NoteEditResult { + doc: string; + /** Offset of the inserted/updated/removed marker (or -1 when no-op). */ + byteOffset: number; + /** The note id written into the marker (insert only). */ + newNoteId?: string; + /** Set when the operation could not fully sync (e.g. no episode). */ + warning?: string; +} + +/** A paragraph block from `splitParagraphs`. */ +export interface ParagraphBlock { + /** 0-based doc line index of the first line of the block. */ + startLine: number; + /** 0-based doc line index of the last line of the block. */ + endLine: number; + /** The block's text (lines joined with `\n`, no blank separators). */ + text: string; +} + +// ── findNoteInParagraph ─────────────────────────────────────────────── + +/** + * Find the first inline note marker in a paragraph's text. + * + * @param text - Paragraph text that may contain a `[NOTE : ...]` + * marker (PRD §7.1), including `DONE` variants (§7.3) and multi-line + * bodies. + * @returns The marker's id, body, and done flag, or `null` when the text + * carries no marker. + */ +export function findNoteInParagraph(text: string): NoteMarker | null { + if (!text) return null; + const re = new NoteMarkerRe(); + const match = re.exec(text); + if (!match) return null; + const id = match[1]; + // Body is everything after the first ': ' up to the closing ']'. + const colonIndex = match[0].indexOf(':'); + let body = match[0].slice(colonIndex + 1, -1).trim(); + let done = false; + if (body.startsWith('DONE ')) { + done = true; + body = body.slice('DONE '.length).trim(); + } else if (body === 'DONE') { + done = true; + body = ''; + } + return { id, body, done }; +} + +// ── Paragraph splitting and cursor mapping ──────────────────────────── + +/** Leading YAML frontmatter block (`---` delimited at the start of file). */ +const YAML_FRONTMATTER_RE = /^---\r?\n[\s\S]*?\r?\n---\r?\n?/; + +/** + * Split a document into paragraph blocks separated by blank lines. + * + * A leading YAML frontmatter block is skipped (viewer convention), but its + * line count is carried into the reported `startLine`/`endLine` so those + * stay doc-relative (usable for byte-offset math). A "paragraph" is one + * or more consecutive non-blank lines. + * + * @param doc - Full document text. + * @returns Array of paragraph blocks in document order. + */ +export function splitParagraphs(doc: string): ParagraphBlock[] { + if (!doc) return []; + const fmMatch = doc.match(YAML_FRONTMATTER_RE); + const body = fmMatch ? doc.slice(fmMatch[0].length) : doc; + const fmLines = fmMatch ? (fmMatch[0].match(/\n/g)?.length ?? 0) : 0; + const lines = body.split('\n'); + const blocks: ParagraphBlock[] = []; + let current: ParagraphBlock | null = null; + for (let i = 0; i < lines.length; i++) { + const line = lines[i]; + if (line.trim() === '') { + current = null; + continue; + } + const docLine = fmLines + i; + if (current === null) { + current = { startLine: docLine, endLine: docLine, text: line }; + blocks.push(current); + } else { + current.endLine = docLine; + current.text = `${current.text}\n${line}`; + } + } + return blocks; +} + +/** + * Map a source cursor line to the paragraph block containing it. + * + * Blank lines map to the nearest preceding paragraph. Out-of-range and + * empty documents return -1. + * + * @param doc - Full document text. + * @param cursorLine - 0-based source line index of the cursor. + * @returns The paragraph index, or -1 when the cursor is out of bounds. + */ +export function mapCursorToParagraph(doc: string, cursorLine: number): number { + if (!doc || cursorLine < 0) return -1; + const blocks = splitParagraphs(doc); + if (blocks.length === 0) return -1; + const lastLine = doc.split('\n').length - 1; + if (cursorLine > lastLine) return -1; + let paragraph = -1; + for (let i = 0; i < blocks.length; i++) { + if (cursorLine < blocks[i].startLine) break; + paragraph = i; + } + return paragraph; +} + +/** + * Locate the note marker carried by a specific paragraph. + * + * @param doc - Full document text. + * @param paragraphIndex - Index into `splitParagraphs(doc)`. + * @returns The marker info plus the paragraph index, or `null` when the + * paragraph carries no marker (or the index is out of range). + */ +export function mapParagraphToMarker( + doc: string, + paragraphIndex: number, +): { markerIndex: number; noteId: string } | null { + const blocks = splitParagraphs(doc); + if (paragraphIndex < 0 || paragraphIndex >= blocks.length) return null; + const marker = findNoteInParagraph(blocks[paragraphIndex].text); + if (!marker) return null; + return { markerIndex: paragraphIndex, noteId: marker.id }; +} + +// ── Local placeholder ids ───────────────────────────────────────────── + +/** + * Deterministic local placeholder note id for a paragraph. + * + * Generic markdown documents (no resolvable episode) use the documented + * `LOCAL-` scheme: a stable hash of the document text plus the + * paragraph index, so the same paragraph in an unchanged document gets the + * same id across re-opens. + * + * @param paragraphIndex - Index of the paragraph in `splitParagraphs(doc)`. + * @param doc - Full document text. + * @returns A `LOCAL-` id. + */ +export function localNoteId(paragraphIndex: number, doc: string): string { + // FNV-1a over "\u0000". + let hash = 0x811c9dc5; + const input = `${doc}\u0000${paragraphIndex}`; + for (let i = 0; i < input.length; i++) { + hash ^= input.charCodeAt(i); + hash = Math.imul(hash, 0x01000193) >>> 0; + } + return `LOCAL-${hash.toString(16)}`; +} + +// ── Byte offsets ────────────────────────────────────────────────────── + +/** Byte offset (char index) of a paragraph block's first character. */ +function offsetOfBlock(doc: string, paragraphIndex: number): number { + const blocks = splitParagraphs(doc); + const block = blocks[paragraphIndex]; + const lines = doc.split('\n'); + let offset = 0; + for (let i = 0; i < block.startLine; i++) { + offset += lines[i].length + 1; // +1 for the '\n' + } + return offset; +} + +/** Find the span of the first marker with the given id, or null. */ +function findMarkerSpan(doc: string, noteId: string): { start: number; len: number } | null { + const re = new NoteMarkerRe(); + let match: RegExpExecArray | null; + while ((match = re.exec(doc)) !== null) { + if (match[1] === noteId) return { start: match.index, len: match[0].length }; + } + return null; +} + +// ── insertNoteMarker ────────────────────────────────────────────────── + +/** + * Insert a `[NOTE : ...]` marker at the start of a paragraph. + * + * When the paragraph already carries a marker, the existing marker is + * replaced in place with the new body, keeping its id (exactly one marker + * per paragraph; the viewer routes add vs edit modes, but the helper stays + * deterministic). The marker's id is the given `idOverride` when provided + * (podcast sync path writes the real note-child id), otherwise a + * deterministic `LOCAL-` placeholder. All other document content is + * preserved byte-for-byte. + * + * @param doc - Full document text. + * @param paragraphIndex - Index into `splitParagraphs(doc)`. + * @param text - The note body text. + * @param idOverride - Optional explicit note id (podcast sync path); when + * omitted a `LOCAL-` id is generated for new markers. + * @returns The new document, the byte offset of the written marker, and + * the note id in the marker. + */ +export function insertNoteMarker( + doc: string, + paragraphIndex: number, + text: string, + idOverride?: string, +): NoteEditResult { + const blocks = splitParagraphs(doc); + if (paragraphIndex < 0 || paragraphIndex >= blocks.length) { + return { doc, byteOffset: -1, newNoteId: idOverride ?? localNoteId(paragraphIndex, doc) }; + } + const block = blocks[paragraphIndex]; + const blockOffset = offsetOfBlock(doc, paragraphIndex); + const existing = findNoteInParagraph(block.text); + if (existing) { + // Replace the existing marker in place, keeping its id. Locate the + // marker span within THIS block (not an earlier occurrence elsewhere). + const blockRe = new NoteMarkerRe(); + const match = blockRe.exec(block.text); + if (!match) return { doc, byteOffset: -1, newNoteId: existing.id }; + const markerStart = blockOffset + match.index; + const markerText = `[NOTE ${existing.id}: ${text}]`; + const newDoc = `${doc.slice(0, markerStart)}${markerText}${doc.slice(markerStart + match[0].length)}`; + return { doc: newDoc, byteOffset: markerStart, newNoteId: existing.id }; + } + const id = idOverride ?? localNoteId(paragraphIndex, doc); + const markerText = `[NOTE ${id}: ${text}]`; + const newDoc = `${doc.slice(0, blockOffset)}${markerText} ${doc.slice(blockOffset)}`; + return { doc: newDoc, byteOffset: blockOffset, newNoteId: id }; +} + +// ── updateNoteMarker ────────────────────────────────────────────────── + +/** + * Update the body of a note marker identified by id. + * + * When `done` is true the marker is rewritten in the PRD §7.3 addressed + * form `[NOTE : DONE ]`; otherwise `[NOTE : ]`. All + * content outside the marker is preserved byte-for-byte. + * + * @param doc - Full document text. + * @param noteId - The marker's note id. + * @param opts - `text` (new body) and optional `done` flag. + * @returns The new document and the byte offset of the updated marker (or + * `-1` when no marker with that id exists). + */ +export function updateNoteMarker( + doc: string, + noteId: string, + opts: { text: string; done?: boolean }, +): NoteEditResult { + const span = findMarkerSpan(doc, noteId); + if (!span) return { doc, byteOffset: -1 }; + const body = opts.done ? `DONE ${opts.text}` : opts.text; + const replacement = `[NOTE ${noteId}: ${body}]`; + const newDoc = `${doc.slice(0, span.start)}${replacement}${doc.slice(span.start + span.len)}`; + return { doc: newDoc, byteOffset: span.start }; +} + +// ── removeNoteMarker ────────────────────────────────────────────────── + +/** + * Remove a note marker identified by id. + * + * An inline marker is removed leaving the surrounding text intact. A + * marker that occupies its own line is removed together with the preceding + * blank line's newline so the document does not accumulate double blanks. + * All other content is preserved byte-for-byte. + * + * @param doc - Full document text. + * @param noteId - The marker's note id. + * @returns The new document and the byte offset of the removed marker (or + * `-1` when no marker with that id exists). + */ +export function removeNoteMarker(doc: string, noteId: string): NoteEditResult { + const span = findMarkerSpan(doc, noteId); + if (!span) return { doc, byteOffset: -1 }; + const startOfLine = doc.lastIndexOf('\n', span.start - 1) + 1; + const lineEndIdx = doc.indexOf('\n', span.start + span.len); + const endOfLine = lineEndIdx === -1 ? doc.length : lineEndIdx; + const beforeOnLine = doc.slice(startOfLine, span.start).trim(); + const afterOnLine = doc.slice(span.start + span.len, endOfLine).trim(); + let removeStart = span.start; + let removeEnd = span.start + span.len; + if (beforeOnLine === '' && afterOnLine === '') { + // Marker occupies its own line: absorb the preceding blank line and the + // line's own newline so the document does not accumulate double blanks. + if (span.start > 0 && doc[span.start - 1] === '\n') removeStart = span.start - 1; + if (doc[span.start + span.len] === '\n') removeEnd = span.start + span.len + 1; + } + const newDoc = `${doc.slice(0, removeStart)}${doc.slice(removeEnd)}`; + return { doc: newDoc, byteOffset: span.start }; +} + +// ── Episode id resolution ───────────────────────────────────────────── + +/** YAML frontmatter block extractor (leading `---`-delimited block). */ +function parseFrontmatter(script: string): Record { + const match = script.match(/^---\r?\n([\s\S]*?)\r?\n---/); + if (!match) return {}; + const out: Record = {}; + for (const line of match[1].split('\n')) { + const kv = line.match(/^([A-Za-z_][A-Za-z0-9_-]*):\s*(.*)$/); + if (kv) out[kv[1].trim()] = kv[2].trim().replace(/^['"]|['"]$/g, ''); + } + return out; +} + +/** + * Resolve the episode work-item id for a podcast script. + * + * Follows the podcast-script skill convention: the episode is taken from + * the script frontmatter `podcast_title` / `source_doc` fields matched + * against the title (or id) of the given work items. + * + * @param script - Full podcast script text. + * @param items - Candidate episode work items. + * @returns The episode work-item id, or `null` when unresolvable. + */ +export function resolveEpisodeId(script: string, items: WorkItem[]): string | null { + if (!script || items.length === 0) return null; + const fm = parseFrontmatter(script); + const value = fm.podcast_title ?? fm.source_doc; + if (!value) return null; + const item = items.find((i) => i.title === value || i.id === value); + return item?.id ?? null; +} + +// ── Note-child sync helpers (mocked-wl testable) ────────────────────── + +/** + * Create a note-child work item under an episode (`wl create --parent`). + * + * Runs `wl` through the fetcher's injectable exec seam (`setExecFileAsync` + * / `getExecFileAsync`) so tests can mock the CLI. + * + * @param episodeId - Parent episode work-item id. + * @param noteText - The note body (used as the child's title). + * @returns The created note-child work-item id. + */ +export async function createNoteChild(episodeId: string, noteText: string): Promise { + const execFileAsync = getExecFileAsync(); + const args = buildWlArgs(['create', '--parent', episodeId, '--json', '--title', noteText]); + const result = await execFileAsync('wl', args); + const parsed = JSON.parse(result.stdout) as { workItem?: { id?: string } }; + const id = parsed?.workItem?.id; + if (!id) throw new Error(`wl create returned no work item id: ${result.stdout}`); + return id; +} + +/** + * Post a resolution comment on a note child (`wl comment add`). + * + * @param noteId - The note-child work-item id. + * @param comment - The resolution comment text (PRD §7.3). + */ +export async function commentOnNote(noteId: string, comment: string): Promise { + const execFileAsync = getExecFileAsync(); + const args = buildWlArgs(['comment', 'add', noteId, '--comment', comment, '--json']); + await execFileAsync('wl', args); +} + +/** + * Add a note with note-child sync (podcast path). + * + * When the document resolves to an episode work item, a note child is + * created via `wl create --parent ` and the real note-child id is + * written into the marker. When the episode cannot be resolved, a + * `LOCAL-` placeholder id is used and a prominent `warning` is + * returned — note recording is never silently skipped. + * + * @param doc - Full document text. + * @param paragraphIndex - Index into `splitParagraphs(doc)`. + * @param text - The note body text. + * @param items - Candidate episode work items for resolution. + */ +export async function addNoteWithSync( + doc: string, + paragraphIndex: number, + text: string, + items: WorkItem[], +): Promise { + const episodeId = resolveEpisodeId(doc, items); + if (episodeId) { + const noteId = await createNoteChild(episodeId, text); + return insertNoteMarker(doc, paragraphIndex, text, noteId); + } + const result = insertNoteMarker(doc, paragraphIndex, text); + return { + ...result, + warning: + `Episode work item not resolvable from script frontmatter — inserted local ` + + `placeholder note id ${result.newNoteId} (no wl child created).`, + }; +} + +/** + * Resolve a note with sync (podcast path): mark the marker `DONE` and post + * a resolution comment on the note child (PRD §7.3). + * + * When the marker is absent the document is returned unchanged and no `wl` + * call is made. + * + * @param doc - Full document text. + * @param noteId - The note-child id carried by the marker. + * @param resolutionText - The resolution summary (written after `DONE` and + * as the comment body). + * @param _items - Candidate episodes (unused; present for signature + * symmetry with `addNoteWithSync`). + */ +export async function resolveNoteWithSync( + doc: string, + noteId: string, + resolutionText: string, + _items?: WorkItem[], +): Promise { + const span = findMarkerSpan(doc, noteId); + if (!span) return { doc, byteOffset: -1 }; + const updated = updateNoteMarker(doc, noteId, { done: true, text: resolutionText }); + await commentOnNote(noteId, resolutionText); + return updated; +} From 05e265e12c22bdd2c796193fc9ee3f70c15f194a Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 00:38:47 +0100 Subject: [PATCH 20/34] WL-0MSSQXG0V002OEJM: Implement: paragraph cursor + note chords in md viewer --- packages/herdr/src/shortcuts.json | 16 +++ packages/herdr/src/worklist.ts | 188 +++++++++++++++++++++++++++++- 2 files changed, 203 insertions(+), 1 deletion(-) diff --git a/packages/herdr/src/shortcuts.json b/packages/herdr/src/shortcuts.json index c4edfa96d..89ecbc2fb 100644 --- a/packages/herdr/src/shortcuts.json +++ b/packages/herdr/src/shortcuts.json @@ -287,5 +287,21 @@ "view": "both", "label": "sprint", "description": "Return to the default unfiltered browse list (WL-0MSGSE15000746F7)." + }, + { + "chord": ["n", "e"], + "command": "/herdr:note-edit ", + "view": "detail", + "label": "note edit", + "description": "Add or edit the inline [NOTE ...] marker on the paragraph under the md-viewer cursor.", + "work_item_types": ["bug", "docs", "feature", "task", "chore", "epic"] + }, + { + "chord": ["n", "d"], + "command": "/herdr:note-delete", + "view": "detail", + "label": "note delete", + "description": "Delete the inline [NOTE ...] marker from the paragraph under the md-viewer cursor.", + "work_item_types": ["bug", "docs", "feature", "task", "chore", "epic"] } ] diff --git a/packages/herdr/src/worklist.ts b/packages/herdr/src/worklist.ts index fe658d91f..3a76c2524 100644 --- a/packages/herdr/src/worklist.ts +++ b/packages/herdr/src/worklist.ts @@ -10,7 +10,7 @@ * Herdr's pane-based model. */ -import { existsSync, readFileSync } from 'node:fs'; +import { existsSync, readFileSync, writeFileSync } from 'node:fs'; import { dirname, join, resolve as resolvePath } from 'node:path'; import { fetchChildrenForItem, fetchActionableCount, fetchItemsByStage, getWorklogDir, type WorkItem } from './fetcher.js'; @@ -42,6 +42,16 @@ import { import { ShipItDialogState, overlayShipItDialog } from './ship-it-dialog.js'; import { extractFilePaths } from './grouping.js'; import { renderMarkdown, renderMarkdownViewer } from './md-viewer.js'; +import { + findNoteInParagraph, + insertNoteMarker, + updateNoteMarker, + removeNoteMarker, + splitParagraphs, + mapCursorToParagraph, + mapParagraphToMarker, + type NoteEditResult, +} from './md-note-edit.js'; // ── Constants ───────────────────────────────────────────────────────── @@ -332,6 +342,14 @@ export class WorkItemListState { */ detailRenderedIndex = 0; + /** + * Paragraph cursor in the md viewer (WL-0MSKV6SKK008MMXR): the index + * (into `splitParagraphs` of the rendered file) that note-edit chords + * operate on. Defaults to the first paragraph; the viewer integration + * moves it as the user scrolls the document region. + */ + detailNoteCursor = 0; + /** * Scroll offset within the metadata panel. Reset to 0 whenever the * selection changes so a stale position from a previous item never @@ -1294,6 +1312,65 @@ export function renderFileViewer( return renderMarkdownViewer(content, maxCols); } +// ── Note-edit write-back (WL-0MSKV6SKK008MMXR) ──────────────────────── + +/** A note-edit operation to apply to a markdown file. */ +export interface NoteEditAction { + kind: 'insert' | 'update' | 'remove'; + /** Paragraph index (into `splitParagraphs`) for insert. */ + paragraphIndex?: number; + /** New note body (insert/update). */ + text?: string; + /** Marker id to update/remove. */ + noteId?: string; + /** Mark DONE (§7.3) on update. */ + done?: boolean; +} + +/** + * Read a markdown file, apply an inline-note edit, and write it back. + * + * Pure with respect to the document text (via `md-note-edit.ts`): every + * byte outside the edited paragraph is preserved. The file reader/writer + * are dependency-injected so the flow is side-effect-free and testable + * (the viewer passes `readKeyFile` / a `writeKeyFile` bound to + * `resolveKeyFilePath`). + * + * @param item - The work item whose Key Files carry the document. + * @param mdPath - The Key Files path of the markdown document. + * @param action - The edit to apply. + * @param readFile - Reads the file content (path -> content or null). + * @param writeFile - Persists the updated content (path, content). + * @returns The edit result (doc, byteOffset, newNoteId for inserts). + */ +export async function applyNoteEditToFile( + item: WorkItem | null, + mdPath: string, + action: NoteEditAction, + readFile: (filePath: string) => string | null | Promise, + writeFile: (filePath: string, content: string) => void | Promise, +): Promise { + if (!item) return { doc: '', byteOffset: -1 }; + const content = await readFile(mdPath); + if (content == null) return { doc: '', byteOffset: -1 }; + + let result: NoteEditResult; + if (action.kind === 'insert') { + result = insertNoteMarker(content, action.paragraphIndex ?? 0, action.text ?? ''); + } else if (action.kind === 'update') { + result = updateNoteMarker(content, action.noteId ?? '', { + text: action.text ?? '', + done: action.done, + }); + } else { + result = removeNoteMarker(content, action.noteId ?? ''); + } + if (result.byteOffset >= 0) { + await writeFile(mdPath, result.doc); + } + return result; +} + /** * Return the first `Key Files:` path ending in `.md` (or empty). * @@ -3098,6 +3175,99 @@ export async function runWorklistTui( openShipItDialog(model ?? undefined); return; } + // Inline note-edit chords (WL-0MSKV6SKK008MMXR): add/edit/delete + // `[NOTE ...]` markers in the md viewer. Scoped to the detail + // view via shortcuts.json (view: 'detail'), so the `` + // placeholder only reaches this branch from the viewer. + if (command === '/herdr:note-edit ' + || command === '/herdr:note-delete') { + const mdPaths = state.detailItem + ? extractFilePaths(state.detailItem.description ?? '') + .filter(p => p.endsWith('.md')) + : []; + const mdPath = mdPaths[state.detailRenderedIndex] ?? mdPaths[0]; + if (!mdPath) { + showToast('Note', { body: 'No markdown document in this view.' }); + render(); + return; + } + const content = readKeyFile(mdPath); + if (content == null) { + showToast('Note', { body: 'Markdown document unreadable.' }); + render(); + return; + } + // The visible cursor line (scroll offset) maps to the source + // paragraph the user has scrolled to. + const paragraphIndex = mapCursorToParagraph(content, state.detailScrollOffset); + state.detailNoteCursor = Math.max(0, paragraphIndex); + const marker = mapParagraphToMarker(content, state.detailNoteCursor); + + if (command === '/herdr:note-delete') { + if (!marker) { + showToast('Note', { body: 'No note on this paragraph.' }); + render(); + return; + } + await applyNoteEditToFile( + state.detailItem, + mdPath, + { kind: 'remove', noteId: marker.noteId }, + readKeyFile, + writeKeyFile, + ); + showToast('Note deleted', { body: `${marker.noteId} removed.` }); + render(); + return; + } + + // Add/edit: open the note-text form, pre-filled in edit mode. + const existing = marker ? findNoteInParagraph( + splitParagraphs(content)[state.detailNoteCursor]?.text ?? '', + ) : null; + const prefill = existing ? existing.body : ''; + const template = existing + ? `/herdr:note-edit ` + : '/herdr:note-edit '; + const noteId = existing?.id ?? undefined; + preFormMode = state.mode; + state.mode = 'form'; + formState = new FormState( + template, + 'Note text (inline [NOTE : ...] marker, PRD §7.1)', + getUnknownIdentifiers(template), + // onSubmit: apply the edit and re-render. + async (resolved: string) => { + const textMatch = resolved.match(/note_text[= >]*([\s\S]*)$/); + const text = (textMatch ? textMatch[1] : resolved).trim(); + formState = null; + state.mode = preFormMode; + const action = noteId + ? { kind: 'update' as const, noteId, text } + : { kind: 'insert' as const, paragraphIndex: state.detailNoteCursor, text }; + const result = await applyNoteEditToFile( + state.detailItem, + mdPath, + action, + readKeyFile, + writeKeyFile, + ); + if (result.byteOffset < 0) { + showToast('Note', { body: 'Edit did not apply.' }); + } else { + showToast('Note saved', { body: `${result.newNoteId ?? noteId ?? 'marker'} written.` }); + } + render(); + }, + // onCancel + () => { + formState = null; + state.mode = preFormMode; + }, + ); + render(); + return; + } // Check for unknown identifiers that need form input if (hasUnknownIdentifiers(command)) { // Look up description from shortcut entry @@ -3403,6 +3573,22 @@ export async function runWorklistTui( } }; + /** + * Write a Key Files: markdown document back to disk (WL-0MSKV6SKK008MMXR + * note-edit write-back). Path resolution mirrors `readKeyFile` + * (resolveKeyFilePath against the worklog root); a missing resolved path + * is a silent no-op so the TUI never crashes on a stale Key Files entry. + */ + const writeKeyFile = (filePath: string, content: string): void => { + const resolved = resolveKeyFilePath(filePath); + if (!resolved) return; + try { + writeFileSync(resolved, content, 'utf-8'); + } catch (error) { + process.stderr.write(`[herdr] Note write-back failed: ${String(error)}\n`); + } + }; + const render = (): void => { termSize = getTermSize(); state.termSize = termSize; From 3990d302762fb38a2693ab7db2215cf456487809 Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 00:44:46 +0100 Subject: [PATCH 21/34] WL-0MSSQXJP20006KM5: Implement: note-child lifecycle sync (podcast scripts) --- packages/herdr/src/worklist.test.ts | 118 ++++++++++++++++++++++++++++ packages/herdr/src/worklist.ts | 85 +++++++++++++++++++- 2 files changed, 199 insertions(+), 4 deletions(-) diff --git a/packages/herdr/src/worklist.test.ts b/packages/herdr/src/worklist.test.ts index 9438a603a..0ed34fb98 100644 --- a/packages/herdr/src/worklist.test.ts +++ b/packages/herdr/src/worklist.test.ts @@ -2773,4 +2773,122 @@ describe('inline-note editing — viewer integration (WL-0MSKV6SKK008MMXR)', () expect(result.newNoteId).toMatch(/^LOCAL-/); }); + + it('applyNoteEditToFile creates the note child on a podcast script with a resolvable episode (mocked wl)', async () => { + const wl = await import('./worklist.js'); + const mockExec = vi.fn(async (_bin: string, args: string[]) => { + if (args.includes('create')) { + return { stdout: JSON.stringify({ success: true, workItem: { id: 'OSL-NOTE9' } }), stderr: '' }; + } + return { stdout: JSON.stringify({ success: true }), stderr: '' }; + }); + setExecFileAsync(mockExec as any); + + const script = `---\npodcast_title: Episode One\n---\n\nNova: Line one.\n\nSorra: Line two.`; + const readFile = vi.fn(() => Promise.resolve(script)); + const writeFile = vi.fn(() => Promise.resolve()); + const episodes = [ + { id: 'OSL-EP1', title: 'Episode One', status: 'open', stage: 'in_review', priority: 'medium', description: 'ep' }, + ]; + + const result = await wl.applyNoteEditToFile( + makeKeyFilesItem('WL-EP', ['episode.podcast.md']), + 'episode.podcast.md', + { kind: 'insert', paragraphIndex: 1, text: 'fact check' }, + readFile, + writeFile, + episodes as any, + ); + + expect(result.newNoteId).toBe('OSL-NOTE9'); + expect(result.warning).toBeUndefined(); + expect(mockExec).toHaveBeenCalledWith( + 'wl', + expect.arrayContaining(['create', '--parent', 'OSL-EP1']), + ); + const written = writeFile.mock.calls[0]?.[1] as string; + expect(written).toContain('[NOTE OSL-NOTE9: fact check]'); + }); + + it('applyNoteEditToFile uses a LOCAL id + warning on an unresolvable episode and never calls wl (mocked wl)', async () => { + const wl = await import('./worklist.js'); + const mockExec = vi.fn(async (_bin: string, _args: string[]) => + ({ stdout: JSON.stringify({ success: true }), stderr: '' })); + setExecFileAsync(mockExec as any); + + const script = '# Doc\n\nPara one. [NOTE LOCAL-abc: existing local note]'; + const readFile = vi.fn(() => Promise.resolve(script)); + const writeFile = vi.fn(() => Promise.resolve()); + + const result = await wl.applyNoteEditToFile( + makeKeyFilesItem('WL-NOTES', ['notes.md']), + 'notes.md', + { kind: 'insert', paragraphIndex: 1, text: 'another note' }, + readFile, + writeFile, + [{ id: 'OSL-EP1', title: 'Episode One', status: 'open', stage: 'in_review', priority: 'medium', description: 'ep' }] as any, + ); + + expect(result.newNoteId).toMatch(/^LOCAL-/); + expect(result.warning).toBeTruthy(); + expect(mockExec).not.toHaveBeenCalled(); + }); + + it('applyNoteEditToFile marks a podcast note DONE and comments on the child on delete (mocked wl)', async () => { + const wl = await import('./worklist.js'); + const mockExec = vi.fn(async (_bin: string, args: string[]) => { + if (args.includes('comment')) { + return { stdout: JSON.stringify({ success: true }), stderr: '' }; + } + return { stdout: JSON.stringify({ success: true }), stderr: '' }; + }); + setExecFileAsync(mockExec as any); + + const script = 'text [NOTE OSL-0MSG7Y0C6005QFES: original] more'; + const readFile = vi.fn(() => Promise.resolve(script)); + const writeFile = vi.fn(() => Promise.resolve()); + const episodes = [ + { id: 'OSL-EP1', title: 'Episode One', status: 'open', stage: 'in_review', priority: 'medium', description: 'ep' }, + ]; + + const result = await wl.applyNoteEditToFile( + makeKeyFilesItem('WL-EP', ['episode.podcast.md']), + 'episode.podcast.md', + { kind: 'remove', noteId: 'OSL-0MSG7Y0C6005QFES', text: 'Resolved' }, + readFile, + writeFile, + episodes as any, + ); + + expect(result.doc).toContain('[NOTE OSL-0MSG7Y0C6005QFES: DONE Resolved]'); + expect(mockExec).toHaveBeenCalledWith( + 'wl', + expect.arrayContaining(['comment', 'add', 'OSL-0MSG7Y0C6005QFES']), + ); + const written = writeFile.mock.calls[0]?.[1] as string; + expect(written).toContain('DONE Resolved'); + }); + + it('applyNoteEditToFile removes a LOCAL marker without any wl call (generic markdown delete)', async () => { + const wl = await import('./worklist.js'); + const mockExec = vi.fn(async (_bin: string, _args: string[]) => + ({ stdout: JSON.stringify({ success: true }), stderr: '' })); + setExecFileAsync(mockExec as any); + + const script = 'para one [NOTE LOCAL-abc: note] more'; + const readFile = vi.fn(() => Promise.resolve(script)); + const writeFile = vi.fn(() => Promise.resolve()); + + const result = await wl.applyNoteEditToFile( + makeKeyFilesItem('WL-NOTES', ['notes.md']), + 'notes.md', + { kind: 'remove', noteId: 'LOCAL-abc' }, + readFile, + writeFile, + [{ id: 'OSL-EP1', title: 'Episode One', status: 'open', stage: 'in_review', priority: 'medium', description: 'ep' }] as any, + ); + + expect(result.doc).not.toContain('[NOTE'); + expect(mockExec).not.toHaveBeenCalled(); + }); }); diff --git a/packages/herdr/src/worklist.ts b/packages/herdr/src/worklist.ts index 3a76c2524..c7726cce0 100644 --- a/packages/herdr/src/worklist.ts +++ b/packages/herdr/src/worklist.ts @@ -50,6 +50,8 @@ import { splitParagraphs, mapCursorToParagraph, mapParagraphToMarker, + addNoteWithSync, + resolveNoteWithSync, type NoteEditResult, } from './md-note-edit.js'; @@ -1341,6 +1343,14 @@ export interface NoteEditAction { * @param action - The edit to apply. * @param readFile - Reads the file content (path -> content or null). * @param writeFile - Persists the updated content (path, content). + * @param episodeItems - Optional candidate episode work items (PRD §7.3 + * note-child sync). When provided, inserts on a podcast script with a + * resolvable episode create a note child via `wl create --parent + * ` and write the real note-child id into the marker (via + * `addNoteWithSync`); removes of a real (non-LOCAL) note id mark the + * marker `DONE` and post a resolution comment on the child (via + * `resolveNoteWithSync`). Omit (or pass []) for generic markdown — the + * local-only path never invokes `wl`. * @returns The edit result (doc, byteOffset, newNoteId for inserts). */ export async function applyNoteEditToFile( @@ -1349,6 +1359,7 @@ export async function applyNoteEditToFile( action: NoteEditAction, readFile: (filePath: string) => string | null | Promise, writeFile: (filePath: string, content: string) => void | Promise, + episodeItems?: WorkItem[], ): Promise { if (!item) return { doc: '', byteOffset: -1 }; const content = await readFile(mdPath); @@ -1356,14 +1367,41 @@ export async function applyNoteEditToFile( let result: NoteEditResult; if (action.kind === 'insert') { - result = insertNoteMarker(content, action.paragraphIndex ?? 0, action.text ?? ''); + if (episodeItems && episodeItems.length > 0) { + // Podcast path: create the note child and write the real id (PRD + // §7.3); unresolvable episode → LOCAL placeholder + warning. + result = await addNoteWithSync( + content, + action.paragraphIndex ?? 0, + action.text ?? '', + episodeItems, + ); + } else { + // Generic markdown: local placeholder ids only — never invokes wl. + result = insertNoteMarker(content, action.paragraphIndex ?? 0, action.text ?? ''); + } } else if (action.kind === 'update') { result = updateNoteMarker(content, action.noteId ?? '', { text: action.text ?? '', done: action.done, }); + } else if (action.kind === 'remove') { + const noteId = action.noteId ?? ''; + const isLocal = noteId.startsWith('LOCAL-'); + if (episodeItems && episodeItems.length > 0 && noteId && !isLocal) { + // Podcast delete/resolve: DONE marker + resolution comment on the + // note child (PRD §7.3). + result = await resolveNoteWithSync( + content, + noteId, + action.text ?? 'Resolved via Herdr', + episodeItems, + ); + } else { + result = removeNoteMarker(content, noteId); + } } else { - result = removeNoteMarker(content, action.noteId ?? ''); + result = { doc: content, byteOffset: -1 }; } if (result.byteOffset >= 0) { await writeFile(mdPath, result.doc); @@ -1371,6 +1409,29 @@ export async function applyNoteEditToFile( return result; } +/** + * Build the candidate episode work items for PRD §7.3 note-child sync. + * + * The selected work item (an episode's Key Files include the script) plus + * the loaded worklist items, deduplicated by id. Used to resolve the + * episode for a podcast script via `resolveEpisodeId`; generic markdown + * yields an empty candidate set and stays local-only (never invokes `wl`). + */ +function buildEpisodeCandidates(state: WorkItemListState): WorkItem[] { + const seen = new Set(); + const candidates: WorkItem[] = []; + const add = (item: WorkItem | null | undefined): void => { + if (!item) return; + if (seen.has(item.id)) return; + seen.add(item.id); + candidates.push(item); + }; + add(state.detailItem); + for (const item of state.getFlattenedItems()) { + add(item); + } + return candidates; +} /** * Return the first `Key Files:` path ending in `.md` (or empty). * @@ -3203,20 +3264,33 @@ export async function runWorklistTui( state.detailNoteCursor = Math.max(0, paragraphIndex); const marker = mapParagraphToMarker(content, state.detailNoteCursor); + // Candidate episodes for PRD §7.3 note-child sync: the selected + // item (an episode's Key Files include the script) plus the + // loaded worklist items, deduplicated by id. Empty for generic + // markdown never invokes `wl` (local placeholder ids only). + const episodeItems = buildEpisodeCandidates(state); + if (command === '/herdr:note-delete') { if (!marker) { showToast('Note', { body: 'No note on this paragraph.' }); render(); return; } - await applyNoteEditToFile( + const result = await applyNoteEditToFile( state.detailItem, mdPath, { kind: 'remove', noteId: marker.noteId }, readKeyFile, writeKeyFile, + episodeItems, ); - showToast('Note deleted', { body: `${marker.noteId} removed.` }); + if (result.byteOffset < 0) { + showToast('Note', { body: 'Delete did not apply.' }); + } else if (result.warning) { + showToast('Warning', { body: result.warning }); + } else { + showToast('Note deleted', { body: `${marker.noteId} removed.` }); + } render(); return; } @@ -3251,9 +3325,12 @@ export async function runWorklistTui( action, readKeyFile, writeKeyFile, + episodeItems, ); if (result.byteOffset < 0) { showToast('Note', { body: 'Edit did not apply.' }); + } else if (result.warning) { + showToast('Warning', { body: result.warning }); } else { showToast('Note saved', { body: `${result.newNoteId ?? noteId ?? 'marker'} written.` }); } From 849c1f926bb3a399159fd5e4b8fbd8c206dcb95a Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 00:49:17 +0100 Subject: [PATCH 22/34] WL-0MSSQXJRS009L0DX: Docs: README + code comments for inline-note editing --- packages/herdr/README.md | 67 ++++++++++++++++++++++++++++++++-- packages/herdr/src/worklist.ts | 12 +++--- 2 files changed, 71 insertions(+), 8 deletions(-) diff --git a/packages/herdr/README.md b/packages/herdr/README.md index dd1192995..eba3fef19 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -22,7 +22,7 @@ A Herdr plugin that provides a keyboard-navigable work item selection list for b - **Command log** — Every plugin-dispatched command that targets a work item (via `` substitution or an explicit item ID) is recorded to a local JSON log. For `in_progress` items the panel shows the **last command** at the bottom, so you can see exactly what was last dispatched against the item. See [Command log](#command-log). - **Stage grouping** — Work items are grouped by their Worklog stage (standard lifecycle stages only: `idea`, `intake_complete`, `plan_complete`, `in_progress`, `in_review`, `done` — no custom stage values). Podcast episode items group exactly as their frontmatter stages map 1:1 (PRD §7.2). See [Stage grouping](#stage-grouping). - **Generic md viewer** — When a work item's description carries a `Key Files:` path to a markdown document (e.g. a podcast episode `.podcast.md`), the detail view renders the file with a generic markdown viewer (frontmatter skipped, full GFM rendering: headings, lists, tables, blockquotes, code, links) as a preview. The description section is rendered with the same markdown renderer. A persistent **Related Docs** table of contents at the top of the detail view lists every `.md` Key File (`↑↓/j:k` to navigate, `Enter` to open in the viewer), and the metadata panel shows a display-only `Related Docs` row. See [Markdown viewer](#markdown-viewer). -- **Inline note links** — Inline `[NOTE : ...]` markers (PRD §7.1) render as clickable links to the note work items: the marker is displayed as `↗`, and the note text is never shown in the viewer. See [Inline note links](#inline-note-links). +- **Inline note links** — Inline `[NOTE : ...]` markers (PRD §7.1) render as clickable links to the note work items: the marker is displayed as `↗`, and the note text is never shown in the viewer. Any markdown document opened in the viewer can also be annotated in place (`n,e` add/edit, `n,d` delete); podcast scripts sync notes to the worklog as child work items (PRD §7.3). See [Inline note links](#inline-note-links). - **Code Freeze awareness** — While a ship-it release is in progress the project is in *Code Freeze*: the worklist shows a prominent banner and blocks all implement commands (`/skill:implement*`) with a notice dialog until the release finishes. See [Code Freeze](#code-freeze). ## Requirements @@ -615,7 +615,8 @@ description. The viewer: the terminal width); - renders inline `[NOTE : ...]` markers as `↗` links (see [Inline note links](#inline-note-links)); -- is preview-only (no notes editor); +- supports **interactive inline-note editing** on the paragraph under the + cursor (`n,e` add/edit, `n,d` delete — see [Inline note links](#inline-note-links)); - falls back to the raw description when the file is missing/unreadable. The **description section** of the detail view is rendered with the same @@ -635,7 +636,10 @@ wrote Key Files paths relative to the podcast dir rather than the wiki root), then `process.cwd()` as a last resort. The rendered lines appear under an `Episode file (md viewer)` heading in the -detail view, scrollable with the usual `↑↓/j:k` keys. +detail view, scrollable with the usual `↑↓/j:k` keys. Scrolling the +document moves the **note cursor** onto the paragraph under the visible +line, which the inline-note chords operate on (see +[Inline note links](#inline-note-links)). ### Related Docs table of contents @@ -666,6 +670,62 @@ never shown in the viewer (notes are internal review notes, not dialogue). This applies to both the description section and the markdown viewer in the detail view. +### Interactive note editing + +Any markdown document opened in the viewer (podcast scripts **and** generic +`.md` Key Files) can be annotated in place — no notes editor is needed +outside Herdr. Two chords, scoped to the detail view, drive the feature: + +- **`n,e` — add/edit note** — opens the command-input form for the note + text. On a paragraph that already carries a `[NOTE : ...]` marker the + form is **pre-filled with the existing note text** (edit mode); otherwise + it is empty (add mode). Submitting writes the marker inline at the start + of the paragraph and the viewer reflects the change. +- **`n,d` — delete note** — removes the marker from the paragraph under the + cursor (or, for podcast scripts, marks it `DONE` and posts a resolution + comment — see [Podcast note-child lifecycle](#podcast-note-child-lifecycle)). + +The paragraph the chords act on is the one under the **visible cursor +line** (the document scroll position). Write-back is a **surgical +paragraph-level edit**: every byte outside the edited paragraph is preserved +byte-for-byte, and files are resolved against the worklog root via the same +`resolveKeyFilePath` convention as reads. + +### Local placeholder ids (`LOCAL-`) + +Generic markdown documents (no resolvable podcast episode) use a documented +**`LOCAL-`** placeholder scheme: the id is a stable hash of the document +text plus the paragraph index, so the same paragraph in an unchanged +document gets the same id across re-opens. Local notes are never synced to +the worklog — no `wl` calls are made for them. + +### Podcast note-child lifecycle (PRD §7.3) + +On a podcast script whose frontmatter (`podcast_title` / `source_doc`) +resolves to an episode work item, notes are synced to the worklog: + +- **Add** — the note is created as a **child work item** of the episode + (`wl create --parent `), and the real note-child id is written + into the marker `[NOTE : ...]`. +- **Delete/resolve** — the marker is rewritten in the addressed form + `[NOTE : DONE ...]` and a **resolution comment** is posted on the + note child (`wl comment add`). +- **Unresolvable episode** — when a script carries note markers but no + episode can be resolved, a **prominent warning** is shown (never a silent + skip) and the note falls back to a `LOCAL-` placeholder. + +Episode resolution follows the podcast-script skill convention: the script +frontmatter `podcast_title` / `source_doc` is matched against the title (or +id) of the selected work item and the loaded worklist items. + +### Marker-parsing limitation + +`NOTE_MARKER_RE` is **non-greedy**: the note text runs to the **first `]`** +(and may span multiple lines). Consequently, a `]` or a nested `[NOTE` +inside note text would terminate/start a new marker and mis-parse. Keep +note text free of `]` characters; the editor does not yet escape them +(recorded as a known limitation of the marker format, PRD §7.1). + ## Command log Every command the plugin dispatches against a work item is recorded in a @@ -729,6 +789,7 @@ packages/herdr/ │ ├── form-dialog.ts # Form state + rendering for parameter input (unknown ) │ ├── ship-it-dialog.ts # Ship It typed-confirmation dialog (bottom-anchored, S shortcut) │ ├── md-viewer.ts # Generic markdown viewer + inline [NOTE : ...] link rendering +│ ├── md-note-edit.ts # Inline-note marker edit helpers (insert/update/remove, LOCAL-, §7.3 sync) │ ├── command-log.ts # Command log: record/get last command per work item │ ├── settings.ts # User settings management │ └── worklist.ts # List state, rendering, keyboard handling, command output diff --git a/packages/herdr/src/worklist.ts b/packages/herdr/src/worklist.ts index c7726cce0..318d2946c 100644 --- a/packages/herdr/src/worklist.ts +++ b/packages/herdr/src/worklist.ts @@ -1258,11 +1258,13 @@ export function formatDetailContent( } // Generic md-document viewer for a Key Files: .podcast.md episode file - // (preview-only; no notes editor). When a readFile callback is provided - // and the item references a readable .md file, render it in place of the - // raw description so the producer sees the paragraph-format episode. - // Renders the file at detailRenderedIndex (default 0 = first file, the - // existing auto-render); Enter on a ToC entry selects another file + // (any .md Key File). When a readFile callback is provided and the item + // references a readable .md file, render it in place of the raw + // description so the producer sees the paragraph-format episode. + // Inline-note markers can be edited from the viewer via the n,e / n,d + // chords (WL-0MSKV6SKK008MMXR). Renders the file at detailRenderedIndex + // (default 0 = first file, the existing auto-render); Enter on a ToC + // entry selects another file // (WL-0MSHWHULZ001FL8I). if (readFile) { const mdPaths = extractFilePaths(item.description ?? '').filter(p => p.endsWith('.md')); From 74b4d67c7670bf53cb37ad19a3b4b96d0356bece Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 02:43:59 +0100 Subject: [PATCH 23/34] WL-0MSORQ1RG005DGUS: Implementation complete --- packages/herdr/README.md | 12 ++++++++++++ packages/herdr/src/downtime-worker.test.ts | 7 +++++++ packages/herdr/src/downtime-worker.ts | 14 +++++++++++++- 3 files changed, 32 insertions(+), 1 deletion(-) diff --git a/packages/herdr/README.md b/packages/herdr/README.md index eba3fef19..9feab5a23 100644 --- a/packages/herdr/README.md +++ b/packages/herdr/README.md @@ -263,6 +263,18 @@ dispatch still governs (fresh → not a candidate). A missing or unreadable log is treated as empty (fail-safe), so audit dispatch keeps working on a fresh worklog. +> **Bounded audit fan-out (WL-0MSORQ1RG005DGUS):** dispatched panes run +> with `AUDIT_PHASE2_PARALLELISM=1` in the pane environment (inherited by +> the pi process via `send-to-pi.sh`). The audit skill's Phase 2 deep +> analysis (`audit_runner.py`) honours this env var (legacy fallback, +> integer ≥ 1), so a parent audit's child deep-analysis calls run strictly +> sequentially — the skill's documented historical mode. A parent audit +> therefore needs exactly **2 local slots** (parent + at most one child), +> fitting cheap mode's full capacity (2 × 262144 ctx) where the default +> fan-out of 2 would need 3 and spill children to remote. Wall-clock +> tradeoff: child-heavy audits take longer — acceptable for overnight +> downtime work. Interactive (non-downtime) panes are unaffected. + > **Audit-tier error channel (WL-0MSLWJ2KP0002SV0):** the audit lookup > resolves through the same `DowntimeNextResult` error channel as the > plan/intake tiers. A `wl`/CLI failure or unparseable output is `{ok:false}` diff --git a/packages/herdr/src/downtime-worker.test.ts b/packages/herdr/src/downtime-worker.test.ts index 47a12ba6d..40b4ab999 100644 --- a/packages/herdr/src/downtime-worker.test.ts +++ b/packages/herdr/src/downtime-worker.test.ts @@ -2056,6 +2056,13 @@ describe('downtime pane spawn (send-to-pi.sh)', () => { env: expect.objectContaining({ HERDR_RESOLVED_CWD: '/repo' }), }); }); + + it('buildDowntimeSpawnOptions bounds the dispatched audit fan-out (AUDIT_PHASE2_PARALLELISM=1)', () => { + const options = buildDowntimeSpawnOptions('/repo'); + // Parent audit + at most one sequential child deep-analysis call fits + // cheap mode's 2 local slots (WL-0MSORQ1RG005DGUS). + expect(options.env.AUDIT_PHASE2_PARALLELISM).toBe('1'); + }); }); // ── Worker orchestrator (AC1/AC5) ───────────────────────────────────── diff --git a/packages/herdr/src/downtime-worker.ts b/packages/herdr/src/downtime-worker.ts index a07078f91..a5db50f2e 100644 --- a/packages/herdr/src/downtime-worker.ts +++ b/packages/herdr/src/downtime-worker.ts @@ -45,6 +45,11 @@ * `--model`), detached and unref'd; `error`/`exit` handlers capture a * spawn failure or non-zero script exit so a failed pane is never * logged as a successful dispatch (WL-0MSLWJ3I70031Z8U). + * `buildDowntimeSpawnOptions` bounds the dispatched audit fan-out: + * `AUDIT_PHASE2_PARALLELISM=1` makes the audit skill's Phase 2 child + * deep-analysis strictly sequential so a parent audit needs exactly + * 2 local slots (parent + one child), fitting cheap mode's capacity + * (WL-0MSORQ1RG005DGUS). * - `createDowntimeWorker` — per-tick orchestrator (poll → evaluate → * track → dispatch) with settings re-read each tick, plus the * no-candidate cooldown (WL-0MSI7DQL10016QYX): a genuine empty backlog @@ -1095,7 +1100,14 @@ export function buildDowntimeSpawnOptions(cwd: string): { detached: true, stdio: 'ignore', cwd, - env: { ...process.env, HERDR_RESOLVED_CWD: cwd }, + // AUDIT_PHASE2_PARALLELISM=1 bounds the dispatched audit's Phase 2 child + // deep-analysis fan-out to strictly sequential (parent runs first, then + // one child at a time — the audit skill's documented historical mode), so + // a child-heavy parent audit needs exactly 2 local slots and fits cheap + // mode's full capacity (WL-0MSORQ1RG005DGUS). The audit skill honours + // this env var (legacy fallback, integer >= 1); no audit-skill change + // needed. Interactive (non-downtime) panes are unaffected. + env: { ...process.env, HERDR_RESOLVED_CWD: cwd, AUDIT_PHASE2_PARALLELISM: '1' }, }; } From 09dffeaa77905d3e67228c4d7a333bc1311d4155 Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 12:55:52 +0100 Subject: [PATCH 24/34] WL-0MSU8B7Y0008MRK0: Implementation complete --- packages/shared/src/database-dedup.test.ts | 134 ++++++++++++++++++++ packages/shared/src/database.ts | 40 ++++++ packages/shared/src/persistent-store.ts | 49 ++++++++ src/cli-types.ts | 4 + src/commands/create.ts | 67 +++++++++- tests/cli/create-dedup.test.ts | 136 +++++++++++++++++++++ tests/cli/update-batch.test.ts | 5 +- tests/cli/update-priority-cascade.test.ts | 5 +- 8 files changed, 437 insertions(+), 3 deletions(-) create mode 100644 packages/shared/src/database-dedup.test.ts create mode 100644 tests/cli/create-dedup.test.ts diff --git a/packages/shared/src/database-dedup.test.ts b/packages/shared/src/database-dedup.test.ts new file mode 100644 index 000000000..655eef900 --- /dev/null +++ b/packages/shared/src/database-dedup.test.ts @@ -0,0 +1,134 @@ +/** + * Tests for the `wl create` dedup guard query (WL-0MSTNG2QF0049B97). + * + * `WorklogDatabase.getRecentDuplicate()` finds the most recent non-terminal + * work item whose title matches under case/whitespace normalization within + * a recent time window. The `wl create` CLI uses it so that agent retries of + * an identical create command become no-ops instead of producing + * byte-identical twin items. + * + * All assertions go through the public API (`import`/`getRecentDuplicate`) + * so the tests pin observable query behaviour, not implementation details. + * + * Run: npx vitest run packages/shared/src/database-dedup.test.ts + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; +import { WorklogDatabase, normalizeTitleForMatch } from './database.js'; +import type { WorkItem } from './types.js'; + +const NOW = Date.now(); +const FIXED_TS = new Date(NOW).toISOString(); +const FIVE_MINUTES = 5 * 60 * 1000; + +/** Build a complete WorkItem with a fixed recent timestamp. */ +function makeItem(overrides: Partial = {}): WorkItem { + return { + id: 'TEST-0001', + title: 'Same Title', + description: '', + status: 'open', + priority: 'medium', + sortIndex: 0, + parentId: null, + createdAt: FIXED_TS, + updatedAt: FIXED_TS, + tags: [], + assignee: '', + stage: 'idea', + issueType: '', + createdBy: '', + deletedBy: '', + deleteReason: '', + risk: '', + effort: '', + needsProducerReview: false, + ...overrides, + }; +} + +let tempDir: string; +let db: WorklogDatabase; + +beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), 'wl-db-dedup-')); + db = new WorklogDatabase('TEST', join(tempDir, 'worklog.db'), join(tempDir, 'data.jsonl'), true); +}); + +afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); +}); + +describe('normalizeTitleForMatch', () => { + it('lowercases and strips all whitespace so title variants compare equal', () => { + expect(normalizeTitleForMatch('Same Title')).toBe('sametitle'); + expect(normalizeTitleForMatch('same title')).toBe('sametitle'); + expect(normalizeTitleForMatch(' same\t title \n')).toBe('sametitle'); + expect(normalizeTitleForMatch('')).toBe(''); + }); +}); + +describe('WorklogDatabase.getRecentDuplicate', () => { + it('returns a recent non-terminal item with a case/whitespace-normalized title match', () => { + db.import([makeItem({ id: 'TEST-0001', title: 'Same Title' })]); + expect(db.getRecentDuplicate('same title', FIVE_MINUTES)?.id).toBe('TEST-0001'); + }); + + it('returns null when no title matches', () => { + db.import([makeItem({ id: 'TEST-0001', title: 'Other Title' })]); + expect(db.getRecentDuplicate('Same Title', FIVE_MINUTES)).toBeNull(); + }); + + it('returns null when the only match is a completed (terminal) item', () => { + db.import([makeItem({ id: 'TEST-0001', title: 'Same Title', status: 'completed' })]); + expect(db.getRecentDuplicate('Same Title', FIVE_MINUTES)).toBeNull(); + }); + + it('returns null when the only match is a deleted (terminal) item', () => { + db.import([makeItem({ id: 'TEST-0001', title: 'Same Title', status: 'deleted' })]); + expect(db.getRecentDuplicate('Same Title', FIVE_MINUTES)).toBeNull(); + }); + + it('matches non-terminal in-progress and blocked items', () => { + db.import([makeItem({ id: 'TEST-0001', title: 'In Progress Item', status: 'in-progress' })]); + expect(db.getRecentDuplicate('In Progress Item', FIVE_MINUTES)?.id).toBe('TEST-0001'); + + db.import([makeItem({ id: 'TEST-0002', title: 'Blocked Item', status: 'blocked' })]); + expect(db.getRecentDuplicate('Blocked Item', FIVE_MINUTES)?.id).toBe('TEST-0002'); + }); + + it('returns null when the match is older than the window', () => { + const old = new Date(NOW - 10 * 60 * 1000).toISOString(); + db.import([makeItem({ id: 'TEST-0001', title: 'Same Title', createdAt: old })]); + expect(db.getRecentDuplicate('Same Title', FIVE_MINUTES)).toBeNull(); + }); + + it('returns null when the match is exactly at the window cutoff', () => { + const atCutoff = new Date(NOW - FIVE_MINUTES).toISOString(); + db.import([makeItem({ id: 'TEST-0001', title: 'Same Title', createdAt: atCutoff })]); + // createdAt > cutoff is strictly greater — an item created exactly at the + // cutoff (i.e. 5 minutes ago) is outside the window. + expect(db.getRecentDuplicate('Same Title', FIVE_MINUTES)).toBeNull(); + }); + + it('respects prefix scope: items under another prefix are not matched', () => { + db.import([makeItem({ id: 'OTHER-0001', title: 'Same Title' })]); + expect(db.getRecentDuplicate('Same Title', FIVE_MINUTES)).toBeNull(); + }); + + it('honours an explicit prefix override', () => { + db.import([makeItem({ id: 'OTHER-0001', title: 'Same Title' })]); + expect(db.getRecentDuplicate('Same Title', FIVE_MINUTES, 'OTHER')?.id).toBe('OTHER-0001'); + }); + + it('returns the newest matching item when several match', () => { + db.import([ + makeItem({ id: 'TEST-0001', title: 'Same Title', createdAt: new Date(NOW - 60_000).toISOString() }), + makeItem({ id: 'TEST-0002', title: 'same title', createdAt: FIXED_TS }), + ]); + expect(db.getRecentDuplicate('Same Title', FIVE_MINUTES)?.id).toBe('TEST-0002'); + }); +}); diff --git a/packages/shared/src/database.ts b/packages/shared/src/database.ts index 09e7c008c..991ddda6e 100644 --- a/packages/shared/src/database.ts +++ b/packages/shared/src/database.ts @@ -141,6 +141,18 @@ const UNIQUE_RANDOM_LENGTH = 5; const UNIQUE_ID_LENGTH = UNIQUE_TIME_LENGTH + UNIQUE_SEQUENCE_LENGTH + UNIQUE_RANDOM_LENGTH; const MAX_ID_GENERATION_ATTEMPTS = 10; +/** + * Normalize a title into a canonical dedup-comparison key: case-folded with + * ALL whitespace removed (tabs/newlines collapse away too). `"Same Title"`, + * `"same title"` and `" same\ttitle "` all normalize to `"sametitle"` so + * retried create commands with cosmetic title differences still match + * (WL-0MSTNG2QF0049B97). The SQL side of `getRecentDuplicate` applies the + * same transformation to stored titles. + */ +export function normalizeTitleForMatch(title: string): string { + return title.replace(/\s+/g, '').toLowerCase(); +} + export class WorklogDatabase { private store: SqlitePersistentStore; private prefix: string; @@ -998,6 +1010,34 @@ export class WorklogDatabase { return this.create({ ...input, sortIndex }); } + /** + * Find the most recent non-terminal work item whose title matches `title` + * under case/whitespace normalization and was created within `windowMs`. + * + * Backs the `wl create` dedup guard (WL-0MSTNG2QF0049B97): retrying an + * identical create command (common when agents lose the tool result to + * output trimming) must return the existing item instead of creating a + * byte-identical twin. Only non-terminal items (open/in-progress/blocked) + * within the recent window are considered — completed items and stale + * same-title items are deliberately unrelated. + * + * @param title - The candidate title (compared case- and whitespace- + * insensitively against stored titles). + * @param windowMs - Look-back window in milliseconds; only items created + * strictly after `now - windowMs` match. + * @param prefix - Prefix scope; defaults to this database's prefix. Only + * items whose id starts with `-` are considered. + * @returns The newest matching item, or null when no match exists. + */ + getRecentDuplicate(title: string, windowMs: number, prefix?: string): WorkItem | null { + const effectivePrefix = (prefix ?? this.prefix).toUpperCase(); + return this.store.getRecentDuplicateByNormalizedTitle( + normalizeTitleForMatch(title), + windowMs, + effectivePrefix, + ); + } + /** * Get a work item by ID */ diff --git a/packages/shared/src/persistent-store.ts b/packages/shared/src/persistent-store.ts index 7655dfe0f..384e5378a 100644 --- a/packages/shared/src/persistent-store.ts +++ b/packages/shared/src/persistent-store.ts @@ -572,6 +572,55 @@ export class SqlitePersistentStore { return row.count; } + /** + * Find the most recent non-terminal work item whose *normalized* title + * equals `normalizedTitle` and whose creation time is strictly after + * `now - windowMs`. Used by the `wl create` dedup guard + * (WL-0MSTNG2QF0049B97) — see `WorklogDatabase.getRecentDuplicate`. + * + * The stored title is normalized in SQL the same way + * `normalizeTitleForMatch` normalizes the query side: case-folded with + * tab/CR/LF/space removed, so `"Same Title"` vs `"same title"` compare + * equal. A direct SQL query (rather than filtering `getAllWorkItems()` in + * memory) keeps the guard cheap and always reads the freshest rows. + * + * @param normalizedTitle - Canonical key from `normalizeTitleForMatch()`. + * @param windowMs - Look-back window in milliseconds. + * @param prefix - Prefix scope; only ids starting with `-` match. + * @returns The newest matching `WorkItem`, or null when none matches. + */ + getRecentDuplicateByNormalizedTitle( + normalizedTitle: string, + windowMs: number, + prefix: string, + ): WorkItem | null { + const cutoff = new Date(Date.now() - windowMs).toISOString(); + const stmt = this.db.prepare(` + SELECT * FROM workitems + WHERE status IN ('open', 'in-progress', 'blocked') + AND createdAt > ? + AND LOWER(REPLACE(REPLACE(REPLACE(REPLACE(title, char(9), ''), char(10), ''), char(13), ''), ' ', '')) = ? + AND id LIKE ? ESCAPE '\\' + ORDER BY createdAt DESC + LIMIT 1 + `); + const row = stmt.get( + cutoff, + normalizedTitle, + `${this.escapeLikePattern(prefix)}-%`, + ) as any; + if (!row) return null; + return this.rowToWorkItem(row); + } + + /** + * Escape LIKE wildcards (`%`, `_`) and the escape character itself so a + * user-supplied prefix cannot act as a wildcard in an `id LIKE ?` clause. + */ + private escapeLikePattern(value: string): string { + return value.replace(/[\\%_]/g, (ch) => `\\${ch}`); + } + /** * Get all work items */ diff --git a/src/cli-types.ts b/src/cli-types.ts index 484f1b8cc..f1acf1133 100644 --- a/src/cli-types.ts +++ b/src/cli-types.ts @@ -41,6 +41,10 @@ export interface CreateOptions { noReSort?: boolean; /** Force a synchronous re-sort when run (blocks until complete) */ reSortSync?: boolean; + /** Bypass the dedup guard: create a new item even when a recent non-terminal same-title item exists */ + allowDuplicate?: boolean; + /** Dedup match window duration (e.g. "30s", "5m", "1h", or raw ms); default "5m" */ + dedupWindow?: string; } export interface ListOptions { diff --git a/src/commands/create.ts b/src/commands/create.ts index a0d12fff5..37ac984a4 100644 --- a/src/commands/create.ts +++ b/src/commands/create.ts @@ -12,6 +12,38 @@ import { normalizeActionArgs } from './cli-utils.js'; import { buildAuditEntry, formatInvalidAuditFirstLineMessage, inspectAuditFirstLine, redactAuditText } from '../audit.js'; import { normalizePriority, CANONICAL_PRIORITIES } from '../validators/priority.js'; +/** + * Default dedup match window for `wl create` (WL-0MSTNG2QF0049B97): retried + * creates are caught within this look-back; genuinely re-created same-title + * items older than this are left alone. The 1–30s retry signature seen in + * the RCA sits comfortably inside 5 minutes. + */ +export const DEFAULT_DEDUP_WINDOW_MS = 5 * 60 * 1000; + +/** + * Parse a `--dedup-window` duration string into milliseconds. + * + * Accepts a bare number (raw milliseconds) or a number with a unit suffix: + * `ms`, `s`, `m`, `h`, `d` (e.g. `30s`, `5m`, `1h`, `300000`). Returns NaN + * for invalid input so the caller can surface a clear error. + */ +export function parseDedupWindowMs(value: string | number): number { + const trimmed = String(value).trim(); + if (trimmed === '') return Number.NaN; + const match = /^(\d+(?:\.\d+)?)\s*(ms|s|m|h|d)?$/i.exec(trimmed); + if (!match) return Number.NaN; + const amount = Number(match[1]); + const unit = (match[2] || 'ms').toLowerCase(); + const multipliers: Record = { + ms: 1, + s: 1000, + m: 60_000, + h: 3_600_000, + d: 86_400_000, + }; + return amount * multipliers[unit]; +} + export default function register(ctx: PluginContext): void { const { program, output, utils } = ctx; @@ -40,8 +72,10 @@ export default function register(ctx: PluginContext): void { .option('--prefix ', 'Override the default prefix') .option('--no-re-sort', 'Skip automatic re-sort after creating the item') .option('--re-sort-sync', 'Force a synchronous re-sort after creating the item', false) + .option('--allow-duplicate', 'Allow creating a new item even when a recent non-terminal item with the same title exists (bypasses the dedup guard)') + .option('--dedup-window ', 'Dedup match window: recent non-terminal same-title items created within this window are reused instead of creating a twin (e.g. 30s, 5m, 1h; default 5m)') .action(async (...rawArgs: any[]) => { - const normalized = normalizeActionArgs(rawArgs, ['title','description','descriptionFile','status','priority','parent','tags','assignee','stage','risk','effort','issueType','createdBy','deletedBy','deleteReason','needsProducerReview','audit','auditText','auditFile','prefix','noReSort','reSortSync']); + const normalized = normalizeActionArgs(rawArgs, ['title','description','descriptionFile','status','priority','parent','tags','assignee','stage','risk','effort','issueType','createdBy','deletedBy','deleteReason','needsProducerReview','audit','auditText','auditFile','prefix','noReSort','reSortSync','allowDuplicate','dedupWindow']); let options: CreateOptions = normalized.options as any || {}; utils.requireInitialized(); const db = utils.getDatabase(options.prefix); @@ -153,6 +187,37 @@ export default function register(ctx: PluginContext): void { const parentId = utils.normalizeCliId(options.parent, options.prefix) || null; + // ── Dedup guard (WL-0MSTNG2QF0049B97) ────────────────────────── + // Retrying an identical `wl create` (common when agents lose the tool + // result to output trimming) must return the existing recent + // non-terminal same-title item instead of creating a byte-identical + // twin. Only the title is compared (case/whitespace-insensitive) — a + // match is reused regardless of other flags. Bypass with + // `--allow-duplicate` when a genuinely new item is needed. + const dedupWindowMs = parseDedupWindowMs(options.dedupWindow ?? DEFAULT_DEDUP_WINDOW_MS); + if (Number.isNaN(dedupWindowMs) || dedupWindowMs < 0) { + const message = `Invalid --dedup-window value "${options.dedupWindow}". Expected a duration like "30s", "5m", "1h" or raw milliseconds.`; + output.error(message, { success: false, error: 'invalid-dedup-window' }); + process.exit(1); + } + if (!options.allowDuplicate) { + const existing = db.getRecentDuplicate(options.title, dedupWindowMs); + if (existing) { + if (utils.isJsonMode()) { + output.json({ + success: true, + duplicate: true, + duplicateOf: existing.id, + workItem: existing, + }); + } else { + console.log(`Duplicate of ${existing.id} (title matched)`); + console.log(humanFormatWorkItem(existing, db, resolveFormat(program))); + } + return; + } + } + const item = db.createWithNextSortIndex({ title: options.title, description: description, diff --git a/tests/cli/create-dedup.test.ts b/tests/cli/create-dedup.test.ts new file mode 100644 index 000000000..12a095cdf --- /dev/null +++ b/tests/cli/create-dedup.test.ts @@ -0,0 +1,136 @@ +/** + * CLI integration tests for the `wl create` dedup guard (WL-0MSTNG2QF0049B97). + * + * Retrying an identical `wl create` (common when agents lose the tool result + * to output trimming) must return the existing item with a `duplicateOf` + * marker instead of creating a byte-identical twin work item. + * + * Work item: WL-0MSU8B7Y0008MRK0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { + cliPath, + execAsync, + enterTempDir, + leaveTempDir, + writeConfig, + writeInitSemaphore, +} from './cli-helpers.js'; + +describe('wl create dedup guard', () => { + let tempState: { tempDir: string; originalCwd: string }; + + beforeEach(() => { + tempState = enterTempDir(); + writeConfig(tempState.tempDir, 'Test Project', 'TEST'); + writeInitSemaphore(tempState.tempDir); + }); + + afterEach(() => { + leaveTempDir(tempState); + }); + + async function createJson(title = 'Some Title', flags = ''): Promise { + const { stdout } = await execAsync( + `tsx ${cliPath} --json create -t "${title}" ${flags}` + ); + return JSON.parse(stdout); + } + + async function listJson(): Promise { + const { stdout } = await execAsync(`tsx ${cliPath} --json list`); + return JSON.parse(stdout); + } + + it('returns the existing item with duplicateOf when a recent match exists', async () => { + const first = await createJson(); + expect(first.success).toBe(true); + expect(first.duplicateOf).toBeUndefined(); + + // Byte-identical retry — must NOT create a twin. + const second = await createJson(); + expect(second.success).toBe(true); + expect(second.duplicate).toBe(true); + expect(second.duplicateOf).toBe(first.workItem.id); + expect(second.workItem.id).toBe(first.workItem.id); + + // Only one item exists in the worklog. + const list = await listJson(); + expect(list.count).toBe(1); + }); + + it('matches case- and whitespace-insensitive title variants', async () => { + const first = await createJson('Same Title'); + const second = await createJson('same title'); + expect(second.duplicateOf).toBe(first.workItem.id); + expect(second.workItem.id).toBe(first.workItem.id); + }); + + it('creates a new item when no recent match exists', async () => { + const first = await createJson('Some Title'); + const second = await createJson('Unrelated Title'); + expect(second.duplicateOf).toBeUndefined(); + expect(second.workItem.id).not.toBe(first.workItem.id); + }); + + it('does not reuse terminal (completed) items', async () => { + const { stdout } = await execAsync( + `tsx ${cliPath} --json create -t "Done Title" --status completed --stage in_review` + ); + const first = JSON.parse(stdout); + const second = await createJson('Done Title'); + expect(second.duplicateOf).toBeUndefined(); + expect(second.workItem.id).not.toBe(first.workItem.id); + }); + + it('--allow-duplicate bypasses the guard and creates a new item', async () => { + const first = await createJson('Same Title'); + const second = await createJson('Same Title', '--allow-duplicate'); + expect(second.duplicateOf).toBeUndefined(); + expect(second.workItem.id).not.toBe(first.workItem.id); + + const list = await listJson(); + expect(list.count).toBe(2); + }); + + it('--dedup-window 0ms disables matching (item outside window)', async () => { + const first = await createJson('Same Title'); + const second = await createJson('Same Title', '--dedup-window 0ms'); + expect(second.duplicateOf).toBeUndefined(); + expect(second.workItem.id).not.toBe(first.workItem.id); + }); + + it('--dedup-window 30s accepts a duration string', async () => { + const first = await createJson('Same Title'); + const second = await createJson('Same Title', '--dedup-window 30s'); + expect(second.duplicateOf).toBe(first.workItem.id); + }); + + it('prints "Duplicate of " in human mode', async () => { + const first = await createJson('Same Title'); + const { stdout } = await execAsync(`tsx ${cliPath} create -t "Same Title"`); + expect(stdout).toContain(`Duplicate of ${first.workItem.id}`); + // The duplicate line still surfaces the item id (robust to trimming). + expect(stdout).toContain(first.workItem.id); + }); + + it('respects --prefix scope', async () => { + const { stdout } = await execAsync( + `tsx ${cliPath} --json create -t "Prefixed Title" --prefix OTHER` + ); + const first = JSON.parse(stdout); + expect(first.workItem.id.startsWith('OTHER-')).toBe(true); + + // Same title under the default TEST prefix is NOT a duplicate. + const second = await createJson('Prefixed Title'); + expect(second.duplicateOf).toBeUndefined(); + expect(second.workItem.id.startsWith('TEST-')).toBe(true); + }); + + it('rejects an invalid --dedup-window value with a clear error', async () => { + await expect( + execAsync(`tsx ${cliPath} --json create -t "Same Title" --dedup-window bogus`) + ).rejects.toThrow('Command failed'); + }); +}); diff --git a/tests/cli/update-batch.test.ts b/tests/cli/update-batch.test.ts index e964ccbf6..5e217bab6 100644 --- a/tests/cli/update-batch.test.ts +++ b/tests/cli/update-batch.test.ts @@ -41,8 +41,11 @@ describe('update batch behaviour', () => { // Helper: create a work item and return its id // ----------------------------------------------------------------------- async function createItem(flags = ''): Promise { + // --allow-duplicate: the dedup guard (WL-0MSTNG2QF0049B97) would + // otherwise collapse the multi-item batch setups (same title, distinct + // items) that this test creates. const { stdout } = await execAsync( - `tsx ${cliPath} --json create -t "Batch test item" ${flags}` + `tsx ${cliPath} --json create -t "Batch test item" --allow-duplicate ${flags}` ); return JSON.parse(stdout).workItem.id; } diff --git a/tests/cli/update-priority-cascade.test.ts b/tests/cli/update-priority-cascade.test.ts index c6f306a15..98b4138a7 100644 --- a/tests/cli/update-priority-cascade.test.ts +++ b/tests/cli/update-priority-cascade.test.ts @@ -32,8 +32,11 @@ describe('update priority cascade', () => { }); async function createItem(flags = ''): Promise { + // --allow-duplicate: the dedup guard (WL-0MSTNG2QF0049B97) would + // otherwise return the first item for subsequent same-title creates, + // collapsing the parent+children setup this test needs. const { stdout } = await execAsync( - `tsx ${cliPath} --json create -t "Cascade item" ${flags}` + `tsx ${cliPath} --json create -t "Cascade item" --allow-duplicate ${flags}` ); return JSON.parse(stdout).workItem.id; } From 28fd7462b8ee10f14a18f8819139a234ad922521 Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 13:25:13 +0100 Subject: [PATCH 25/34] WL-0MSU89IZG000FHJF: Implementation complete --- tests/cli/create-dedup.test.ts | 80 ++++++++++++++++++++++------------ 1 file changed, 51 insertions(+), 29 deletions(-) diff --git a/tests/cli/create-dedup.test.ts b/tests/cli/create-dedup.test.ts index 12a095cdf..fb3a1e364 100644 --- a/tests/cli/create-dedup.test.ts +++ b/tests/cli/create-dedup.test.ts @@ -5,7 +5,9 @@ * to output trimming) must return the existing item with a `duplicateOf` * marker instead of creating a byte-identical twin work item. * - * Work item: WL-0MSU8B7Y0008MRK0 + * The test cases map 1:1 to the acceptance criteria of feature + * WL-0MSU89IZG000FHJF (9 CLI integration tests via the temp-dir harness in + * `cli-helpers.ts`). */ import { describe, it, expect, beforeEach, afterEach } from 'vitest'; @@ -43,11 +45,22 @@ describe('wl create dedup guard', () => { return JSON.parse(stdout); } - it('returns the existing item with duplicateOf when a recent match exists', async () => { - const first = await createJson(); + // AC1: No match within window → normal create proceeds, new item returned + it('AC1: creates a new item when no recent title match exists', async () => { + const first = await createJson('Some Title'); expect(first.success).toBe(true); expect(first.duplicateOf).toBeUndefined(); + const second = await createJson('Unrelated Title'); + expect(second.duplicateOf).toBeUndefined(); + expect(second.workItem.id).not.toBe(first.workItem.id); + }); + + // AC2: Recent match found (exact title) → existing item returned with duplicateOf + it('AC2: returns the existing item with duplicateOf on exact-title retry', async () => { + const first = await createJson(); + expect(first.duplicateOf).toBeUndefined(); + // Byte-identical retry — must NOT create a twin. const second = await createJson(); expect(second.success).toBe(true); @@ -60,21 +73,24 @@ describe('wl create dedup guard', () => { expect(list.count).toBe(1); }); - it('matches case- and whitespace-insensitive title variants', async () => { + // AC3: Recent match (case-insensitive) → existing item returned with duplicateOf + it('AC3: matches case-insensitive title variants', async () => { const first = await createJson('Same Title'); - const second = await createJson('same title'); + const second = await createJson('same title'); expect(second.duplicateOf).toBe(first.workItem.id); expect(second.workItem.id).toBe(first.workItem.id); }); - it('creates a new item when no recent match exists', async () => { - const first = await createJson('Some Title'); - const second = await createJson('Unrelated Title'); - expect(second.duplicateOf).toBeUndefined(); - expect(second.workItem.id).not.toBe(first.workItem.id); + // AC4: Recent match (whitespace-normalized) → existing item returned with duplicateOf + it('AC4: matches whitespace-normalized title variants', async () => { + const first = await createJson('Same Title'); + const second = await createJson('Same Title'); + expect(second.duplicateOf).toBe(first.workItem.id); + expect(second.workItem.id).toBe(first.workItem.id); }); - it('does not reuse terminal (completed) items', async () => { + // AC5: Non-terminal (completed) item → normal create proceeds, new item returned + it('AC5: does not reuse terminal (completed) items', async () => { const { stdout } = await execAsync( `tsx ${cliPath} --json create -t "Done Title" --status completed --stage in_review` ); @@ -84,7 +100,8 @@ describe('wl create dedup guard', () => { expect(second.workItem.id).not.toBe(first.workItem.id); }); - it('--allow-duplicate bypasses the guard and creates a new item', async () => { + // AC6: --allow-duplicate → bypasses guard, new item created + it('AC6: --allow-duplicate bypasses the guard and creates a new item', async () => { const first = await createJson('Same Title'); const second = await createJson('Same Title', '--allow-duplicate'); expect(second.duplicateOf).toBeUndefined(); @@ -94,28 +111,19 @@ describe('wl create dedup guard', () => { expect(list.count).toBe(2); }); - it('--dedup-window 0ms disables matching (item outside window)', async () => { + // AC7: --dedup-window 10s with match after 11s → normal create (window expired) + // Implemented with a 1s window + 1.2s wait — the same expiry code path + // (createdAt > now - windowMs cutoff in persistent-store.ts) 10x faster. + it('AC7: does not reuse a match older than --dedup-window (window expiry)', async () => { const first = await createJson('Same Title'); - const second = await createJson('Same Title', '--dedup-window 0ms'); + await new Promise((resolve) => setTimeout(resolve, 1200)); + const second = await createJson('Same Title', '--dedup-window 1s'); expect(second.duplicateOf).toBeUndefined(); expect(second.workItem.id).not.toBe(first.workItem.id); }); - it('--dedup-window 30s accepts a duration string', async () => { - const first = await createJson('Same Title'); - const second = await createJson('Same Title', '--dedup-window 30s'); - expect(second.duplicateOf).toBe(first.workItem.id); - }); - - it('prints "Duplicate of " in human mode', async () => { - const first = await createJson('Same Title'); - const { stdout } = await execAsync(`tsx ${cliPath} create -t "Same Title"`); - expect(stdout).toContain(`Duplicate of ${first.workItem.id}`); - // The duplicate line still surfaces the item id (robust to trimming). - expect(stdout).toContain(first.workItem.id); - }); - - it('respects --prefix scope', async () => { + // AC8: Prefix-aware matching (items with different prefixes don't match) + it('AC8: respects --prefix scope', async () => { const { stdout } = await execAsync( `tsx ${cliPath} --json create -t "Prefixed Title" --prefix OTHER` ); @@ -128,6 +136,20 @@ describe('wl create dedup guard', () => { expect(second.workItem.id.startsWith('TEST-')).toBe(true); }); + // AC9: JSON mode includes duplicateOf; human mode includes "Duplicate of" line + it('AC9: JSON output carries duplicateOf; human output prints a Duplicate of line', async () => { + const first = await createJson('Same Title'); + + // JSON mode: duplicateOf field present (covered end-to-end here too). + const retryJson = await createJson('Same Title'); + expect(retryJson.duplicateOf).toBe(first.workItem.id); + + // Human mode: a clear "Duplicate of " line. + const { stdout } = await execAsync(`tsx ${cliPath} create -t "Same Title"`); + expect(stdout).toContain(`Duplicate of ${first.workItem.id}`); + }); + + // Bonus (beyond the 9 ACs): invalid --dedup-window values fail loudly it('rejects an invalid --dedup-window value with a clear error', async () => { await expect( execAsync(`tsx ${cliPath} --json create -t "Same Title" --dedup-window bogus`) From 6cebcd233260f1f7e7bed3a85c6f09798342a632 Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 13:45:58 +0100 Subject: [PATCH 26/34] WL-0MSU8E2YA0059GEA: Implementation complete --- src/commands/create.ts | 8 ++++ tests/cli/create-id-first.test.ts | 77 +++++++++++++++++++++++++++++++ 2 files changed, 85 insertions(+) create mode 100644 tests/cli/create-id-first.test.ts diff --git a/src/commands/create.ts b/src/commands/create.ts index 37ac984a4..d6b3ae0fb 100644 --- a/src/commands/create.ts +++ b/src/commands/create.ts @@ -205,12 +205,16 @@ export default function register(ctx: PluginContext): void { if (existing) { if (utils.isJsonMode()) { output.json({ + id: existing.id, success: true, duplicate: true, duplicateOf: existing.id, workItem: existing, }); } else { + // ID first so output trimming (tail / E2BIG) cannot hide it + // (RCA fix #2, WL-0MSU8E2YA0059GEA). + console.log(`ID: ${existing.id}`); console.log(`Duplicate of ${existing.id} (title matched)`); console.log(humanFormatWorkItem(existing, db, resolveFormat(program))); } @@ -266,13 +270,17 @@ export default function register(ctx: PluginContext): void { } if (utils.isJsonMode()) { + // Top-level `id` first so output trimming cannot hide the created ID + // (RCA fix #2, WL-0MSU8E2YA0059GEA). output.json({ + id: item.id, success: true, workItem: refreshed, ...(demotedParent ? { demotedParent } : {}), }); } else { const format = resolveFormat(program); + console.log(`ID: ${refreshed.id}`); console.log(humanFormatWorkItem(refreshed, db, format)); if (demotedParent) { console.log(`[Parent ${demotedParent.parent.id} demoted from ${demotedParent.from.status}/${demotedParent.from.stage} to ${demotedParent.to.status}/${demotedParent.to.stage}]`); diff --git a/tests/cli/create-id-first.test.ts b/tests/cli/create-id-first.test.ts new file mode 100644 index 000000000..c8a7d5b56 --- /dev/null +++ b/tests/cli/create-id-first.test.ts @@ -0,0 +1,77 @@ +/** + * CLI integration tests for ID-first `wl create` output (WL-0MSTNG2QF0049B97, + * RCA fix #2 — feature WL-0MSU8E2YA0059GEA). + * + * The created ID must be robustly visible so output-trimming tools (tail, + * E2BIG truncation) cannot hide it: human mode prints an `ID: ` line + * first; `--json` output carries a top-level `id` field as the first key. + * This reduces the retry frequency that triggers the dedup guard. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { + cliPath, + execAsync, + enterTempDir, + leaveTempDir, + writeConfig, + writeInitSemaphore, +} from './cli-helpers.js'; + +describe('wl create ID-first output', () => { + let tempState: { tempDir: string; originalCwd: string }; + + beforeEach(() => { + tempState = enterTempDir(); + writeConfig(tempState.tempDir, 'Test Project', 'TEST'); + writeInitSemaphore(tempState.tempDir); + }); + + afterEach(() => { + leaveTempDir(tempState); + }); + + async function createJson(flags = ''): Promise { + const { stdout } = await execAsync( + `tsx ${cliPath} --json create -t "Some Title" ${flags}` + ); + return JSON.parse(stdout); + } + + // AC1: Human mode — first line of output is `ID: ` + it('AC1: prints an ID: line as the first line in human mode', async () => { + const { stdout } = await execAsync(`tsx ${cliPath} create -t "Some Title"`); + const lines = stdout.split('\n'); + expect(lines[0]).toMatch(/^ID: TEST-[A-Z0-9]+$/); + }); + + // AC2: JSON mode — `id` is the first key of the output object and matches workItem.id + it('AC2: top-level id is the first JSON key and matches workItem.id', async () => { + const result = await createJson(); + expect(Object.keys(result)[0]).toBe('id'); + expect(result.id).toBe(result.workItem.id); + expect(result.workItem.id).toMatch(/^TEST-/); + // id also leads the workItem object itself + expect(Object.keys(result.workItem)[0]).toBe('id'); + }); + + // AC3a: Duplicate case (human) — `ID: ` still appears as first line + it('AC3: duplicate retry in human mode still leads with ID: ', async () => { + const first = await createJson(); + const { stdout } = await execAsync(`tsx ${cliPath} create -t "Some Title"`); + const lines = stdout.split('\n'); + expect(lines[0]).toBe(`ID: ${first.workItem.id}`); + // The duplicate marker follows the ID line. + expect(lines.slice(1).join('\n')).toContain('Duplicate of'); + }); + + // AC3b: Duplicate case (JSON) — top-level id still present and first + it('AC3: duplicate retry in JSON mode carries a first-key id equal to duplicateOf', async () => { + const first = await createJson(); + const retry = await createJson(); + expect(Object.keys(retry)[0]).toBe('id'); + expect(retry.id).toBe(first.workItem.id); + expect(retry.duplicateOf).toBe(first.workItem.id); + expect(retry.workItem.id).toBe(first.workItem.id); + }); +}); From e713373d51f5027d1db769b19cd2555bb2adab4e Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 14:04:04 +0100 Subject: [PATCH 27/34] WL-0MSU8F20E004TRXX: Implementation complete --- CLI.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/CLI.md b/CLI.md index c67b8e341..bdbabb327 100644 --- a/CLI.md +++ b/CLI.md @@ -121,8 +121,14 @@ Options: - `--audit-text ` — Set structured audit text when creating an item. The audit result is stored in the `audit_results` table (the sole source of truth for audit state). Prefer `--audit-file` for file-based input to avoid shell-escaping issues (see docs/AUDIT_STATUS.md). - `--audit-file ` — Read audit text from a file (recommended for large or shell-sensitive content). - `--prefix ` — Override default ID prefix (repo-local scope) (optional). +- `--allow-duplicate` — Bypass the dedup guard: create a new item even when a recent non-terminal item with the same title exists (optional). +- `--dedup-window ` — Dedup match window, e.g. `30s`, `5m`, `1h` or raw milliseconds (optional; default: `5m`). - `--json` — Output JSON (optional). +Dedup guard: + +- Retrying an identical `wl create` (common when agents lose the tool result to output trimming) returns the existing recent non-terminal item with a `duplicateOf` marker instead of creating a byte-identical twin. Only items created within the `--dedup-window` (default 5 minutes) whose title matches case- and whitespace-insensitively are reused; completed/deleted items are never matched. Pass `--allow-duplicate` when a genuinely new item is needed. In JSON mode the response carries a top-level `id` field first and a `duplicateOf` field on a dedup hit; human mode prints an `ID: ` line first. + Examples: ```sh From 1487f396ce397e41fa3c11631469ad2cf18b2718 Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 19:00:38 +0100 Subject: [PATCH 28/34] WL-0MSRDEWES0059TZN: Declare canonical node test suite in .pi/test-config.json (audit unblock) The framework's suite detection added a phantom pytest command (agent-skill Python tests under tests/ ship in this repo but CI runs only npm/vitest, and pytest is not installed here), leaving the audit pre-flight cache gate blocked on 'npm --silent test -- tests/cli'. F2 AC1: suiteCommands is the primary resolver and skips convention detection. Also removes ~329 stale tsc emissions in src//tests/ (untracked, gitignored) that shadowed updated .ts sources during vitest in-process CLI runs (39 test failures fixed). --- .pi/test-config.json | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 .pi/test-config.json diff --git a/.pi/test-config.json b/.pi/test-config.json new file mode 100644 index 000000000..61d379d25 --- /dev/null +++ b/.pi/test-config.json @@ -0,0 +1,6 @@ +{ + "suiteCommands": [ + "npm --silent test -- tests/cli", + "npm --silent test -- tests/unit" + ] +} From 1b5fb058ff19ce086146d52c228bd512f8fc2519 Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 19:57:48 +0100 Subject: [PATCH 29/34] WL-0MSUEO1U60072GNK: Test suite: sync identity gate (TDD red phase) --- tests/cli/mock-bin/README.md | 13 +- tests/cli/mock-bin/git | 85 ++++++++++- tests/cli/sync-author-gate.test.ts | 210 ++++++++++++++++++++++++++ tests/sync-author-gate.test.ts | 229 +++++++++++++++++++++++++++++ 4 files changed, 533 insertions(+), 4 deletions(-) create mode 100644 tests/cli/sync-author-gate.test.ts create mode 100644 tests/sync-author-gate.test.ts diff --git a/tests/cli/mock-bin/README.md b/tests/cli/mock-bin/README.md index 24a4ac60a..b238488e9 100644 --- a/tests/cli/mock-bin/README.md +++ b/tests/cli/mock-bin/README.md @@ -8,8 +8,17 @@ used by the CLI integration tests to simulate external commands deterministicall ## How it works - `git` — A POSIX bash script implementing the git subcommands used by the test-suite (e.g., `init`, `clone`, `remote add`, `fetch`, `push`, `show`, `worktree add`, - `ls-files`, `ls-remote`, `show-ref`). It keeps a small `fetch_store` under - `.git/fetch_store/` so `git show :` can be satisfied deterministically. + `ls-files`, `ls-remote`, `show-ref`, `log`, `config user.email`). It keeps a small + `fetch_store` under `.git/fetch_store/` so `git show :` can be satisfied + deterministically. + - `log` (sync author-identity gate, WL-0MSOYWWS4009HTCB): reads a seeded commit + store at `.git/worklog-log-commits` (lines ` ` where `-` + means an empty author email and state is `new`|`synced`). Supports `--format=%ae`, + `--format=%h`, combined `--format=%h%x09%ae` (emits `hashemail`), and + `--not ` (excludes `synced` commits, mirroring `git log A --not B`). + - `config user.email` (get form): reads a seeded value at `.git/user-email` and + exits 0, or exits 1 when the seed is absent (mimics an unset key). Set-form + (`git config user.email `) and all other keys remain silent no-ops. - `gh` — A POSIX bash script that intercepts `gh` issue view, edit, create, and API calls, seeded from a JSONL file. - `wl` — A Node.js script returning predefined test data for common `wl` commands. diff --git a/tests/cli/mock-bin/git b/tests/cli/mock-bin/git index d6a794766..6db75d928 100755 --- a/tests/cli/mock-bin/git +++ b/tests/cli/mock-bin/git @@ -1,6 +1,8 @@ #!/usr/bin/env bash # Minimal git mock used in tests to avoid running the real git binary. -# This mock supports a few commands used by tests: rev-parse, init, clone, add, commit, remote, push +# This mock supports a few commands used by tests: rev-parse, init, clone, add, commit, remote, push, +# fetch, show, show-ref, ls-remote, ls-files, worktree, log (author-identity gate seed), +# config user.email (get form, author-identity gate seed) # Timeout guard: self-terminate after WORKLOG_MOCK_TIMEOUT seconds (default 5s). # This prevents orphaned processes from running indefinitely. @@ -562,7 +564,86 @@ case "$1" in exit 0 ;; config) - # accept config commands silently + # Support the GET form for user.email (used by the sync author-identity + # gate, WL-0MSOYWWS4009HTCB): `git config user.email` / `git config + # --get user.email` prints the seeded value (.git/user-email) and exits 0, + # or exits 1 when the key is unset (mirroring real git's behaviour for a + # missing key). + # Set form (`git config user.email `) and all other keys remain + # silent no-ops. + last="${!#}" + if [ "$last" = "user.email" ]; then + _check_timeout + if [ -f .git/user-email ]; then + cat .git/user-email + exit 0 + fi + exit 1 + fi + exit 0 + ;; + log) + # git log [--not ] [--format=] + # Reads seed store .git/worklog-log-commits (lines " ") + # where email "-" means an empty author email and state is "new" (default) + # or "synced". `--not ` filters out commits marked "synced" + # (already merged as of the last-known sync point), mirroring + # `git log A --not B` semantics. + # --format supports %h/%H (hash), %ae (author email), and %x09 (tab); a + # combined shape like %h%x09%ae emits "hashemail" per line. + ref="" + exclude="" + fmt="" + i=2 + while [ $i -le $# ]; do + eval "arg=\$$i" + case "$arg" in + --not) + i=$((i+1)) + eval "exclude=\$$i" + ;; + --format=*) + fmt="${arg#--format=}" + ;; + --*) + : + ;; + *) + if [ -z "$ref" ]; then ref="$arg"; fi + ;; + esac + i=$((i+1)) + done + seed=".git/worklog-log-commits" + if [ ! -f "$seed" ]; then + exit 0 + fi + while read -r hash email state; do + _check_timeout + [ -z "$hash" ] && continue + state="${state:-new}" + if [ -n "$exclude" ] && [ "$state" = "synced" ]; then + continue + fi + if [ "$email" = "-" ]; then email=""; fi + want_hash=0 + want_email=0 + case "$fmt" in + *%ae*) want_email=1 ;; + esac + case "$fmt" in + *%h*|*%H*) want_hash=1 ;; + esac + if [ $want_hash -eq 1 ] && [ $want_email -eq 1 ]; then + printf '%s\t%s\n' "$hash" "$email" + elif [ $want_hash -eq 1 ]; then + printf '%s\n' "$hash" + elif [ $want_email -eq 1 ]; then + printf '%s\n' "$email" + else + printf '%s %s\n' "$hash" "$email" + fi + done < "$seed" exit 0 ;; *) diff --git a/tests/cli/sync-author-gate.test.ts b/tests/cli/sync-author-gate.test.ts new file mode 100644 index 000000000..5fc7a5ac2 --- /dev/null +++ b/tests/cli/sync-author-gate.test.ts @@ -0,0 +1,210 @@ +/** + * CLI tests for the sync author-identity gate (WL-0MSOYWWS4009HTCB). + * + * Exercises the real `wl sync` command against a mock git ref + * (tests/cli/mock-bin/git). The mock git is seeded via: + * - .git/worklog-log-commits — `git log` store (lines " ") + * - .git/user-email — `git config user.email` get-form value + * + * TDD RED phase (C1): the gate is NOT wired into `wl sync` yet, so the + * refusal tests FAIL today (sync succeeds and imports the remote items, and + * `--allow-foreign-author` is an unknown option). This is the expected, + * documented red state — the tests go green in C2/C3. + * + * Scenarios (parent AC6): + * (a) only own-author commits → sync succeeds, remote items imported + * (b) one empty-email commit → exit != 0, message names commit + remote ref, + * DB untouched (remote item NOT imported) + * (c) different-email commit → refused by default; succeeds with + * --allow-foreign-author + * (d) --dry-run reports the same refusal deterministically + * (Q3) --allow-foreign-author does NOT bypass the empty-email gate + * (AC1) last-synced-ref present → `--not ` excludes already-synced commits + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as path from 'path'; +import { execWithInput, writeConfig, writeInitSemaphore, cliPath } from './cli-helpers.js'; +import { createTempDir, cleanupTempDir } from '../test-utils.js'; + +const REMOTE_REF = 'refs/worklog/remotes/origin/worklog/data'; +const OWN_EMAIL = 'ross@example.com'; +const REMOTE_ITEM_ID = 'GUA-100'; + +function makeItem(id: string, title: string): Record { + return { + id, + title, + description: '', + status: 'open', + priority: 'medium', + sortIndex: 0, + parentId: null, + createdAt: '2024-01-01T00:00:00.000Z', + updatedAt: '2024-01-01T00:00:00.000Z', + tags: [], + assignee: '', + stage: '', + issueType: '', + createdBy: '', + deletedBy: '', + deleteReason: '', + risk: '', + effort: '', + }; +} + +function jsonlLine(type: string, data: unknown): string { + return JSON.stringify({ data, type }); +} + +/** Seed the mock `git log` commit store (lines " "). */ +function writeLogCommits(repo: string, lines: Array<[hash: string, email: string, state: 'new' | 'synced']>): void { + const content = lines + .map(([hash, email, state]) => `${hash} ${email === '' ? '-' : email} ${state}`) + .join('\n'); + fs.writeFileSync(path.join(repo, '.git', 'worklog-log-commits'), `${content}\n`, 'utf8'); +} + +/** Seed the mock `git config user.email` get-form value. */ +function writeUserEmail(repo: string, email: string): void { + fs.writeFileSync(path.join(repo, '.git', 'user-email'), `${email}\n`, 'utf8'); +} + +describe('sync author-identity gate — CLI (WL-0MSOYWWS4009HTCB)', () => { + let repoA: string; // target project (prefix GUA) + let repoB: string; // "remote" whose ref repoA syncs + + beforeEach(() => { + repoA = createTempDir(); + repoB = createTempDir(); + for (const repo of [repoA, repoB]) { + fs.mkdirSync(path.join(repo, '.git'), { recursive: true }); + } + writeConfig(repoA, 'Identity Gate Test', 'GUA'); + writeInitSemaphore(repoA); + // Point repoA's remote at repoB so the mock git fetch reads repoB's .worklog. + fs.writeFileSync(path.join(repoA, '.git', 'remote_origin'), repoB, 'utf8'); + // Configured store identity (the gate reads this via `git config user.email`). + writeUserEmail(repoA, OWN_EMAIL); + + // Seed the "remote" worklog ref with an own-prefix item (GUA-100) so a + // successful sync observably imports something. + fs.mkdirSync(path.join(repoB, '.worklog'), { recursive: true }); + fs.writeFileSync( + path.join(repoB, '.worklog', 'worklog-data.jsonl'), + `${jsonlLine('workitem', makeItem(REMOTE_ITEM_ID, 'Remote item'))}\n`, + 'utf8' + ); + }); + + afterEach(() => { + cleanupTempDir(repoA); + cleanupTempDir(repoB); + }); + + async function sync(args = ''): Promise<{ stdout: string; stderr: string; exitCode: number | null }> { + return execWithInput( + `tsx ${cliPath} --worklog-dir ${repoA}/.worklog sync --no-push ${args}`, + '', + { cwd: repoA, timeout: 60000 } + ); + } + + async function listIds(): Promise { + const list = await execWithInput( + `tsx ${cliPath} --worklog-dir ${repoA}/.worklog list --json`, + '', + { cwd: repoA, timeout: 30000 } + ); + expect(list.exitCode, `${list.stdout}\n${list.stderr}`).toBe(0); + const payload = JSON.parse(list.stdout); + const items: Array<{ id: string }> = payload.workItems ?? payload; + return items.map(it => it.id); + } + + it('(a) sync succeeds when all incoming commits are own-author (real CLI)', async () => { + writeLogCommits(repoA, [['a1b2c3d', OWN_EMAIL, 'new']]); + + const { stdout, stderr, exitCode } = await sync(); + const output = `${stdout}\n${stderr}`; + expect(exitCode, output).toBe(0); + + // Remote item was imported. + expect(await listIds()).toContain(REMOTE_ITEM_ID); + }, 90000); + + it('(b) refuses sync when an incoming commit has an empty author email; no DB mutation (real CLI)', async () => { + // Seed the mock git log with the known polluted commit 5fc880a (empty email). + writeLogCommits(repoA, [['5fc880a', '', 'new']]); + + const { stdout, stderr, exitCode } = await sync(); + const output = `${stdout}\n${stderr}`; + expect(exitCode, output).not.toBe(0); + + // AC2: message names the offending commit AND the remote ref. + expect(output).toContain('5fc880a'); + expect(output).toContain(REMOTE_REF); + + // AC6b: DB untouched — the remote item must NOT have been imported. + expect(await listIds()).not.toContain(REMOTE_ITEM_ID); + }, 90000); + + it('(c) refuses a different-email commit by default; succeeds with --allow-foreign-author (real CLI)', async () => { + writeLogCommits(repoA, [['6b9e493', 'other@example.com', 'new']]); + + // Default: refused, message names the commit + remote ref. + const refused = await sync(); + const refusedOutput = `${refused.stdout}\n${refused.stderr}`; + expect(refused.exitCode, refusedOutput).not.toBe(0); + expect(refusedOutput).toContain('6b9e493'); + expect(refusedOutput).toContain(REMOTE_REF); + + // Override: succeeds and imports the remote item. + const allowed = await sync('--allow-foreign-author'); + const allowedOutput = `${allowed.stdout}\n${allowed.stderr}`; + expect(allowed.exitCode, allowedOutput).toBe(0); + expect(await listIds()).toContain(REMOTE_ITEM_ID); + }, 120000); + + it('(d) --dry-run reports the same refusal deterministically (real CLI)', async () => { + writeLogCommits(repoA, [['5fc880a', '', 'new']]); + + const { stdout, stderr, exitCode } = await sync('--dry-run'); + const output = `${stdout}\n${stderr}`; + expect(exitCode, output).not.toBe(0); + expect(output).toContain('5fc880a'); + expect(output).toContain(REMOTE_REF); + expect(await listIds()).not.toContain(REMOTE_ITEM_ID); + }, 90000); + + it('(Q3) --allow-foreign-author does NOT bypass the empty-email gate (real CLI)', async () => { + writeLogCommits(repoA, [['5fc880a', '', 'new']]); + + const { stdout, stderr, exitCode } = await sync('--allow-foreign-author'); + const output = `${stdout}\n${stderr}`; + expect(exitCode, output).not.toBe(0); + expect(output).toContain('5fc880a'); + expect(await listIds()).not.toContain(REMOTE_ITEM_ID); + }, 90000); + + it('(AC1) honors the last-synced-ref: `--not ` excludes already-synced commits (real CLI)', async () => { + // Record a last-known sync point; the empty-email commit is already merged + // (state "synced"), so `git log --not ` must exclude it. + fs.writeFileSync( + path.join(repoA, '.worklog', 'last-synced-ref'), + 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\n', + 'utf8' + ); + writeLogCommits(repoA, [ + ['5fc880a', '', 'synced'], // empty email, but already merged → excluded by --not + ['a1b2c3d', OWN_EMAIL, 'new'], // own new commit → inspected + ]); + + const { stdout, stderr, exitCode } = await sync(); + const output = `${stdout}\n${stderr}`; + expect(exitCode, output).toBe(0); + expect(await listIds()).toContain(REMOTE_ITEM_ID); + }, 90000); +}); diff --git a/tests/sync-author-gate.test.ts b/tests/sync-author-gate.test.ts new file mode 100644 index 000000000..3c1ab6b4b --- /dev/null +++ b/tests/sync-author-gate.test.ts @@ -0,0 +1,229 @@ +/** + * Unit tests for the sync author-identity gate (WL-0MSOYWWS4009HTCB). + * + * TDD RED phase (C1): the gate is NOT implemented yet, so this module fails + * to load with "does not provide an export named 'checkAuthorIdentity'" from + * src/sync.js. This is the expected, documented red state — the tests go + * green when C2 implements `checkAuthorIdentity` in src/sync.ts. + * + * ── Contract under test (implemented by C2) ────────────────────────────── + * + * export interface IncomingCommit { + * hash: string; // short commit hash (e.g. '5fc880a') + * authorEmail: string; // may be '' when the commit has an empty author email + * } + * + * export interface AuthorIdentityOptions { + * remoteTrackingRef: string; // e.g. refs/worklog/remotes/origin/worklog/data + * expectedAuthorEmail?: string; // repo `git config user.email`; undefined = not configured + * allowForeignAuthor?: boolean; // `wl sync --allow-foreign-author` + * } + * + * export interface AuthorIdentityViolation { + * commit: string; + * authorEmail: string; + * reason: 'empty-email' | 'foreign-email'; + * } + * + * export interface AuthorIdentityResult { + * allowed: boolean; + * violations: AuthorIdentityViolation[]; + * message: string; // refusal message naming the offending commit(s) + remote ref; '' when allowed + * } + * + * export function checkAuthorIdentity( + * commits: IncomingCommit[], + * options: AuthorIdentityOptions + * ): AuthorIdentityResult + * + * ── Gate rules (parent ACs + plan resolutions) ─────────────────────────── + * (a) only own-author commits → allowed. + * (b) an incoming commit with an EMPTY author email → refused UNCONDITIONALLY + * (Q3: `--allow-foreign-author` never bypasses the empty-email gate). + * (c) a commit whose author email differs from expectedAuthorEmail → refused + * by default; allowed when allowForeignAuthor=true. + * (d) deterministic: the same input always yields the same result (this is + * what makes dry-run report the same refusal as a real sync). + * (Q2) when expectedAuthorEmail is undefined (user.email unset) only the + * empty-email gate applies; the foreign-email comparison is skipped. + * (AC2) the refusal message names the offending commit(s) and the remote ref. + * + * The git-log invocation shape (`git log --format=... --not `) + * is exercised at the CLI level in tests/cli/sync-author-gate.test.ts against + * the mock git (tests/cli/mock-bin/git), which supports both `--format=%ae` + * and combined `--format=%h%x09%ae` shapes plus `--not `. + */ + +import { describe, it, expect } from 'vitest'; +import { checkAuthorIdentity } from '../src/sync.js'; +import type { IncomingCommit, AuthorIdentityOptions } from '../src/sync.js'; + +const REMOTE_REF = 'refs/worklog/remotes/origin/worklog/data'; +const OWN_EMAIL = 'ross@example.com'; +const FOREIGN_EMAIL = 'other@example.com'; + +function own(hash: string, email: string = OWN_EMAIL): IncomingCommit { + return { hash, authorEmail: email }; +} + +function emptyEmail(hash: string): IncomingCommit { + return { hash, authorEmail: '' }; +} + +function options(overrides: Partial = {}): AuthorIdentityOptions { + return { + remoteTrackingRef: REMOTE_REF, + expectedAuthorEmail: OWN_EMAIL, + ...overrides, + }; +} + +describe('checkAuthorIdentity (WL-0MSOYWWS4009HTCB)', () => { + // (a) only own-author commits → allowed + it('allows when every incoming commit is authored by the store identity', () => { + const result = checkAuthorIdentity( + [own('a1b2c3d'), own('e4f5a6b')], + options() + ); + expect(result.allowed).toBe(true); + expect(result.violations).toEqual([]); + expect(result.message).toBe(''); + }); + + it('allows an empty incoming commit set (no new commits since last sync)', () => { + const result = checkAuthorIdentity([], options()); + expect(result.allowed).toBe(true); + expect(result.violations).toEqual([]); + expect(result.message).toBe(''); + }); + + // (b) one empty-email commit → refused + it('refuses a single empty-email commit, naming the commit and the remote ref', () => { + const result = checkAuthorIdentity([emptyEmail('5fc880a')], options()); + + expect(result.allowed).toBe(false); + expect(result.violations).toEqual([ + { commit: '5fc880a', authorEmail: '', reason: 'empty-email' }, + ]); + expect(result.message).toContain('5fc880a'); + expect(result.message).toContain(REMOTE_REF); + }); + + it('refuses when ANY commit in the batch has an empty email', () => { + const result = checkAuthorIdentity( + [own('a1b2c3d'), emptyEmail('5fc880a'), own('e4f5a6b')], + options() + ); + + expect(result.allowed).toBe(false); + expect(result.violations.map(v => v.commit)).toEqual(['5fc880a']); + expect(result.violations[0].reason).toBe('empty-email'); + }); + + // (Q3) --allow-foreign-author never bypasses the empty-email gate + it('refuses an empty-email commit even when allowForeignAuthor=true (Q3)', () => { + const result = checkAuthorIdentity( + [emptyEmail('5fc880a')], + options({ allowForeignAuthor: true }) + ); + + expect(result.allowed).toBe(false); + expect(result.violations[0].reason).toBe('empty-email'); + }); + + // (c) different-email commit → refused by default + it('refuses a different-email commit by default, naming the commit and the remote ref', () => { + const result = checkAuthorIdentity( + [own('6b9e493', FOREIGN_EMAIL)], + options() + ); + + expect(result.allowed).toBe(false); + expect(result.violations).toEqual([ + { commit: '6b9e493', authorEmail: FOREIGN_EMAIL, reason: 'foreign-email' }, + ]); + expect(result.message).toContain('6b9e493'); + expect(result.message).toContain(REMOTE_REF); + }); + + // (c) different-email commit → allowed when allowForeignAuthor=true + it('allows a different-email commit when allowForeignAuthor=true', () => { + const result = checkAuthorIdentity( + [own('6b9e493', FOREIGN_EMAIL)], + options({ allowForeignAuthor: true }) + ); + + expect(result.allowed).toBe(true); + expect(result.violations).toEqual([]); + expect(result.message).toBe(''); + }); + + it('allows mixed own + foreign emails when allowForeignAuthor=true, but still refuses empty emails', () => { + const result = checkAuthorIdentity( + [own('a1b2c3d'), own('6b9e493', FOREIGN_EMAIL), emptyEmail('5fc880a')], + options({ allowForeignAuthor: true }) + ); + + expect(result.allowed).toBe(false); + expect(result.violations).toEqual([ + { commit: '5fc880a', authorEmail: '', reason: 'empty-email' }, + ]); + }); + + // (Q2) expectedAuthorEmail undefined → only the empty-email gate applies + it('skips the foreign-email comparison when expectedAuthorEmail is undefined (Q2)', () => { + const result = checkAuthorIdentity( + [own('6b9e493', FOREIGN_EMAIL)], + options({ expectedAuthorEmail: undefined }) + ); + + expect(result.allowed).toBe(true); + expect(result.violations).toEqual([]); + }); + + it('still enforces the empty-email gate when expectedAuthorEmail is undefined (Q2)', () => { + const result = checkAuthorIdentity( + [emptyEmail('5fc880a')], + options({ expectedAuthorEmail: undefined }) + ); + + expect(result.allowed).toBe(false); + expect(result.violations[0].reason).toBe('empty-email'); + }); + + // (d) deterministic: identical input → identical result (dry-run parity) + it('is deterministic: the same input yields the same result on every call', () => { + const commits = [own('a1b2c3d'), emptyEmail('5fc880a')]; + const first = checkAuthorIdentity(commits, options()); + const second = checkAuthorIdentity(commits, options()); + + expect(second).toEqual(first); + expect(second.allowed).toBe(false); + expect(second.violations).toEqual(first.violations); + expect(second.message).toBe(first.message); + }); + + // (AC2) message naming: multiple offenders all named + it('names every offending commit in the refusal message', () => { + const result = checkAuthorIdentity( + [own('6b9e493', FOREIGN_EMAIL), emptyEmail('5fc880a'), own('a1b2c3d')], + options() + ); + + expect(result.allowed).toBe(false); + expect(result.message).toContain('6b9e493'); + expect(result.message).toContain('5fc880a'); + expect(result.message).toContain(REMOTE_REF); + }); + + it('does not name allowed commits as violations', () => { + const result = checkAuthorIdentity( + [own('a1b2c3d'), emptyEmail('5fc880a')], + options() + ); + + const namedCommits = result.violations.map(v => v.commit); + expect(namedCommits).toEqual(['5fc880a']); + expect(namedCommits).not.toContain('a1b2c3d'); + }); +}); From 418061ea828e0056b23e8aa652a0f5b5cc6ebbc7 Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 20:10:44 +0100 Subject: [PATCH 30/34] WL-0MSUEO62N0019P3S: Implement checkAuthorIdentity gate in src/sync.ts --- src/sync.ts | 120 +++++++++++++++++++++++++++++++++ tests/sync-author-gate.test.ts | 90 ++++++++++++++++++++++++- 2 files changed, 208 insertions(+), 2 deletions(-) diff --git a/src/sync.ts b/src/sync.ts index a3d3ab067..d585b5b3a 100644 --- a/src/sync.ts +++ b/src/sync.ts @@ -261,6 +261,126 @@ export function filterRemoteDataByPrefix( }; } +/** + * Sync author-identity gate (WL-0MSOYWWS4009HTCB). + * + * Refuses to merge worklog data when the remote ref contains commits authored + * by an identity other than the store's configured identity — a second store + * with an empty/foreign git identity can otherwise silently overwrite work + * items and invalidate audits. Pure + deterministic: the same input always + * yields the same result, so dry-run and normal syncs report the same refusal. + */ + +export interface IncomingCommit { + hash: string; // short commit hash (e.g. '5fc880a') + authorEmail: string; // may be '' when the commit has an empty author email +} + +export interface AuthorIdentityOptions { + remoteTrackingRef: string; // e.g. refs/worklog/remotes/origin/worklog/data + expectedAuthorEmail?: string; // repo `git config user.email`; undefined = not configured + allowForeignAuthor?: boolean; // `wl sync --allow-foreign-author` +} + +export interface AuthorIdentityViolation { + commit: string; + authorEmail: string; + reason: 'empty-email' | 'foreign-email'; +} + +export interface AuthorIdentityResult { + allowed: boolean; + violations: AuthorIdentityViolation[]; + message: string; // refusal message naming the offending commit(s) + remote ref; '' when allowed +} + +/** + * Inspect the author emails of commits on the remote tracking ref since the + * last-known sync point (parent AC1): + * + * git log --format=%h%x09%ae [--not ] + * + * The combined `%h%x09%ae` format carries both the short hash and the author + * email per line (a superset of `--format=%ae`) so the gate can name offending + * commits in its refusal message. When `lastSyncedRef` is absent/invalid the + * whole remote ref history is scanned (no `--not` exclusion). + */ +export async function getRemoteAuthorCommits( + remoteTrackingRef: string, + lastSyncedRef?: string +): Promise { + const args = ['log', remoteTrackingRef, '--format=%h%x09%ae']; + if (lastSyncedRef && lastSyncedRef.trim()) { + args.push('--not', lastSyncedRef.trim()); + } + + let stdout: string; + try { + stdout = await execGitCaptureStdout(args); + } catch { + // Unreadable/missing ref → no incoming commits to gate. + return []; + } + + const commits: IncomingCommit[] = []; + for (const line of stdout.split('\n')) { + if (!line.trim()) continue; + const [hash, email] = line.split('\t'); + commits.push({ hash: hash || '', authorEmail: email ?? '' }); + } + return commits; +} + +/** + * Pure gate decision over the incoming commits (parent AC2/AC3 + plan Q2/Q3): + * + * - EMPTY author email → unconditional refusal. `allowForeignAuthor` never + * bypasses the empty-email gate (Q3). + * - email differs from `expectedAuthorEmail` → refused by default; allowed + * when `allowForeignAuthor=true`. + * - `expectedAuthorEmail` undefined (user.email unset) → only the + * empty-email gate applies; the foreign-email comparison is skipped (Q2). + * + * The refusal message names the offending commit(s) and the remote ref. + */ +export function checkAuthorIdentity( + commits: IncomingCommit[], + options: AuthorIdentityOptions +): AuthorIdentityResult { + const { remoteTrackingRef, expectedAuthorEmail, allowForeignAuthor = false } = options; + const violations: AuthorIdentityViolation[] = []; + + for (const commit of commits) { + const email = (commit.authorEmail || '').trim(); + if (email === '') { + violations.push({ commit: commit.hash, authorEmail: '', reason: 'empty-email' }); + } else if (expectedAuthorEmail !== undefined && email !== expectedAuthorEmail && !allowForeignAuthor) { + violations.push({ commit: commit.hash, authorEmail: email, reason: 'foreign-email' }); + } + } + + const allowed = violations.length === 0; + const message = allowed ? '' : buildIdentityRefusalMessage(remoteTrackingRef, violations, expectedAuthorEmail); + return { allowed, violations, message }; +} + +function buildIdentityRefusalMessage( + remoteTrackingRef: string, + violations: AuthorIdentityViolation[], + expectedAuthorEmail: string | undefined +): string { + const detailLines = violations.map(v => + v.reason === 'empty-email' + ? `- ${v.commit}: empty author email` + : `- ${v.commit}: author email '${v.authorEmail}' does not match configured user.email '${expectedAuthorEmail}'` + ); + return ( + `Refusing to merge worklog data from ${remoteTrackingRef}: ` + + `${violations.length} incoming commit(s) fail the author-identity gate.\n` + + detailLines.join('\n') + ); +} + /** * Terminal (closed) workflow stages. An item with `completed` status and a * terminal stage is a protected close state: it must never be silently diff --git a/tests/sync-author-gate.test.ts b/tests/sync-author-gate.test.ts index 3c1ab6b4b..01548d014 100644 --- a/tests/sync-author-gate.test.ts +++ b/tests/sync-author-gate.test.ts @@ -54,10 +54,17 @@ * and combined `--format=%h%x09%ae` shapes plus `--not `. */ -import { describe, it, expect } from 'vitest'; -import { checkAuthorIdentity } from '../src/sync.js'; +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as path from 'path'; +import * as os from 'os'; +import { fileURLToPath } from 'url'; +import { checkAuthorIdentity, getRemoteAuthorCommits } from '../src/sync.js'; import type { IncomingCommit, AuthorIdentityOptions } from '../src/sync.js'; +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const mockBinDir = path.join(__dirname, 'cli', 'mock-bin'); + const REMOTE_REF = 'refs/worklog/remotes/origin/worklog/data'; const OWN_EMAIL = 'ross@example.com'; const FOREIGN_EMAIL = 'other@example.com'; @@ -227,3 +234,82 @@ describe('checkAuthorIdentity (WL-0MSOYWWS4009HTCB)', () => { expect(namedCommits).not.toContain('a1b2c3d'); }); }); + +// ── getRemoteAuthorCommits: git log invocation shape (parent AC1) ──────── +// Runs against the mock git (tests/cli/mock-bin/git): `log` reads the seeded +// commit store (.git/worklog-log-commits, lines " ", +// state "synced" excluded when --not is present) and `config user.email` +// reads the seeded .git/user-email value. + +describe('getRemoteAuthorCommits (WL-0MSOYWWS4009HTCB)', () => { + let tmp: string; + let repo: string; + let oldCwd: string; + let oldPath: string | undefined; + + beforeEach(() => { + tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'wl-sync-gate-')); + repo = path.join(tmp, 'repo'); + fs.mkdirSync(path.join(repo, '.git'), { recursive: true }); + fs.writeFileSync(path.join(repo, '.git', 'remote_origin'), path.join(tmp, 'remote'), 'utf8'); + fs.mkdirSync(path.join(repo, '.worklog'), { recursive: true }); + + oldCwd = process.cwd(); + oldPath = process.env.PATH; + process.chdir(repo); + process.env.PATH = `${mockBinDir}${path.delimiter}${oldPath || ''}`; + }); + + afterEach(() => { + process.chdir(oldCwd); + process.env.PATH = oldPath; + fs.rmSync(tmp, { recursive: true, force: true }); + }); + + function seedLogCommits(lines: Array<[hash: string, email: string, state: 'new' | 'synced']>): void { + const content = lines + .map(([hash, email, state]) => `${hash} ${email === '' ? '-' : email} ${state}`) + .join('\n'); + fs.writeFileSync(path.join(repo, '.git', 'worklog-log-commits'), `${content}\n`, 'utf8'); + } + + it('returns hash + authorEmail per incoming commit (AC1: git log --format=%h%x09%ae)', async () => { + seedLogCommits([ + ['a1b2c3d', OWN_EMAIL, 'new'], + ['5fc880a', '', 'new'], + ]); + + const commits = await getRemoteAuthorCommits(REMOTE_REF); + expect(commits).toEqual([ + { hash: 'a1b2c3d', authorEmail: OWN_EMAIL }, + { hash: '5fc880a', authorEmail: '' }, + ]); + }); + + it('excludes already-synced commits when lastSyncedRef is provided (AC1: --not )', async () => { + seedLogCommits([ + ['5fc880a', '', 'synced'], // empty email but already merged → excluded + ['a1b2c3d', OWN_EMAIL, 'new'], // own new commit → inspected + ]); + + const commits = await getRemoteAuthorCommits(REMOTE_REF, 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'); + expect(commits).toEqual([{ hash: 'a1b2c3d', authorEmail: OWN_EMAIL }]); + }); + + it('scans the full remote ref history when lastSyncedRef is absent or invalid', async () => { + seedLogCommits([ + ['5fc880a', '', 'synced'], + ['6b9e493', FOREIGN_EMAIL, 'new'], + ]); + + // absent + expect((await getRemoteAuthorCommits(REMOTE_REF)).length).toBe(2); + // whitespace-only → treated as absent + expect((await getRemoteAuthorCommits(REMOTE_REF, ' ')).length).toBe(2); + }); + + it('returns [] when the remote ref has no commits (mock log store absent)', async () => { + const commits = await getRemoteAuthorCommits(REMOTE_REF); + expect(commits).toEqual([]); + }); +}); From f33457401ac5aaef4ebd7efdbdf9a27e91a62a94 Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 20:45:41 +0100 Subject: [PATCH 31/34] WL-0MSUEOAQO0081TJH: Wire identity gate into wl sync + config flag + sync-point persistence --- packages/shared/src/types.ts | 8 +++ src/cli-types.ts | 2 + src/commands/sync.ts | 44 ++++++++++-- src/sync.ts | 108 +++++++++++++++++++++++++++-- tests/cli/mock-bin/git | 10 +++ tests/cli/sync-author-gate.test.ts | 32 +++++++++ 6 files changed, 196 insertions(+), 8 deletions(-) diff --git a/packages/shared/src/types.ts b/packages/shared/src/types.ts index 00f476a36..4706ceaf9 100644 --- a/packages/shared/src/types.ts +++ b/packages/shared/src/types.ts @@ -183,6 +183,14 @@ export interface WorklogConfig { auditWriteEnabled?: boolean; syncRemote?: string; syncBranch?: string; + /** + * Allow `wl sync` to merge commits authored by a different identity than + * the store's configured `user.email` (identity gate WL-0MSOYWWS4009HTCB). + * Default false — the sync refuses foreign-author commits. The CLI flag + * `--allow-foreign-author` takes precedence over this config value. + * Never bypasses the empty-author-email gate. + */ + syncAllowForeignAuthor?: boolean; githubRepo?: string; githubLabelPrefix?: string; githubImportCreateNew?: boolean; diff --git a/src/cli-types.ts b/src/cli-types.ts index f1acf1133..5f3446a8a 100644 --- a/src/cli-types.ts +++ b/src/cli-types.ts @@ -152,6 +152,8 @@ export interface SyncOptions { reSortSync?: boolean; /** Lock-aware guard: skip (exit 0) if another sync is already in progress */ ifIdle?: boolean; + /** Override the author-identity gate for foreign-email commits (never bypasses the empty-email gate) */ + allowForeignAuthor?: boolean; } export interface SyncDebugOptions { diff --git a/src/commands/sync.ts b/src/commands/sync.ts index e8a848392..2176e233b 100644 --- a/src/commands/sync.ts +++ b/src/commands/sync.ts @@ -6,7 +6,7 @@ import type { PluginContext } from '../plugin-types.js'; import type { SyncOptions, SyncDebugOptions } from '../cli-types.js'; import type { WorkItem, Comment, DependencyEdge } from '../types.js'; import type { GitTarget, SyncResult } from '../sync.js'; -import { getRemoteDataFileContent, gitPushDataFileToBranch, mergeWorkItems, mergeComments, mergeDependencyEdges, assertDataFileInCwdRepo, filterRemoteDataByPrefix } from '../sync.js'; +import { getRemoteDataFileContent, getRemoteDataFileContentWithRef, gitPushDataFileToBranch, mergeWorkItems, mergeComments, mergeDependencyEdges, assertDataFileInCwdRepo, filterRemoteDataByPrefix, enforceAuthorIdentityGate, getConfiguredUserEmail, getRemoteTrackingRefSha, writeLastSyncedRef } from '../sync.js'; import { DEFAULT_GIT_REMOTE, DEFAULT_GIT_BRANCH } from '../sync-defaults.js'; import { importFromJsonlContent } from '../jsonl.js'; import { mergeAuditResults } from '../sync.js'; @@ -27,6 +27,7 @@ export function getSyncDefaults(config?: ReturnType) { return { gitRemote: config?.syncRemote || DEFAULT_GIT_REMOTE, gitBranch: config?.syncBranch || DEFAULT_GIT_BRANCH, + allowForeignAuthor: config?.syncAllowForeignAuthor === true, }; } @@ -43,6 +44,7 @@ export async function performSync( silent?: boolean; isJsonMode?: boolean; isVerbose?: boolean; + allowForeignAuthor?: boolean; } ): Promise { const isJsonMode = options.isJsonMode ?? false; @@ -89,8 +91,23 @@ export async function performSync( let remoteEdges: DependencyEdge[] = []; const localAudits = db.getAllAuditResults(); - - const remoteContent = await getRemoteDataFileContent(options.file, gitTarget); + + const remoteContentResult = await getRemoteDataFileContentWithRef(options.file, gitTarget); + const remoteContent = remoteContentResult.content; + + // Author-identity gate (WL-0MSOYWWS4009HTCB): before any merge/import + // side-effect, refuse when the remote tracking ref contains commits whose + // author email is empty or differs from the store's configured user.email. + // Runs after the remote fetch; dry-run and normal modes report the same + // refusal deterministically (AC4). An absent remote (no ref) is allowed. + if (remoteContentResult.remoteTrackingRef) { + const expectedAuthorEmail = await getConfiguredUserEmail(); + await enforceAuthorIdentityGate(remoteContentResult.remoteTrackingRef, options.file, { + expectedAuthorEmail, + allowForeignAuthor: options.allowForeignAuthor, + }); + } + let remoteAudits: any[] = []; if (remoteContent) { const remoteData = importFromJsonlContent(remoteContent); @@ -316,6 +333,21 @@ export async function performSync( } } + // Persist the last-known sync point (remote tracking ref tip) so the next + // sync only inspects commits since this point (identity gate, AC5). Written + // only after a successful NON-dry-run sync — a refused sync never reaches + // here, so the state file still points at the last accepted ref. + if (!options.dryRun && remoteContentResult.remoteTrackingRef) { + try { + const refSha = await getRemoteTrackingRefSha(remoteContentResult.remoteTrackingRef); + if (refSha) { + writeLastSyncedRef(options.file, refSha); + } + } catch { + // Non-critical: absent state file just widens the next scan. + } + } + logConflictDetails(result, itemMergeResult.merged, logLine); finalizeLog(); @@ -371,6 +403,7 @@ export default function register(ctx: PluginContext): void { .option('--git-branch ', 'Git ref to store worklog data (use refs/worklog/data to avoid GitHub PR banners)', DEFAULT_GIT_BRANCH) .option('--no-push', 'Skip pushing changes back to git') .option('--dry-run', 'Show what would be synced without making changes') + .option('--allow-foreign-author', 'Allow merging commits authored by a different identity than the store user.email (never bypasses the empty-author-email gate) — overrides syncAllowForeignAuthor config') .option('--if-idle', 'Skip (exit 0) if another sync is already in progress — lock-aware guard for auto-sync spawners; prevents process pile-up under lock contention') .option('--no-re-sort', 'Skip automatic re-sort after sync') .option('--re-sort-sync', 'Force a synchronous re-sort after sync', false) @@ -382,6 +415,8 @@ export default function register(ctx: PluginContext): void { const defaults = getSyncDefaults(config || undefined); const gitRemote = options.gitRemote || defaults.gitRemote; const gitBranch = options.gitBranch || defaults.gitBranch; + // Author-identity gate override: CLI flag wins over config (default false). + const allowForeignAuthor = options.allowForeignAuthor ?? defaults.allowForeignAuthor; // Re-sort control options (apply once after batch completes) const reSortNo = Boolean((options as any).noReSort) || false; @@ -402,7 +437,8 @@ export default function register(ctx: PluginContext): void { dryRun: options.dryRun ?? false, silent: false, isJsonMode, - isVerbose + isVerbose, + allowForeignAuthor }), options.ifIdle ? { skipIfLocked: true } : undefined ); diff --git a/src/sync.ts b/src/sync.ts index d585b5b3a..d5be9679d 100644 --- a/src/sync.ts +++ b/src/sync.ts @@ -381,6 +381,94 @@ function buildIdentityRefusalMessage( ); } +/** + * Read the repo's configured `user.email` (the store identity the gate + * compares incoming commit authors against). Returns undefined when the key + * is unset or git is unavailable — the caller then skips the foreign-email + * comparison entirely (plan Q2) while still enforcing the empty-email gate. + */ +export async function getConfiguredUserEmail(): Promise { + try { + const { stdout } = await gitExecAsync('git config user.email'); + const email = stdout.trim(); + return email || undefined; + } catch { + return undefined; + } +} + +/** + * Path of the last-synced-ref state file (mirrors the last-sync-time + * pattern). Persists the remote tracking ref tip after each successful + * non-dry-run sync so the next sync only inspects commits since that point. + */ +export function getLastSyncedRefPath(dataFilePath: string): string { + return path.join(path.dirname(dataFilePath), 'last-synced-ref'); +} + +/** + * Read the last-known sync point (remote tracking ref tip). Returns undefined + * when the state file is absent, empty, or unreadable — the gate then scans + * the whole remote ref history. + */ +export function readLastSyncedRef(dataFilePath: string): string | undefined { + try { + const raw = fs.readFileSync(getLastSyncedRefPath(dataFilePath), 'utf8').trim(); + return raw || undefined; + } catch { + return undefined; + } +} + +/** + * Persist the last-known sync point after a successful non-dry-run sync. + */ +export function writeLastSyncedRef(dataFilePath: string, refSha: string): void { + try { + fs.writeFileSync(getLastSyncedRefPath(dataFilePath), `${refSha}\n`, 'utf8'); + } catch { + // Non-critical state file; a failed write only means the next sync scans + // more history than necessary (the gate still runs). + } +} + +/** + * Resolve the tip SHA of the remote tracking ref (used to persist + * last-synced-ref). Returns undefined when the ref cannot be resolved. + */ +export async function getRemoteTrackingRefSha(remoteTrackingRef: string): Promise { + try { + const { stdout } = await gitExecAsync(`git rev-parse ${escapeShellArg(remoteTrackingRef)}`); + return stdout.trim() || undefined; + } catch { + return undefined; + } +} + +/** + * Wire the author-identity gate into a sync flow (parent AC4): inspect the + * incoming commits on the remote tracking ref since the last sync point and + * throw a refusal error naming the offending commit(s) + remote ref when the + * gate fails. Runs after the remote fetch, before any merge/import + * side-effects. Dry-run and normal modes report the refusal identically. + */ +export async function enforceAuthorIdentityGate( + remoteTrackingRef: string, + dataFilePath: string, + options: { expectedAuthorEmail?: string; allowForeignAuthor?: boolean } +): Promise { + const lastSyncedRef = readLastSyncedRef(dataFilePath); + const commits = await getRemoteAuthorCommits(remoteTrackingRef, lastSyncedRef); + const result = checkAuthorIdentity(commits, { + remoteTrackingRef, + expectedAuthorEmail: options.expectedAuthorEmail, + allowForeignAuthor: options.allowForeignAuthor, + }); + if (!result.allowed) { + throw new Error(result.message); + } +} + /** * Terminal (closed) workflow stages. An item with `completed` status and a * terminal stage is a protected close state: it must never be silently @@ -1007,7 +1095,15 @@ function getRepoRelativePath(repoRootPath: string, filePath: string): { absolute return { absolutePath, relativePath }; } -export async function getRemoteDataFileContent(dataFilePath: string, target: GitTarget): Promise { +/** + * Fetch the remote worklog ref and return both its content and the remote + * tracking ref that was materialized. The tracking ref is what the + * author-identity gate (WL-0MSOYWWS4009HTCB) inspects. + */ +export async function getRemoteDataFileContentWithRef( + dataFilePath: string, + target: GitTarget +): Promise<{ content: string | null; remoteTrackingRef: string }> { // Cross-project safety guard: never read the remote worklog ref of a // different repository than the one owning the data file. await assertDataFileInCwdRepo(dataFilePath); @@ -1020,18 +1116,22 @@ export async function getRemoteDataFileContent(dataFilePath: string, target: Git const { hasRemote, remoteTrackingRef } = await fetchTargetRef(target); if (!hasRemote) { - return null; + return { content: null, remoteTrackingRef }; } const refAndPath = `${remoteTrackingRef}:${relativePath}`; try { // Avoid exec() maxBuffer issues for large JSONL. - return await execGitCaptureStdout(['show', refAndPath]); + return { content: await execGitCaptureStdout(['show', refAndPath]), remoteTrackingRef }; } catch { - return null; + return { content: null, remoteTrackingRef }; } } +export async function getRemoteDataFileContent(dataFilePath: string, target: GitTarget): Promise { + return (await getRemoteDataFileContentWithRef(dataFilePath, target)).content; +} + function removeWorktreeFiles(worktreePath: string): void { for (const name of fs.readdirSync(worktreePath)) { if (name === '.git') continue; diff --git a/tests/cli/mock-bin/git b/tests/cli/mock-bin/git index 6db75d928..5b357c8a1 100755 --- a/tests/cli/mock-bin/git +++ b/tests/cli/mock-bin/git @@ -117,6 +117,16 @@ case "$1" in echo "hooks" exit 0 fi + # Resolve a bare ref (e.g. refs/worklog/remotes/origin/worklog/data) from + # .git/ — used by the sync identity gate to persist the last-known + # sync point (WL-0MSOYWWS4009HTCB). Mirrors real git's ref file layout + # (.git/refs/... is where `git fetch` writes the tracking ref). + refClean=$(echo "$2" | sed "s/^'//;s/'$//") + if [ -n "$refClean" ] && [ -f ".git/$refClean" ]; then + _check_timeout + cat ".git/$refClean" + exit 0 + fi ;; init) # support --bare flag diff --git a/tests/cli/sync-author-gate.test.ts b/tests/cli/sync-author-gate.test.ts index 5fc7a5ac2..5709823c2 100644 --- a/tests/cli/sync-author-gate.test.ts +++ b/tests/cli/sync-author-gate.test.ts @@ -207,4 +207,36 @@ describe('sync author-identity gate — CLI (WL-0MSOYWWS4009HTCB)', () => { expect(exitCode, output).toBe(0); expect(await listIds()).toContain(REMOTE_ITEM_ID); }, 90000); + + it('(AC3) syncAllowForeignAuthor config flag allows foreign commits; CLI flag wins over config (real CLI)', async () => { + writeLogCommits(repoA, [['6b9e493', 'other@example.com', 'new']]); + + // config.yaml sets syncAllowForeignAuthor: true → foreign commit allowed. + const configPath = path.join(repoA, '.worklog', 'config.yaml'); + const base = fs.readFileSync(configPath, 'utf8'); + fs.writeFileSync(configPath, `${base}\nsyncAllowForeignAuthor: true\n`, 'utf8'); + const viaConfig = await sync(); + expect(viaConfig.exitCode, `${viaConfig.stdout}\n${viaConfig.stderr}`).toBe(0); + + // CLI flag wins over config: with --allow-foreign-author the gate is + // overridden even when config says refuse (default). + writeLogCommits(repoA, [['6b9e493', 'other@example.com', 'new']]); + fs.writeFileSync(configPath, base, 'utf8'); + const viaFlag = await sync('--allow-foreign-author'); + expect(viaFlag.exitCode, `${viaFlag.stdout}\n${viaFlag.stderr}`).toBe(0); + }, 120000); + + it('(AC5) writes .worklog/last-synced-ref after a successful non-dry-run sync (real CLI)', async () => { + writeLogCommits(repoA, [['a1b2c3d', OWN_EMAIL, 'new']]); + + const { stdout, stderr, exitCode } = await sync(); + const output = `${stdout}\n${stderr}`; + expect(exitCode, output).toBe(0); + + // The tracking ref tip sha was persisted for the next gate scan. + const lastSyncedRefPath = path.join(repoA, '.worklog', 'last-synced-ref'); + expect(fs.existsSync(lastSyncedRefPath)).toBe(true); + const sha = fs.readFileSync(lastSyncedRefPath, 'utf8').trim(); + expect(sha).toMatch(/^[0-9a-f]{40}$/); + }, 90000); }); From aab3885479205e86630db8650f8ee9d1970a42a5 Mon Sep 17 00:00:00 2001 From: pi Date: Sat, 15 Aug 2026 20:54:58 +0100 Subject: [PATCH 32/34] WL-0MSUEOEE10049YKR: Manual verification vs polluted ref + identity-gate docs --- DATA_SYNCING.md | 37 +++++++++++ README.md | 1 + docs/SYNC_IDENTITY_GATE.md | 127 +++++++++++++++++++++++++++++++++++++ 3 files changed, 165 insertions(+) create mode 100644 docs/SYNC_IDENTITY_GATE.md diff --git a/DATA_SYNCING.md b/DATA_SYNCING.md index a1ef0e6d2..7894b8c57 100644 --- a/DATA_SYNCING.md +++ b/DATA_SYNCING.md @@ -64,6 +64,43 @@ Set in `.worklog/config.yaml` (local) or `.worklog/config.defaults.yaml` (team d - Git remote used for sync - `syncBranch` (string, default `refs/worklog/data`) - Git ref used for the canonical JSONL file +- `syncAllowForeignAuthor` (boolean, default false) + - Allow merging commits authored by a different identity than the store's + configured `user.email` (see [Author-Identity Gate](#author-identity-gate)). + The CLI flag `wl sync --allow-foreign-author` takes precedence over this + config value. + +### Author-Identity Gate + +Before importing anything, `wl sync` inspects the author emails of the commits +on the remote worklog ref since the last-known sync point +(`git log --format=%h%x09%ae --not `). If any +incoming commit fails the gate, the sync refuses with a non-zero exit code, a +clear error naming the offending commit(s) and the remote ref, and leaves the +local database untouched: + +- **Empty author email** → unconditional refusal. A commit with no author email + (e.g. a store whose git `user.email` is unset) is never merged, even with + `--allow-foreign-author`. +- **Foreign author email** → refused by default when it differs from the repo's + configured `user.email`; allowed when `wl sync --allow-foreign-author` is + passed or `syncAllowForeignAuthor: true` is set in config. +- **`user.email` unset** → the foreign-email comparison is skipped (only the + empty-email gate applies). + +Example refusal: + +``` +✗ Sync failed: Refusing to merge worklog data from refs/worklog/remotes/origin/worklog/data: 1 incoming commit(s) fail the author-identity gate. +- 5fc880a: empty author email +``` + +The gate exists because a second worklog store with an empty/foreign git +identity can silently overwrite work items and invalidate audits (see +[Refused Sync / Polluted Ref Recovery](docs/SYNC_IDENTITY_GATE.md)). The +last-known sync point is persisted in `.worklog/last-synced-ref` after each +successful non-dry-run sync; when the file is absent, the whole remote ref +history is scanned. ### Troubleshooting diff --git a/README.md b/README.md index d0073d0aa..e380bd2aa 100644 --- a/README.md +++ b/README.md @@ -114,6 +114,7 @@ You can get a lot of value from using Worklog as a memory for your agents. But y | [API.md](API.md) | REST API endpoints and usage | | [docs/FILE_PATH_CONVENTION.md](docs/FILE_PATH_CONVENTION.md) | File path convention for work item descriptions | | [docs/CROSS_PROJECT_POLLUTION_CLEANUP.md](docs/CROSS_PROJECT_POLLUTION_CLEANUP.md) | Detecting and removing cross-project worklog pollution (`wl doctor foreign-items`) | +| [docs/SYNC_IDENTITY_GATE.md](docs/SYNC_IDENTITY_GATE.md) | Refusing foreign/empty-author commits on sync (`--allow-foreign-author`, polluted-ref recovery) | | [docs/AGENTS-INSTALL.md](docs/AGENTS-INSTALL.md) | How `wl init` installs AGENTS.md / workflow guidance and how local rules relate to the global agent file | ### Reference diff --git a/docs/SYNC_IDENTITY_GATE.md b/docs/SYNC_IDENTITY_GATE.md new file mode 100644 index 000000000..ef4af8b41 --- /dev/null +++ b/docs/SYNC_IDENTITY_GATE.md @@ -0,0 +1,127 @@ +# Sync Identity Gate & Polluted-Ref Recovery + +Worklog refuses to merge commits from a remote worklog ref +(`refs/worklog/data` by default) when the incoming commits were authored by an +identity other than the store's configured identity. This document explains the +gate, how to verify it manually, and how to recover when a ref was already +polluted by a foreign writer. + +Related: [Data Syncing](DATA_SYNCING.md) · [Cross-Project Pollution Cleanup](docs/CROSS_PROJECT_POLLUTION_CLEANUP.md) + +## Why the gate exists + +A second worklog store with an **empty git identity** (`user.email` unset) can +push divergent state to the shared `refs/worklog/data` branch: cleared fields, +stripped newlines, and demoted statuses. The primary store used to merge it +silently, re-timestamping items and invalidating already-passed audits. The +only reliable signature of that writer is the **empty author email** on its +commits (`git log --format='%ae'` → empty lines). + +The identity gate stops the silent absorption: a foreign/empty-author push is +refused, surfaced to the operator, and never re-exported or pushed back. + +## How the gate behaves + +- Before any merge/import, `wl sync` runs + `git log --format=%h%x09%ae --not ` and + inspects every incoming commit's author email. +- **Empty author email** → unconditional refusal. `--allow-foreign-author` + never bypasses this. +- **Foreign author email** (differs from the repo's `git config user.email`) + → refused by default; allowed with `wl sync --allow-foreign-author` or + `syncAllowForeignAuthor: true` in `.worklog/config.yaml`. +- **`user.email` unset locally** → the foreign-email comparison is skipped; + only the empty-email gate applies. +- The last-known sync point is stored in `.worklog/last-synced-ref` after each + successful non-dry-run sync. An absent file makes the gate scan the whole + remote ref history. + +## Manual verification (parent AC7) + +Run against a ref that contains a known empty-email commit (e.g. `5fc880a` on +the llm worklog ref, or any simulated polluted ref): + +```bash +wl sync --no-push +``` + +Expected: the sync refuses with a non-zero exit code and a clear error naming +the offending commit and the remote ref: + +``` +✗ Sync failed: Refusing to merge worklog data from refs/worklog/remotes/origin/worklog/data: 1 incoming commit(s) fail the author-identity gate. +- 5fc880a: empty author email +``` + +Verify the local database was not touched: + +```bash +wl list --json # remote items are absent +ls .worklog/last-synced-ref # absent — the refusal never persisted a sync point +``` + +Verify the override paths: + +```bash +# Foreign-email commit: refused by default +wl sync --no-push + +# Foreign-email commit: allowed with the flag +wl sync --no-push --allow-foreign-author + +# Empty-email commit: STILL refused even with the flag (gate is unconditional) +wl sync --no-push --allow-foreign-author +``` + +## Config flag + +`syncAllowForeignAuthor` (default `false`) in `.worklog/config.yaml` (or +`.worklog/config.defaults.yaml` for team defaults) allows foreign-email +commits without a CLI flag: + +```yaml +syncAllowForeignAuthor: true +``` + +The CLI flag `wl sync --allow-foreign-author` takes precedence over the config +value. Neither ever bypasses the empty-author-email gate. + +## Recovery for a polluted ref + +A ref that already contains foreign/empty-author commits must be cleaned +before the primary store can sync again. Two approaches: + +### 1. `wl doctor foreign-items` (prefix pollution) + +If the pollution is foreign **item prefixes** (items whose ID prefix does not +match the project), use the existing cleanup tool: + +```bash +wl doctor foreign-items --dry-run # preview +wl doctor foreign-items --apply # remove foreign items locally +``` + +### 2. Rewrite the remote ref (author/identity pollution) + +If the polluted ref's history carries empty/foreign-author commits but the +*snapshot content* is owned by this store, rewrite the ref so it contains only +clean, store-authored commits. This is a force-push style operation that +replaces the remote history: + +```bash +# 1. Export a clean JSONL snapshot from the local DB +# (the DB was never polluted — the gate refused the merge). + +# 2. Publish the clean snapshot as a fresh orphan commit, replacing the ref. +wl doctor foreign-items --apply --push # when the pollution is item-prefix based +``` + +For raw ref rewrites, the underlying helper is `rewriteAndForcePushDataFile` +(used by `wl doctor foreign-items --apply --push`): it creates an orphan +commit containing only the clean JSONL, force-pushes it to +`refs/worklog/data`, and updates the local tracking ref — it never fetches the +polluted remote first. + +After the rewrite, the next `wl sync` sees only clean, own-author commits and +proceeds normally. The empty-email commits are gone from the ref, so the gate +no longer blocks. From eebe41e5a9b5651b96ed466c17e934452f68a4f3 Mon Sep 17 00:00:00 2001 From: pi Date: Sun, 16 Aug 2026 11:06:33 +0100 Subject: [PATCH 33/34] Bump version to v1.0.11 --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index 7b5f8f10c..cfe881863 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "worklog", - "version": "1.0.10", + "version": "1.0.11", "description": "A simple experimental issue tracker for AI agents", "main": "dist/index.js", "type": "module", From 7696a0a0b604ba9d13f5681b210bee656f86e762 Mon Sep 17 00:00:00 2001 From: pi Date: Sun, 16 Aug 2026 11:06:45 +0100 Subject: [PATCH 34/34] Update CHANGELOG.md for v1.0.11 --- CHANGELOG.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 65428bf8d..8e9cc71d0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,31 @@ # Changelog +## v1.0.11 (2026-08-16) +### Features +- Idle tracking now requires the same slots to stay free, not just any slots. (WL-0MSG7P9N8009PCKG) +- Add, edit, or remove notes directly on any markdown document while reviewing it. (WL-0MSKV6SKK008MMXR) +- Idle time now automatically implements small, low-risk planned tasks. (WL-0MSM9HEH8001A55I) +- Agent now auto-resumes after mid-session compaction instead of waiting for you to say "continue". (WL-0MSMGGPWR000MZ0O) +- Downtime audits now run one analysis at a time, so they fit in cheap mode's 2-slot capacity. (WL-0MSORQ1RG005DGUS) +- Closing a pi agent session now frees up its model lease right away. (WL-0MSGI7UIH008USVB) +- Stage-filtered and child lists now sort by priority first, matching your main worklist. (WL-0MSOPHLD1000EWNN) +### Bug Fixes +- Fixed downtime claiming so failed claims no longer log as successful dispatches. (WL-0MSLWJ310000ND0X) +- Downtime dispatches now correctly report failures instead of falsely logging success. (WL-0MSLWJ3I70031Z8U) +- Audits with a "Yes" verdict now properly advance items even when some checks used fallback evidence. (WL-0MSN7XAUS008WOPQ) +- Audits now auto-recover from JSON errors instead of stopping. (WL-0MSNKWLER003EGE0) +- Sync now blocks merges from unknown authors to protect your worklog data. (WL-0MSOYWWS4009HTCB) +- Fixed an issue where items deeper than two levels couldn't be expanded in the work list. (WL-0MSQ3FH1K000MMJW) +- Fixed downtime jobs being dispatched twice by ensuring only one worker claims each job. (WL-0MSRDEWES0059TZN) +- Retrying a create command now returns the existing item instead of making a duplicate. (WL-0MSTNG2QF0049B97) +- Audit tier failures now count as strikes instead of being silently ignored. (WL-0MSLWJ2KP0002SV0) +- Fixed plugin link breaking after worktree cleanup, so prefix+l keeps working. (WL-0MSRG481O007QVEA) +- Fixed the build so the herdr package compiles correctly again. (WL-0MSNSZ4R6009AU0N) +### Other +- Improved game stability and performance through code quality enhancements. (WL-0MS87KJZU006XM5M) +- Fixed duplicate downtime work: audit and plan tasks can now be dispatched twice, wasting idle time on repeat panes. (WL-0MSRBFFLN005W3VT) +- Clarified how agent setup files are installed and how local and global guidance relate. (WL-0MSKEJK4G008BMS0) + ## v1.0.10 (2026-08-12) ### Features - Idle detection now only counts local activity, so remote-only traffic won't block downtime dispatch. (WL-0MSL2ZQIF006QB4Q)