Repository navigation
feat(retrieval): EsCompatRetriever + xerjSidecar — Stage 2b mechanism - #12
Merged
Merged
Conversation
BM25-only Retriever over any ES-compatible endpoint with provenance-carrying ids; optional one-call sidecar runner (spawn/health/resumable reconcile from a consumer-supplied corpus enumerator; degraded results, never throws, never installs silently). Hermetic tests over a fake endpoint. v0.7.0.
… typing, null guards
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Stage 2b of the xerj adoption arc (Lionidas
plans/20261021T032458Z-xerj-stage2-plan.md, part B; evidence: Stage-1 spike on lionidaswt/xerj-stage1).EsCompatRetriever— aRetrieverover any Elasticsearch-compatible HTTP endpoint. BM25-only by design: the Stage-1 A/B measured parity with in-memory BM25 (0.400 vs 0.400 top-3) at ~42–51 ms vs ~18 s per query over a 26k-doc corpus, while hybrid/semantic regressed and cost 10–30 s per fresh query. Hit ids carry provenance (<docSha256>#<ordinal>/note:<id>), so consumers' citation contracts are untouched. Failures throw — degradation is the runner's decision.xerjSidecar()— optional one-call runner: locate/spawn the sidecar (loopback, pinned flags per xerj's security model), health-wait, resumable reconcile (mget existence checks + bulk retries) from a consumer-suppliedcorpusenumerator (butler has no list-all seam). Returns{retriever, degraded, indexedCount, stop}; every failure is a degraded result with the explicit install hint — never a throw, never a silent install, nothing spawns at import.chunkToEsDoc/noteToEsDoc; hermetic tests over a fake ES endpoint (injectable fetch/spawn).Verification
pnpm test: 36 files, 209 passed, 5 skipped (pre-existing pg/env skips) — including 7 new tests covering provenance mapping, throw contract, attach/reconcile/idempotent-skip, and both degraded paths (attach-only miss; spawn-never-healthy kills the child and surfaces the install hint).Notes for reviewer