Repository navigation
chore: upgrade Agentmemory to v0.9.30 while preserving fork fixes - #9
Merged
Merged
Conversation
* fix(cli): anchor engine cwd and rewrite bundled config with absolute paths The bundled iii-config.yaml uses cwd-relative paths and the engine was spawned without a cwd, so on global and npx installs ./data/state_store.db and ./data/stream_store landed in whatever directory the user ran the CLI from, and the iii-exec supervision block (src/**/*.ts watch, node dist/index.mjs exec) never resolved, meaning the engine never supervised a worker and nothing respawned it after the in-process worker died. That surfaced as all data gone reports against a live REST port. startIiiBin now prepares the launch: when the resolved config is the bundled one it writes ~/.agentmemory/iii-config.runtime.yaml (regenerated each boot) with absolute data paths under ~/.agentmemory/data and an absolute node exec line for the installed worker entry, copies any legacy ./data stores from the invocation directory on first run, and spawns the engine with cwd anchored at ~/.agentmemory. Repo checkouts keep the cwd config and repo-root cwd, so dev behavior is unchanged. User overrides via env or ~/.agentmemory/iii-config.yaml are passed through verbatim. agentmemory remove gains a plan item for the generated runtime config. Covered by test/engine-launch.test.ts including a drift guard that rewrites the repo's real iii-config.yaml and asserts no relative paths remain. * fix: make fresh installs portable and persistent * docs: refresh generated config reference
* chore(engine): pin iii-sdk and engine at 0.19.7 Move the iii-sdk dependency and the pinned iii-engine from 0.11.2 to 0.19.7, the last engine before the 0.20 SDK reorganization. It carries the upstream route-ownership fix, so /agentmemory/* no longer answers 404 after the engine reconnects (rohitg00#1013). Engines from 0.19.4 persist each builtin worker's config block into data/configuration on first boot and ignore the config file afterwards, which made --port, --instance and data-dir changes silently ineffective. The CLI now clears those persisted entries before every engine spawn so the runtime config rendered from the environment is the seed each time. Also: agentmemory upgrade pins the SDK to the same constant instead of a hardcoded version, Dependabot ignores iii-sdk, and every pin mention in docs, translations, deploy templates, the eval harness and .env.example is updated. * fix(cli): harden the engine pin follow-ups from review agentmemory upgrade now pins iii-sdk to the version this package was built against instead of whatever AGENTMEMORY_III_VERSION overrides the engine to. A persisted builtin config entry that exists but cannot be removed is surfaced as a warning naming the directory, rather than silently leaving the engine on stale ports; a test covers it. The bundled docker-compose.yml keeps the engine's persisted builtin config on a tmpfs so every container start re-seeds from config.yaml and the III_* env vars, matching the native CLI path. The deploy templates go back to the engine pair of the published npm package; they move together with AGENTMEMORY_VERSION at release time. * fix(index): persist the search snapshot continuously and bound worker shutdown The BM25/vector snapshot was written only by a graceful shutdown, after a full rebuild, or by delete paths. Live observation, compression and memory writes added to the in-memory index without ever scheduling the debounced save, so a crash, SIGKILL or power loss booted the previous snapshot; the rebuild was skipped because that snapshot was not empty, and every observation captured since the last clean stop stayed invisible to search. Every live add now schedules the 5 s debounced save, bulk indexing does the same, and on boot the worker compares the loaded snapshot with each session's observation count and re-indexes what is missing. The SIGTERM handler awaited a state write that cannot land while the SDK is reconnecting, so a worker whose engine had already gone never exited and `agentmemory stop` fell through to SIGKILL. The final flush and the SDK shutdown are now time-boxed at 4 s each with a hard exit at 8 s, open viewer connections are closed instead of awaited, and the CLI's orphan reap grace matches that budget. Review follow-ups: the compose tmpfs now targets the engine image's real working directory (/home/nonroot/data/configuration; the previous path never received the persisted entries), a persisted builtin config entry that cannot be removed aborts `agentmemory start` instead of launching an engine that would ignore the requested ports, and the deploy Dockerfiles refuse to build when III_VERSION and III_SDK_VERSION differ. Verified against the containerized 0.19.7 engine: SIGKILL 1 s after an observation is healed by the boot reconcile, SIGKILL 8 s after is covered by the debounced save, SIGTERM with the engine stopped exits in about 5 s instead of hanging, no persisted config survives a container restart, and the Fly image builds with matching versions and fails with mismatched ones. * fix(index): replace on re-add, widen the boot reconcile, surface SDK shutdown errors SearchIndex.add on an id that is already indexed now removes the old document first. Before, the entry was overwritten but the old postings and the old term count stayed behind, which skewed BM25 scoring every time compression replaced a synthetic observation under the same id, and would have done the same for a reconcile racing a live write. The boot reconcile also treats a session whose observationCount is 0 or missing as unknown and reads it instead of skipping it, and re-checks membership right before it indexes what it collected. settleWithin no longer turns a rejected promise into a successful settlement; the SDK shutdown call site logs the failure and still exits. * fix(cli): clear persisted builtin config under a custom configuration directory The engine's configuration worker accepts an fs adapter directory; a custom engine config that sets it makes the engine persist the builtin entries there instead of data/configuration, so the CLI's cleanup missed them and the rendered ports were ignored on the next start. The cleanup now reads the configuration worker block of the rendered runtime config and clears that directory too, resolved against the engine working directory the way the engine resolves it. * fix: stop the viewer listener before dropping its connections on shutdown closeAllConnections ran before close, so a request accepted in between kept the shutdown waiting on a connection nothing was going to drain. The listener now closes first and the open connections are dropped while it winds down.
* chore(engine): move iii-sdk and the engine pin to 0.22.1 Stage 2 of the engine upgrade, on top of 0.19.7. The 0.20 SDK reorganization renames the client handle (ISdk to IIIClient) and moves the buffered HTTP request type to @iii-dev/helpers/http (ApiRequest to HttpRequest); those are the only source changes the SDK needs, so helpers becomes a pinned dependency next to iii-sdk and `agentmemory upgrade` pins both. Engine 0.22.1 persists each builtin worker's config under config/ in its working directory instead of data/configuration/, so the pre-spawn cleanup now covers the engine cwd, the directory of the runtime config file, and the legacy location, for both the prefixed and the unprefixed entry names. The bundled compose file keeps /app/config on a tmpfs for the same reason. The bundled configs keep the iii- prefixed builtin names: on 0.22.1 an unprefixed http/state/queue/pubsub/cron entry resolves to the standalone registry worker and fails to boot without iii-worker, which is the 0.23 migration, not this one. The in-engine queue stays listed explicitly as iii-queue so durable:subscriber triggers keep working. Verified against engine 0.22.1: standalone worker (boot, observe, search, engine restart with REST back in 1 s, clean SIGTERM), the real CLI on a fresh HOME (engine download, port change across restarts through the persisted-config cleanup), a copy of a real store, the Docker compose path, and the crash-safety and shutdown probes from stage 1. * docs: sweep the remaining engine references for 0.22.1 Full-repo pass after the pin: the contributor guide (AGENTS.md) now names the 0.22 client and request types and where HttpRequest is imported from; the Python examples pin iii-sdk to the same release as the engine; the README and its eleven translations describe the shipped observability sampling ratio (0.1, not 1.0). Checked and left alone: every engine function id and trigger type the worker uses is registered by 0.22.1 (state, stream, mcp, engine::workers, http, durable:subscriber); no workflow, script, plugin or package other than the root package.json references the SDK or the engine version; the "Invalid function id format: publish" line in the engine log is the engine's own pubsub builtin at shutdown and appears on 0.19.7 as well. * feat(cli): agentmemory console through the pinned engine Engine 0.22 ships the iii web console as a subcommand of the engine binary and downloads the console on first use, so the separate install prompt on start (an engine installer piped to sh) and the skipConsoleInstall preference are gone. `agentmemory console` resolves the pinned iii (PATH if it matches the pin, else the private install) and runs `iii console` against the REST, streams and bridge ports agentmemory resolved, serving one port above the viewer by default; --console-port overrides, --port and --instance pick the instance, other flags pass through. The ready panel points at the command instead of a binary path or an install hint. Verified on an isolated HOME: the console serves with the engine ports injected, registers its functions on the agentmemory engine, and warns but still starts when no engine answers. * chore(engine): start the pinned engine quiet and align the deploy configs The CLI spawns the pinned engine with --no-update-check, so a pinned release never looks up newer releases or advisories on GitHub at boot, and with iii's anonymous usage telemetry off unless the user exported III_TELEMETRY_ENABLED themselves; the bundled compose file passes the same flag and variable. The deploy entrypoints generated sampling_ratio 1.0 with console log output on, the combination behind the log feedback loop the bundled config already avoids; they now ship the bundled values. README: the "extend with one command" section listed builtins that already ship in iii-config.yaml and linked a registry page that no longer exists (iii-database is database); it now names the running builtins, the current registry names, where a custom config lives (~/.agentmemory/iii-config.yaml) and how the worker runtime is installed. The translations only get the link fix; their prose is resynced in the usual batch. * docs: pin .env.example to 0.22.1 and place the engine note under the port table .env.example still named 0.19.7 in the engine pin block and lacked III_TELEMETRY_ENABLED next to the pin. The README paragraph about the quiet engine start had landed above the banner; it now follows the port table where the engine flags are described. * docs: describe trace spans as sampled in the observability paragraph The README and the translations kept promising a trace span for every memory operation next to sampling_ratio: 0.1; the sentence now says every operation emits a structured log and one in ten also carries a span. * fix(cli): only remove engine-written config entries, and let console find every pinned iii With a custom engine config the engine's working directory is the config file's directory, so the persisted-config cleanup looked for the seeded builtin names under a project's own config/ folder and would have removed files such as http.yaml or state.yaml that belong to the project. Every entry the engine writes starts with an `id: <worker>` line (0.19.7 and 0.22.1 both), so the cleanup now reads each candidate and removes it only when that line matches the file name. agentmemory console resolved iii from PATH or the private install only; it now walks the same candidate list as engine start, so it finds a compatible engine from the fallback locations as well.
…00#1408) * feat(status): GET /agentmemory/status and a viewer Health tab One report of what is wrong with an install and how to fix it: health monitor state and alerts, LLM and embedding providers, the circuit breaker, functions failing on at least 20% of 5+ calls (with a provider-specific fix for summarize, compress and graph extraction), observations missing from the search index, graph snapshot age, ports, uptime, the pinned engine version and every feature flag. The missing-observation check reuses the boot reconcile scan (findUnindexedObservations, split out of reconcileIndex), so it counts exactly what search cannot find instead of comparing approximate session counters. Every probe is time-boxed at 5 s. Browsers get an HTML page with no scripts behind a strict nonce CSP; other clients get JSON, and ?format=json|html forces either. Same bearer auth as the rest of the API. The flag list moves into buildConfigFlags() so config/flags and status share it, and the engine pin moves into src/version.ts so the CLI and the status report read the same constant. * chore(skills): regenerate the REST reference for the new endpoint npm run skills:check fails CI when plugin/skills/agentmemory-rest-api/ REFERENCE.md does not list every registered REST route; regenerated with npm run skills:gen. * fix(status): report a missing health snapshot and share one index scan A status report with no health snapshot (none yet, or the probe timed out) added no problem and could read ok; it now adds an info-level health-check-unavailable finding. Every status request started its own findUnindexedObservations scan, and the 5 s probe timeout only stopped waiting, so repeated requests stacked full storage scans. singleFlight() shares one in-flight scan between overlapping requests and reuses its result for 30 s; a failed scan is dropped so the next request retries.
… failed refresh, correct dashboard numbers (rohitg00#1407) * fix(viewer): coalesce live reloads, keep data on failed refresh, correct dashboard numbers Every stream event and poll tick reloaded the whole dashboard (about ten requests each), so a burst of observations fanned out hundreds of requests and could exhaust the browser's connection pool. Reloads are now coalesced into one per 2 s and never overlap; live buffers on the Timeline and Activity tabs are capped. A refresh that fails keeps the previous panel data behind a "could not refresh" note instead of dropping to the new-install screen, and a first load that cannot reach the API offers a retry. Dashboard numbers: Memories uses the API total, token savings counts only sessions with observations, and the Graph Nodes card shows the snapshot time. Session summaries stored as objects render as text and comma-separated tags are split instead of breaking a tab. GET /agentmemory/memories now sorts newest first before paging. Rebuild Graph asks for confirmation, ignores repeat clicks and reports its result. The live stream reconnects when the tab becomes visible again. * fix(viewer): drive the dashboard from the live stream instead of re-fetching The dashboard blinked because it re-fetched everything and wiped itself to "Loading dashboard..." on a 30 s timer and on every stream event, even while the stream was connected: 73 rebuilds and 20 requests a minute on an idle store, 200 requests for 21 live events. The worker now pushes session.updated and session.deleted from the sessions state trigger, and memory.updated and memory.deleted from a new memories state trigger, on the mem-live viewer group. The viewer applies those and the existing session.activity events (it used to drop them by reading evt.data instead of evt.event) to the lists it already holds, batches redraws to one per 250 ms, and skips a redraw when the HTML did not change. The 30 s timer is gone and no refresh path resets a loaded view. Polling stays as the fallback when the stream cannot connect. The stream connects with the join mode engine 0.22 serves first and flips to the direct URL only after two failures. * feat(viewer): explain consolidation per layer and extract procedures from sessions Consolidation Status was three bare counters and Procedural Memory stayed empty on almost every install. Procedures only came from a cross-session pass that needs two pattern memories each seen in two or more sessions, and mem::skill-extract, which turns one finished session into a procedure, had no caller. The consolidation pipeline now stores each run's per-tier results under consolidation:lastPipelineRun, and GET /agentmemory/consolidation/status turns them plus the tier inputs (summaries, recurring patterns, counts, flags) into one status per layer: off with the setting to change, waiting with progress, ready, active with the last run's output, or failed with the error. The dashboard replaces the Semantic, Procedural and Consolidation Status cards with one Memory layers panel built from it, previewing the first items of each layer. mem::skill-extract runs for the stopping session inside the existing consolidation cooldown, only when consolidation is on and an LLM provider is configured. * chore(skills): regenerate the REST reference for the new endpoint npm run skills:check fails CI when plugin/skills/agentmemory-rest-api/ REFERENCE.md does not list every registered REST route; regenerated with npm run skills:gen. * fix(viewer): keep live session and memory views consistent Engine 0.22 reports state deletes as "state:deleted" with a null new_value; the session trigger compared against "delete", so deleted sessions were pushed as updates. Both state triggers now share one delete check, and with an isolated agent scope they drop records that belong to another agent, matching the REST filters. In the viewer: - a reconnect re-fetches the views that were already loaded, so changes made while the stream was down show up; - session events are sequenced and replayed onto any session list fetched while they arrived, so a delete is not undone by an older response; - observation count updates reuse the cached session detail instead of blanking and re-fetching it; the detail refetches in place only when that session's record changes or another session is selected; - memory events coalesce into at most one memory list reload in flight, and only the newest memory count response is applied; - the direct/join failure count resets on the first delivered message instead of on open. * fix(viewer): honor stale retries and keep counts and rebuild notes honest - The consolidation status endpoint listed five full state scopes on every request. The store has no count primitive, so overlapping requests now share one scan through singleFlight and reuse its counts for 30 s, the same bound the status page uses for its index scan. - The "Retrying automatically" banner had no retry behind it once the 30 s dashboard timer was removed. A stale dashboard now schedules one retry on its own 15 s timer, separate from the 2 s reload coalescer. - The dashboard's memory total is applied only when no newer count request started while it was loading. - A graph rebuild whose follow-up refresh fails no longer reports success. - The consolidation header shows the date of the last run, not only the time.
…0#1409) The memory alert divided heapUsed by heapTotal. Node sizes heapTotal just above heapUsed and grows it on demand, so a normal process reads 90% or more, and once RSS passed the 512 MB floor the health check raised "Memory elevated" or "critically high" with nothing wrong. The health snapshot now records v8 heap_size_limit and the thresholds divide by it, falling back to heapTotal for snapshots written before this change. The viewer's Heap gauge shows used against the limit, and the alert wording says "of the heap limit". Fixes rohitg00#1406
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 25.9.8 to 26.6.2. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 26.6.2 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 25.9.8 to 26.6.2. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 26.6.2 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(audit): make index_persist audit entries opt-in Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com> * test(audit): isolate the index_persist gate tests from the ambient env AGENTMEMORY_* variables are documented as living in ~/.agentmemory/.env, so the suite can inherit AGENTMEMORY_AUDIT_INDEX_PERSIST from a configured machine. The default-off case then ran with auditing on and failed, and afterEach deleted whatever value the developer had set. Capture and restore around each case, and cover the whole vocabulary the parser accepts — trimmed and case-insensitive 1/true — plus the values an operator would reach for to turn it off. Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com> --------- Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com>
…ohitg00#1415) * perf(index): throttle index saves and report each leg's save state Every observation, compression and memory save scheduled a full index save 5 s later, and each save re-serializes the whole BM25 and vector index and rewrites every shard. On a busy store that is a full index rewrite almost continuously. Saves now run at most once per AGENTMEMORY_INDEX_SAVE_INTERVAL_MS (default 10 minutes), measured from the start of the previous save. Explicit saves (deletes, shutdown) still run immediately. Only one save runs at a time; saves requested while one is running collapse into a single follow-up save. The BM25 and vector saves now fail independently. Before, a BM25 failure skipped the vector save in the same run and both were reported as one warning. Each leg records when it last saved, its last error, since when it has unsaved changes, and its serialized size. /agentmemory/status reports a failing leg as an error, unsaved changes older than twice the interval as a warning, and a vector index above 80% of the Node.js string limit (536,870,888 characters) as a warning, since past that limit the index can no longer be saved or loaded. The status page shows each leg's save state and the vector index size. * chore: credit co-authors The per-leg save failure handling and the single save queue build on the fix in rohitg00#1256. The problem was measured and reported in discussion rohitg00#1358. Co-authored-by: Omar Gerardo <omargrard@gmail.com> Co-authored-by: Vladimir <7998636+MarvinFS@users.noreply.github.com> * fix(index-persistence): snapshot dirtyEpoch per leg before serializing runSave() captured one dirtyEpoch before the BM25 leg's write and reused it for the vector leg's dirty check. When scheduleSave() ran while the BM25 write was in flight, the vector leg would serialize the current (already up to date) state but still compare against the stale epoch, leaving vector.dirtySince set even though the just-written snapshot already reflected the change. status() would report the vector leg as dirty until an unrelated follow-up save cleared it. saveLeg() now snapshots dirtyEpoch itself, immediately before invoking the leg's own serialize+write callback, so each leg judges its dirty state against the epoch as of its own write rather than one captured before an earlier leg ran. Added a regression test that gates the BM25 manifest write, triggers a scheduleSave() while it is held, and asserts the vector leg's dirtySince clears once its own write completes. --------- Co-authored-by: Omar Gerardo <omargrard@gmail.com> Co-authored-by: Vladimir <7998636+MarvinFS@users.noreply.github.com>
…tent (rohitg00#1416) * perf(index): throttle index saves and report each leg's save state Every observation, compression and memory save scheduled a full index save 5 s later, and each save re-serializes the whole BM25 and vector index and rewrites every shard. On a busy store that is a full index rewrite almost continuously. Saves now run at most once per AGENTMEMORY_INDEX_SAVE_INTERVAL_MS (default 10 minutes), measured from the start of the previous save. Explicit saves (deletes, shutdown) still run immediately. Only one save runs at a time; saves requested while one is running collapse into a single follow-up save. The BM25 and vector saves now fail independently. Before, a BM25 failure skipped the vector save in the same run and both were reported as one warning. Each leg records when it last saved, its last error, since when it has unsaved changes, and its serialized size. /agentmemory/status reports a failing leg as an error, unsaved changes older than twice the interval as a warning, and a vector index above 80% of the Node.js string limit (536,870,888 characters) as a warning, since past that limit the index can no longer be saved or loaded. The status page shows each leg's save state and the vector index size. * perf(index): store vectors in fixed buckets and rebuild BM25 from content The vector index was persisted by serializing every vector into one JSON string, cutting that string into shards and writing a new generation of all shards on every save. Node.js cannot hold a string longer than 536,870,888 characters, which a 2,048-dimension index reaches near 49,000 vectors; past it the index can no longer be saved or loaded, silently. Each save also rewrote the whole index. Vectors now live in a fixed set of state scopes (256 by default, AGENTMEMORY_VECTOR_BUCKETS), chosen by a stable hash of the id, with one key per vector holding its id, session and base64 embedding. The index tracks which ids were added or removed since the last save, and a save writes or deletes only those keys, then updates a small metadata record. Loading lists the bucket scopes one at a time. Nothing builds a string of the whole index, and scope names never change between saves. One key per vector, rather than one packed value per bucket: the iii file store rewrites a whole scope when any key in it changes, so both layouts cost the same on disk per flush (one bucket scope, 1/256 of the index). Per-key writes avoid re-encoding a whole bucket in the worker for every changed vector, a single added vector is exactly one state write, and a failed write retries only that vector. BM25 is no longer persisted. It is rebuilt at boot from stored observations and latest memories, the same documents the live paths index. A 3,000-document rebuild takes about 120 ms in the unit test; real boots add the engine round-trips for one list per session, and the boot log now reports the rebuild time. The rebuild also drops entries for content that no longer exists. Documents added after the last vector save that have no vector are embedded in the background; when no vector state was ever saved, every document without a vector is. Existing installs migrate on first boot: the old sharded or monolithic vector snapshot is read once, written into buckets, and only then are the old vector and BM25 shards and manifests removed. If the old snapshot cannot be read it is left untouched. If writing the buckets fails, the old snapshot stays and the next save finishes the job. No vector is lost and nothing is re-embedded. Changing the bucket count moves vectors to their new buckets on the next boot. The dimension-mismatch guard is unchanged; when AGENTMEMORY_DROP_STALE_INDEX discards persisted vectors, the next save now deletes them from storage too. The status page drops the string-size warning, which no longer applies, and shows the bucket count and unsaved vector changes. * chore: credit co-authors The per-leg save failure handling and the single save queue build on the fix in rohitg00#1256. The problem was measured and reported in discussion rohitg00#1358. Co-authored-by: Omar Gerardo <omargrard@gmail.com> Co-authored-by: Vladimir <7998636+MarvinFS@users.noreply.github.com> * chore: credit co-authors The string-limit and whole-generation rewrite analysis comes from discussion rohitg00#1358; the reclaim work in rohitg00#1256 mapped the generation lifecycle this replaces. Co-authored-by: Omar Gerardo <omargrard@gmail.com> Co-authored-by: Vladimir <7998636+MarvinFS@users.noreply.github.com> * fix(index-persistence): snapshot dirtyEpoch per leg before serializing runSave() captured one dirtyEpoch before the BM25 leg's write and reused it for the vector leg's dirty check. When scheduleSave() ran while the BM25 write was in flight, the vector leg would serialize the current (already up to date) state but still compare against the stale epoch, leaving vector.dirtySince set even though the just-written snapshot already reflected the change. status() would report the vector leg as dirty until an unrelated follow-up save cleared it. saveLeg() now snapshots dirtyEpoch itself, immediately before invoking the leg's own serialize+write callback, so each leg judges its dirty state against the epoch as of its own write rather than one captured before an earlier leg ran. Added a regression test that gates the BM25 manifest write, triggers a scheduleSave() while it is held, and asserts the vector leg's dirtySince clears once its own write completes. * fix(index): close vector-index safety gaps from PR review Guard rebuildKeywordIndex's sessions listing with a retried helper instead of a bare kv.list call; a failure now reports an incomplete BM25 rebuild on status instead of throwing. mem::search's cold-start path now calls rebuildKeywordIndex instead of rebuildIndex, so a query against an empty index can never clear or re-embed the persisted vector index; only an explicit dimension mismatch at boot still drops vectors. Gate the vector backfill so a whole-store embed never starts silently: AGENTMEMORY_VECTOR_BACKFILL=all is required when there is no prior save to catch up from, backfills are capped per boot by AGENTMEMORY_VECTOR_BACKFILL_MAX (default 500), the pending count is reported on /agentmemory/status, and backfillVectors now saves progress every few batches so a restart does not repeat finished embedding work. On load, compare the vector count actually read against the count the last save recorded; a shortfall is reported on status and falls back to a bounded backfill for the missing ids instead of trusting the saved timestamp. Meta is still written only after every bucket write of a save succeeds. Replace hash-mod-N vector bucketing with fill-and-roll: every vector's bucket is tracked in memory, built by reading all known buckets at load; new vectors fill the current open bucket up to AGENTMEMORY_VECTOR_BUCKET_SIZE (default 500) before a new bucket opens, and updates/deletes go straight to a vector's existing bucket. A save under load now touches only the buckets it actually changed instead of rewriting the whole store, and a bucket-size change no longer requires moving existing data. Migration from the pre-PR sharded/monolithic format is unchanged; the hash-bucket layout from this PR's earlier, unreleased commit is dropped rather than migrated. Tests: sessions-listing failure keeps vectors untouched and marks the rebuild incomplete; whole-store backfill opt-in and per-boot cap; partial bucket set detected against the saved count; a save spread over several waves of adds touches only the open bucket; deletes touch only the owning bucket; migration from the old sharded format still passes. * fix(index): remove legacy BM25 shards when missing keys read back undefined The engine SDK returns undefined for a missing key. The legacy cleanup treated undefined as a failed read and returned early, so after migration the old BM25 shards and manifest stayed in the store for good. Both legacy cleanups now read through readIndexValue, which only reports failure when the read throws. --------- Co-authored-by: Omar Gerardo <omargrard@gmail.com> Co-authored-by: Vladimir <7998636+MarvinFS@users.noreply.github.com>
) * perf(consolidation-pipeline): skip decay writes for unchanged memories The decay tier of mem::consolidate-pipeline listed every semantic and procedural memory and wrote every row back on every run, even when applyDecay left the strength untouched (the common case for anything not yet due for decay). Each write is a full state::set RPC and marks the whole scope dirty for the engine's next flush, so this scaled with total memory count on every consolidation cycle regardless of how much actually decayed. Capture each item's strength before calling applyDecay and only call kv.set for items whose strength changed by more than a small float tolerance, mirroring the dirty-only pattern already used by mem::lesson-decay-sweep. applyDecay itself is unchanged, so the decay trajectory over time is identical to before; only the redundant no-op writes for untouched rows are removed. results.decay now reports scanned/written counts per tier instead of just the row count. Added tests covering: unchanged rows are not written, changed rows are written with correct scanned/written counts, and decay progression across simulated time lands at the same strengths as the unconditional write path did. * perf(consolidation): write decayed rows one at a time The first decay run after an upgrade can find most rows due at once. Writing them all in parallel would fire that many state::set calls at the engine together, where they queue behind its single store lock. Sequential writes keep the load the same shape as before. * test(consolidation-pipeline): assert persistence at each decay step The simulated-time decay test only checked kv.list()[0].strength, but mockKV.list returns live object references that applyDecay mutates in place. That let the assertions pass even if a kv.set write were skipped, since the in-memory object already reflected the mutation regardless of persistence. Spy on kv.set and assert exactly one semantic write after the 45-day step and a second after the 95-day step, so a missed persistence write would actually fail the test. * fix(consolidation): decay at most once per period applyDecay recomputed daysSince from lastAccessedAt on every pipeline run, and lastAccessedAt never advances, so each run re-applied the 0.9^periods factor to the already-decayed strength. With the default consolidation cooldown this compounded a month-old fact down to the 0.1 floor within hours of active use instead of over months. Track lastDecayedAt on semantic and procedural rows and anchor the next decay check to max(lastAccessedAt, lastDecayedAt), only applying another decay step once decayDays have passed since that anchor. Rows without the field decay once, same as before, and get it set. Updated the simulated-time test to assert once-per-period decay (including a run inside the same period that must not write) instead of pinning the old compounded value.
…payload (rohitg00#1422) api::export always returned status_code 200, even when mem::export detected the response would exceed the transport frame limit and returned its {success:false, oversized:true, ...} refusal object instead of the export data. A caller checking only the HTTP status saw a normal successful export and could write out the tiny refusal object as if it were a real backup. Detect the refusal shape in the api::export trigger and return 413 with the refusal body, matching the pattern api::mesh-export already uses for the same guard. mem::export itself is unchanged: it still returns 200 for a real export and the refusal object for an oversized one, the HTTP layer just now reports the right status. The MCP standalone proxy's memory_export path called handle.call(), which throws on any non-2xx response and let the generic proxy-error handler in handleToolCall silently fall back to local in-memory KV data, masking the refusal behind a small, unrelated response instead of surfacing it. rest-proxy.ts now throws a ProxyCallError carrying the parsed response body, and the memory_export case in standalone.ts returns that body directly instead of letting it fall through to the local-KV fallback. Every other proxied tool keeps its existing fallback-on-error behavior unchanged. The viewer does not call /agentmemory/export, so no viewer change is needed. mcp/server.ts's in-process memory_export handler calls mem::export directly and always wraps the result as MCP text content regardless of shape, so it already surfaces the refusal and needed no change.
…ohitg00#1423) * perf(remember): move the embedding call out of the mem:remember lock mem::remember wrapped its whole body in withKeyedLock("mem:remember", ...), including the vectorIndexAddGuarded call, which makes a network round-trip to the embedding provider. Every save serialized behind that network call, so two unrelated saves that happened to land at the same time queued behind each other's embedding latency even though they touch different memory ids and do not conflict. Split mem::remember so the locked callback covers only the read-dedup-write critical section (candidate generation, Jaccard supersede check, the memory's KV writes, and the BM25 index update) and returns the saved memory. The embedding call, the mem::cascade-update fire-and-forget trigger, and the final log line now run after the lock releases, using the memory the locked section already wrote. Dedup/supersede correctness is unchanged: two concurrent saves for identical content still serialize on the read-dedup-write section (that part is still inside the lock), so the second call's candidate scan still sees the first call's already-written memory and correctly supersedes it. Only the embedding network call, which never reads or writes the supersede state, moved outside. Added test/remember-lock-scope.test.ts: one test with a slow fake embedder proves two concurrent saves' embed calls overlap in wall time instead of serializing, and one test proves two concurrent saves of identical content still dedup correctly (first becomes v1, second supersedes it as v2) even with the slow embedder in place. * fix(remember): keep superseded memories out of the vector index The embedding call for a saved memory runs outside the mem:remember lock so concurrent saves can overlap. If a second save supersedes the first memory while its embedding is still pending, the pending vector add could finish after the supersede's vectorIndexRemove call and put the superseded memory back into the vector index. Move the final vector-index write into the same mem:remember lock the supersede path already uses, guarded by a fresh isLatest check read right before the write. The embedding computation itself stays outside the lock, so concurrent saves still overlap on the slow part; only the cheap check-and-write is serialized against supersession. Added a test that races two identical concurrent saves with a slow embedder and asserts the superseded memory's id never lands in the vector index. * perf(remember): schedule an index save after the vector commit and split failure logging The deferred vector-index add scheduled no index save after it landed, so if the embedding call outlasted the 5s debounce window from the earlier BM25 add, the just-added vector could sit unsaved until some unrelated write happened to reschedule it. Call scheduleIndexSave() right after the commit's vector-index add. vectorIndexAddGuarded ran the deferred commit callback inside the same try/catch as the embedding call, so a failure in the commit (for example the KV read that rechecks isLatest) was logged as "embed failed" with the embedding provider's name, which is misleading when diagnosing a KV problem. The embed call and the commit callback now have their own try/catch with distinct log messages. Added a test for each: one confirms scheduleIndexSave is called again after the vector commit resolves, the other confirms a throwing commit callback is logged as a commit failure, not an embed failure. Also addressed the review's flaky wall-clock assertion in remember-lock-scope.test.ts: dropped the elapsed-time bound and kept the start/end cross-check assertions, which already prove the two embeds overlapped without depending on machine timing.
* perf(patterns): bound mem::patterns to the most recent sessions mem::patterns listed every session in the store and then issued kv.list(KV.observations(id)) once per session with no bound at all when the caller omitted project (the MCP tool's default call shape). On a large store this is the N+1 issue rohitg00#1226 measured directly: a walk of 97,385 observations that killed the invocation after ~3s and took the whole iii worker down with it. Add an optional limit (default 50, max 500, clamped) and sort sessions by startedAt before slicing, so the project filter (when given) narrows the pool first and the recency limit then bounds it to the most recent sessions instead of every session ever recorded. The batched kv.list fan-out (already bounded to 10 concurrent sessions per batch) also now tracks total observations scanned and stops issuing further batches once MAX_OBSERVATIONS_SCANNED (5,000) is crossed, so one huge session inside the session limit can't still blow the invocation timeout. The response now reports sessionLimit, sessionsInScope, sessionsProcessed, and observationsScanned so a caller can see when it was bounded. MCP tool contract stays backward compatible: memory_patterns gets one new optional limit property in tools-registry.ts (existing callers that don't pass it get the same bounded-by-default behavior), the server.ts handler passes args.limit through, and api::patterns's request type picks up the same optional field (it already forwarded the whole POST body, so no behavior change there beyond the new field being honored). Regenerated plugin/skills/agentmemory-mcp-tools/REFERENCE.md via npm run skills:gen to pick up the new tool parameter. Added test/patterns-session-bound.test.ts: default-to-50, explicit limit, clamp to 500, project filter applied before the recency limit, recency direction (the most recent sessions are scanned, not the oldest), and the MAX_OBSERVATIONS_SCANNED early stop within an under-limit session count. * chore(patterns): drop an inline comment * fix(patterns): bound the session limit and per-session observation work A positive fractional limit (e.g. 0.5) passed the finite/positive check in resolveSessionLimit but then floored to zero, silently returning an empty session slice. Clamp the floored value to at least 1. The MCP tool schema for mem::patterns' limit now declares "integer" instead of "number" so clients validate the input shape up front. The observation budget was only checked between batches of 10 session loads, so a single session with far more observations than MAX_OBSERVATIONS_SCANNED still had its full observation list processed before the next check could trip. The per-session loop now checks the running budget before processing each loaded session and truncates a session's observations to the remaining budget, so processing work cannot exceed the configured cap even when one session is unusually large. The parallel batch read itself is unaffected: kv.list/state::list has no bounded-read primitive to limit the fetch size, so the initial load for sessions already in flight in a batch is not shrunk; only the downstream CPU work is capped. * perf(patterns): skip oversized sessions before listing them mem::patterns bounded per-session observation processing with MAX_OBSERVATIONS_SCANNED, but still called kv.list on every session in a batch before applying that bound. A single session that alone exceeds the remaining budget was still pulled whole in one frame, which is exactly the failure mode (a worker drop on an oversized frame) the budget was meant to prevent for rohitg00#1226-sized stores. Use the Session.observationCount already tracked on every observe to skip a session's kv.list entirely when its count exceeds the batch's remaining budget, and report the skipped session ids as sessionsSkipped in the result instead of silently dropping them. Also accept a missing payload on mem::patterns (data ?? {}) so callers that omit project/limit do not throw. * fix(api): whitelist the patterns request body api::patterns forwarded req.body straight into sdk.trigger, against the project rule that REST endpoints must whitelist fields instead of passing the raw request body to sdk.trigger(). mem::patterns only reads project and limit, so the exposure was latent, but a bodyless POST also threw once mem::patterns started reading data.limit before this fix, since req.body was undefined and nothing defaulted it. Build the payload explicitly from req.body?.project and req.body?.limit so unexpected fields never reach mem::trigger and a missing body no longer errors. * perf(patterns): fill each batch only up to the remaining observation budget Each session was checked against the whole remaining budget on its own, so a batch of ten sessions that each fit could still list up to ten times the budget in parallel. A batch now takes sessions in recency order only while their combined observationCount fits the remaining budget; the next batch continues from there, and a session larger than the remaining budget is still skipped without being listed.
…1421) * fix(state): close the KV.sessions concurrent read-modify-write race Every whole-object writer to KV.sessions (api::session::start, event::session::started, api::session::commit, plus the bulk import paths in migrate/snapshot/replay/export-import) read the session, mutated a local copy, and wrote it back with no lock, or with a lock under a different namespace than mem::observe's obs:${sessionId}. A concurrent kv.update from observe (or from any other writer) landing between the read and the write got silently reverted by the later whole-object set. Route every session mutation through the same obs:${sessionId} keyed lock observe.ts already uses, so no two writers can interleave a read against a write for the same session. api::session::commit keeps its get-then-set shape (it needs the current commitShas to dedupe) but now shares the lock namespace instead of its own session:${sessionId} key, which also keeps it compatible with the pending session::end lock in PR rohitg00#1252. session::end and event::session::ended already mutate via kv.update path ops, which the engine applies atomically per call, so they are left as-is. Adds test/session-kv-races.test.ts, which reproduces the lost-update race deterministically (a gated mock KV that pauses a read mid-flight so a concurrent writer can land in between) and fails on main before this change. * fix(graph): serialize persistGraphDelta so concurrent extracts cannot duplicate nodes persistGraphDelta read the graph snapshot once, resolved each node/edge through the name/edge index with a get-then-set (no lock anywhere in graph.ts), and wrote the accumulated snapshot back once at the end. Two concurrent mem::graph-extract calls (routine when session-stops land within seconds of each other) could both miss the same name-index key and each create their own node for the same (type, name) pair, or both merge into the same existing node from a stale copy and drop one side's sourceObservationIds, or overwrite each other's snapshot stat deltas. The node/edge keys aren't known until the name/edge index lookup resolves, and the snapshot is a single shared aggregate mutated in-memory across the whole batch, so a per-key lock alone would still let two batches race the aggregate snapshot write. Serialize the whole function body behind one graph:persist keyed lock instead, matching mesh.ts's existing per-item locking style for the parts that are per-key. graph-import.ts's structural importer calls the same function, so it gets the same protection for free. Adds test/graph-extract-concurrency.test.ts: two concurrent persistGraphDelta calls for the same (type, name) key must not create duplicate nodes, and two concurrent merges into an existing node must not lose either batch's sourceObservationIds. Both fail on main before this change. * fix(graph): serialize graph-row writers outside persistGraphDelta on graph:persist persistGraphDelta already serializes mem::graph-extract under the graph:persist keyed lock, but temporal extraction, mesh sync, export/import, snapshot restore, and cascade-update all read and wrote KV.graphNodes/KV.graphEdges without it. A concurrent call from any of these paths could overlap persistGraphDelta's read-modify-write and drop the loser's changes. Route each of those write paths through the same graph:persist lock (sibling acquisitions, never nested, since withKeyedLock is not re-entrant): temporal-graph-extract's merge loop, mesh's lwwMergeGraphNodes and a new lwwMergeGraphEdges (replacing the generic per-item lock for graph edges), export/import's graph delete and import loops, snapshot-restore's graph node writes, and cascade-update's stale-flag writes (which now also re-read the latest row inside the lock before writing, instead of writing back the pre-lock in-memory copy). Snapshot/index maintenance (name index, edge index, degree, the graph:snapshot cache) is still only performed by persistGraphDelta and the explicit rebuild/reset functions; that gap predates this change and is out of scope here. This commit only closes the lost-update race on the raw node/edge rows. Added test/graph-cross-path-locks.test.ts, which races persistGraphDelta against mem::cascade-update on the same node and asserts both the merge and the stale flag survive. * fix(state): close remaining session and graph lock gaps from review Session end (api::session::end and event::session::ended) wrote endedAt/status with an unlocked kv.update, so the locked commit linker could still read-modify-write the session row around it and revert the end. Both now take the obs:${sessionId} lock, the same key the commit linker and observe use, so the two orderings serialize instead of racing. Session start (api::session::start and event::session::started) built a fresh row and blind-set it under the lock, so if observe's implicit create had already landed, start would reset observationCount to zero and drop firstPrompt/summary/commitShas. Both now read the existing row inside the lock and merge onto it, keeping observationCount, commitShas, and firstPrompt/summary when the incoming request does not explicitly supply them. Graph import/delete (mem::import) took the graph:persist lock once per item inside the 20-wide chunked loop, which serialized the four graph node/edge import and delete loops down to one operation at a time. Each of the four loops now takes graph:persist once for the whole loop, restoring the chunk's intended concurrency. mem::graph-reset and mem::graph-snapshot-rebuild wrote the graph snapshot without the graph:persist lock, so a reset could race a concurrent extract's read-modify-write of the same snapshot and lose resetAt (breaking the post-reset orphan check) or a rebuilt count. Both writes now take graph:persist. The graph-cross-path-locks regression test gated the name-index read inside persistGraphDelta, which lets cascade's write land and then be silently re-merged back to the correct value, so the test passed even without the fix it was meant to guard. Moved the gate to the node-row read, the point the lost update actually depends on; verified it now fails on origin/main and passes on this branch. Added a session-kv-races test for api::session::commit racing api::session::end, and a sequential test for api::session::start running after mem::observe's implicit create, covering the merge behavior above. * fix(graph): hold graph:persist for the whole snapshot rebuild mem::graph-snapshot-rebuild listed nodes and edges and backfilled the name, edge-key and degree indexes outside the lock, and took graph:persist only for the final snapshot write. A delta that committed between the listing and that write was overwritten by a snapshot built from the older lists, and the backfill could reset degree counters the delta had just raised. The listing, index backfill and snapshot write now run in one graph:persist critical section, so an extract either lands before the rebuild lists or waits until it finishes. Nothing inside the section takes the lock again. The new concurrency test pauses the rebuild's node listing, commits a delta meanwhile, and checks that the final snapshot holds both nodes. It fails without this change.
…p index (rohitg00#1418) * perf(context): add per-project recent-sessions index mem::context listed the entire mem:sessions scope on every session start to find one project's 10 most recent sessions, then discarded everything else. The iii file store rewrites and clones a whole scope on every list, so this cost grows with total sessions across every project, forever, on the highest-frequency path in the product. Add a small per-project index (scope mem:idx:project-sessions, one key per project) holding up to 50 recent session ids and startedAt timestamps, capped and sorted newest-first. Maintain it wherever a new session is written: event::session::started, api::session::start, the implicit session create in mem::observe, and mem::import; remove entries wherever a session is deleted: mem::evict, mem::forget, and the bulk replace path in mem::import. mem::context now reads the index and kv.get's only those sessions instead of listing the whole scope. If a project has no index yet, it falls back to the previous full-list behavior once and writes the index from that result so subsequent calls stay bounded. * perf(smart-search): add sharded obsId -> sessionId reverse index findObservation's fallback path (used by memory_smart_search's expandIds when no sessionIdHint is available, which is the common case from the documented MCP call path) listed every session and scanned each one's observation scope in batches of 5 until it found a match. Worst case that is O(sessions) kv.get round-trips per id, times up to 20 ids per call. Add a reverse index from obsId to sessionId, sharded across 64 fixed scopes (mem:idx:obs:<n>, bucket = hash(obsId) mod 64) so no single scope grows with the corpus and the file store's whole-scope rewrite on flush stays cheap. Maintain it wherever an observation is written (mem::observe's per-observation hot path, mem::import) and remove it wherever an observation is deleted (mem::evict, mem::forget, mem::auto-forget, the bulk replace path in mem::import). findObservation now tries the reverse index before falling back to the session scan, and backfills the index when the fallback path finds a match so the next lookup for that id is O(1). * chore: credit co-authors The memories-first observation lookup follows rohitg00#1091, and rohitg00#1230 traced the expandIds path this index serves. Co-authored-by: Erik Bogado <erikbogado@gmail.com> Co-authored-by: cmondragon023 <cmondragon023@outlook.com> * fix(session-index): close index consistency gaps from review Three gaps in the new per-project session index and its context.ts fallback could silently hide sessions from mem::context: - addSessionToProjectIndex built a fresh index containing only the session being added when no index existed yet, discarding every other session already stored for that project. Cold start now seeds the index from the project's stored sessions before merging in the new entry. - context.ts's fallback (full scan + write-back when the index is missing) ran outside any lock, so it could race a concurrent addSessionToProjectIndex call and overwrite its update. Both paths now go through ensureProjectSessionIndex, which rechecks under the same per-project lock before writing. - The 50-entry cap picked the most recent sessions across all agents sharing a project, so an agent with less session volume than others could be pushed out of the index entirely and lose all context. The cap now reserves up to 10 slots per agent before filling the remainder by recency, and removeSessionFromProjectIndex replenishes from stored sessions when a removal drops the index below the cap instead of letting it shrink permanently. ProjectSessionIndexEntry gained an optional agentId used only for the fairness reservation; existing entries without it keep behaving exactly as before (single implicit bucket). Left the evict.ts observation reverse-index entries alone: that path never deletes the observations themselves (mem::summarize creates a summary for the recovered session first), matching the same long-lived reverse-index behavior a normal event::session::stopped leaves in place. Removing the mapping there would break expandIds for observations mem::smart-search's own query path can still legitimately return as hits. * perf(session-index): refill only a full index after a removal An index holding fewer than 50 entries already lists every session in its project, so removing one needs no refill. Refilling only when the index was full keeps deletions in small projects from listing the whole sessions scope each time. * fix(session-index): self-heal untracked writers and drop O(N^2) refill Replay, migrate, and snapshot-restore wrote sessions directly to KV without touching the per-project session index, so those sessions were permanently invisible to mem::context. A failed sessions listing during the index's cold-seed path was silently swallowed and replaced with an empty array, so a transient state::list failure could wipe a project's visible history. Session removal rescanned the full sessions scope every time the index crossed the 50-entry cap, turning bulk eviction or a replace-strategy import into an O(N^2) sweep over the sessions scope. Fix: replay, migrate, and snapshot-restore now call addSessionToProjectIndex after writing each session row. The cold-seed list no longer swallows a failed state::list, so a failure leaves the index key absent instead of persisting an empty one. Session removal now only drops the matching entry and never re-lists. A new rebuildAllProjectSessionIndexes function lists every session once, groups by project, and rewrites each project's index key; it runs once at boot behind a KV.config generation marker and is registered as mem::diagnostic::session-index-rebuild for use after bulk operations. Tests exercise mem::forget, mem::evict, mem::import (replace), mem::replay::import-jsonl, and mem::snapshot-restore through their real registered functions, including an assertion that removing 100 sessions from one project performs at most one sessions listing. * fix(session-index): unindex evicted observations and keep concurrent index changes across a rebuild Evicting a stale session deleted the session record but left its observations in the obsId to session reverse index, so expandIds could still return an observation from the evicted session. The eviction now removes those entries using the observations it already loaded. The boot rebuild runs in the background and lists sessions once before writing each project index. A session started or removed between that listing and the write was overwritten by the older snapshot, and the generation marker then stopped any later repair. Adds and removals made while a rebuild is running are now recorded and applied to the rebuilt index under the project lock. * fix(session-index): drop the project index when an add cannot be written A failed index write left the existing index in place without the new session, and since the index was present no later read or boot rebuilt it, so the session never reached mem::context. The add now deletes that project index on a failed write, so the next read rebuilds it from the session scan, and records the change first so a running rebuild keeps it. --------- Co-authored-by: Erik Bogado <erikbogado@gmail.com> Co-authored-by: cmondragon023 <cmondragon023@outlook.com>
…klog (rohitg00#1420) * perf(stream): stop writing per-session mem-live groups, bound the viewer group The engine's file-based stream store rewrites a whole (stream, group) scope on every dirty flush tick, the same whole-scope-rewrite cost as state. observe.ts and compress.ts wrote every raw and compressed observation into a per-session mem-live group (group id = sessionId) in addition to the shared viewer group, but grepping src/, plugin/, integrations/, packages/ and the viewer for stream::get, stream::list, stream::delete or any WS subscribe targeting a session-id group turned up zero reads anywhere: the viewer only ever joins stream/mem-live/viewer. That write was pure waste, doubling the persisted cost of every observation for nothing, so this drops it and removes the now-dead STREAM.group helper. The remaining shared "viewer" group is written on every observation via one persisted stream::set call (the synthetic/non-auto-compress path in observe.ts; the stream::send calls elsewhere only publish to live subscribers and never touch storage, confirmed against the engine's stream::send handler) and was never pruned, so a long-running daemon accumulates one entry per observation forever and every viewer join/reconnect replays the whole backlog. src/state/viewer-stream.ts tracks written item ids in memory and, every 50 writes, deletes the oldest items beyond AGENTMEMORY_VIEWER_STREAM_MAX (default 500) via stream::delete, batched rather than per-write. Delete failures are logged and swallowed (Promise.allSettled) so pruning never blocks observation ingestion. Did not implement a one-time boot cleanup of pre-existing per-session groups: the engine has stream::list_groups to enumerate groups cheaply, but stream::list/get_group returns only stored values, never item ids, and stream::delete requires an item id per call with no group-level or bulk delete anywhere in the stream or state workers. Reconstructing ids from nested value fields across every historical session group would be fragile and, at the reported scale (600+ sessions, up to 500 items each), not cheap - the opposite of what this fix is for. Per the task's own conditional, reporting instead of building it. * perf(stream): trim the stored viewer backlog after restarts The prune tracker only knew about items written by the running process, so every item stored before a restart stayed in the viewer group for good, and that is the backlog existing stores already have. At boot the tracker is now seeded from one listing of the viewer group, oldest first, and anything beyond the cap is deleted in batches of 100 in the background. Pruning no longer holds up the observation that triggered it. * chore: credit co-authors The persisted per-session stream groups were measured and reported in discussion rohitg00#1358. Co-authored-by: Vladimir <7998636+MarvinFS@users.noreply.github.com> * fix(viewer-stream): retry failed prunes and reject malformed stream cap Keep a failed stream::delete id tracked instead of dropping it, so the next prune retries it rather than letting the backlog grow past the limit forever. Recheck for overflow once an in-progress prune finishes if writes arrived while it was busy, instead of losing that check when the write counter resets. Seed and requeue large id lists in bounded chunks instead of spreading them into function arguments, which could throw once a stored backlog got large enough. Reject a non-empty AGENTMEMORY_VIEWER_STREAM_MAX value with trailing junk (e.g. "500000junk") and fall back to the documented default instead of silently taking the numeric prefix. * perf(viewer-stream): delete overflow newest-first, seed in listing order The engine's stream store removes items with IndexMap::shift_remove, whose cost is proportional to how many entries sit after the removed one. Pruning took the oldest tracked ids first and deleted them front to back, so each delete shifted almost the whole remaining store; a large boot backlog made every stream::set queue behind a prune that ran quadratic in the store size. Deleting the same oldest-ids selection but issuing the deletes newest-of-the- overflow-first means each removal only shifts the retained tail, bounded by the configured cap instead of the store size. The boot seed sorted the listed backlog by the hook's client-supplied timestamp before choosing what counts as oldest. An unparseable or skewed timestamp could make the newest item look oldest and get pruned first. The engine already returns the group in insertion order, so the seed now trusts that order directly and never parses timestamps. AGENTMEMORY_VIEWER_STREAM_MAX accepted any integer, including negative values that emptied the tracker (and deleted every stored item) on the next prune, and values under 200 that silently shrank the backlog below what the viewer's join sync displays. The getter now rejects negative values (falls back to the default) and floors any other value at 200. The boot stream::list call had no timeout, so an oversized backlog — or a list response that exceeds the SDK socket's payload limit and closes the connection without rejecting the pending call — could hang the seed indefinitely during startup. It now carries a 5s timeoutMs; a failure of any kind is already treated as "skip seeding this boot". trackViewerStreamItem and the seed now share one id set, so a write that races the boot seed for the same id is tracked once instead of twice. The PR description's claim that the old per-session groups "would need ids the stored values do not carry reliably" is wrong: both writers set item_id = obsId with observation.id === obsId, so the ids are reliable. Leaving those groups alone is a scope decision, not an ids problem. --------- Co-authored-by: Vladimir <7998636+MarvinFS@users.noreply.github.com>
* perf(stream): bound the viewer stream backlog (base from rohitg00#1420) The stream group changes from rohitg00#1420, which the live viewer builds on. This commit disappears from the diff once rohitg00#1420 merges. * feat(api): stream every viewer scope and add viewer endpoints State triggers and viewer events for memories, lessons, actions, crystals, semantic facts, procedures, audit, graph deltas, consolidation runs and health; one snapshot endpoint and a counts endpoint; cursor pagination, filters and facets on the list endpoints; graph node, observation locate and memories-by-session lookups; and the backend fixes the viewer audit found (LLM detection, profile files, status wording, mem::evolve, lesson sourceIds, audit filters). * feat(viewer): rebuild the viewer on one streamed store Every page renders from one store fed by the snapshot and stream events, with no polling. Grouped navigation, glossary help, self-explaining empty states, and rebuilt Memories, Sessions (with the timeline merged in), Graph, Health with a consolidation explainer, Audit, Activity, Profile, Replay, Lessons, Actions and Crystals.
* perf(audit): shard mem:audit into month scopes so writes stop paying for the whole log The iii file store rewrites a whole scope on every flush when any key in it changed, and the audit log lived in one scope (mem:audit) forever, so a mature store paid O(log-size) disk I/O on every audited operation. One store had 583k rows, 534k of them index_persist noise written before that kind of row became opt-in. recordAudit now writes into a UTC month scope (mem:audit:YYYY-MM) derived from the entry timestamp, with a small index key (mem:audit:months) tracking which month scopes exist so readers never have to list_groups. queryAudit, and everything that goes through it (the memory_audit MCP tool, GET /agentmemory/audit, the viewer audit tab, mem::heal, diagnostics), reads month scopes newest-first and stops once it has the requested limit, falling back to the legacy mem:audit scope only if it still needs more. A background migration runs at boot, bounded to 200 rows per tick, moving legacy rows into their month scopes and dropping legacy index_persist rows outright (system noise, not deletion records); it records one summary audit row for the purge and leaves every other row alone. It resumes on the next boot if interrupted, since queryAudit's legacy fallback keeps unmigrated rows readable in the meantime. AGENTMEMORY_AUDIT_RETENTION_MONTHS (default 0, keep forever) adds a daily sweep that drops whole month scopes older than N months; off by default so no install loses audit history without asking for it. * perf(audit): migrate the legacy audit log in two fast passes Moving one row at a time kept the legacy scope large for the whole migration, and the engine rewrites a changed scope in full on every 5 s flush, so a store with hundreds of thousands of rows would have rewritten that file hundreds of times. The migration now copies the rows it keeps into their month scopes, records each month in the index once, and then deletes the legacy keys, both passes with bounded concurrency and no pauses, so the legacy scope empties within a few flushes. A migration that stops early leaves the legacy rows readable and resumes on the next boot. * chore: credit co-authors The audit growth (583,035 rows, 534k index_persist) was measured and reported in discussion rohitg00#1358. Co-authored-by: Vladimir <7998636+MarvinFS@users.noreply.github.com> * fix(audit): close the lost-update race on the month index markAuditMonth and runAuditRetentionSweep each did a read, modify, write round trip against the single mem:audit:months index key with no coordination between callers. startAuditMigration runs unawaited at boot alongside request-time recordAudit calls, and the retention sweep reads the index once at the start of a run that can take a while to delete rows. In every case, a second writer could read the stale index before the first writer's write landed, and the later write would silently drop the other writer's month from the index, making its rows unreachable by queryAudit and the retention sweep. Serialize index mutations through the existing keyed mutex (already used elsewhere in this file's neighborhood for the same class of read-modify-write race) and have the sweep re-read the index inside the lock right before its final write, instead of writing back a snapshot captured before the sweep started. Also run the audit retention sweep once at startup, after audit migration finishes, instead of only on a 24h interval. Deployments that restart daily before the first interval fires never get a sweep. * perf(audit): gate the legacy audit migration on a local size probe The migration used to unconditionally state::list the legacy mem:audit scope on every boot, which can return a payload over the SDK socket's 100 MiB limit and hang boot for the invocation timeout, and it deleted legacy keys in insertion order, which is quadratic against the engine's IndexMap-backed store. queryAudit had the same unconditional list on its legacy fallback path, with no de-duplication against rows already copied into month scopes. Replace the unconditional list with a local file-system probe of the engine's own state store directory (mem%3Aaudit.bin) so a store's size is known before ever asking the engine for it. Under AGENTMEMORY_AUDIT_MIGRATE_MAX_BYTES (default 32 MiB) and locally readable, the migration copies rows into their month scopes, waits at least two engine save intervals so the copies are durable, then deletes the legacy keys in reverse insertion order with lower concurrency. Anything larger, or not locally readable (for example an engine on another host), is left frozen and reported through GET /agentmemory/status instead of migrated. Progress is tracked in a migration marker (mem:audit:months/migration) so a crash mid-pass resumes without re-deriving counts from a shrinking legacy scope, and the completion summary is written exactly once behind that marker. queryAudit only reads the legacy scope when the marker says it is safe, de-duplicates by id, and reports a legacyFrozen flag plus size when older rows are known to remain frozen. The retention sweep also deletes in reverse insertion order. Fixed a re-entrant deadlock this introduced: the migration's summary-write step called safeAudit under the same keyed lock used by markAuditMonth, which queued behind itself forever. It now uses its own lock key. AuditMonthIndex and the new migration/query types now live in src/types.ts per the KV-scope convention. standalone MCP's memory_audit handler now goes through queryAudit like every other reader instead of listing the legacy scope directly. * fix(audit): stop calling a copied legacy log frozen and export the deploy data dir A migration in the copied state has already moved every legacy row into the month scopes; only the delete pass is pending. queryAudit and the status page treated any state other than done as frozen, so a failed delete pass reported rows as missing from audit queries until the next boot. Only too-large and unreadable now count as frozen. The deploy entrypoints kept /data in a shell-only DATA_DIR variable while the engine config writes state under /data. With AGENTMEMORY_DATA_DIR unset, the CLI resolved a platform default directory instead, so the legacy audit probe looked in the wrong place and left the migration frozen. The entrypoints now export the directory they use. --------- Co-authored-by: Vladimir <7998636+MarvinFS@users.noreply.github.com>
* feat(state): add opt-in redis backend for iii-state and iii-stream
The bundled file-based kv adapter rewrites a whole scope on every flush
and holds the entire store in the engine's heap, which does not scale
for a shared daemon with several concurrent writers. iii-engine 0.22.1
already ships a redis adapter for both iii-state and iii-stream that
writes per key (HSET) instead of rewriting a whole scope, so expose it
as an opt-in instead of building a new storage layer.
AGENTMEMORY_STATE_BACKEND=redis (plus AGENTMEMORY_REDIS_URL) switches
both iii-state and iii-stream to that adapter when agentmemory renders
its runtime iii-config. The file-based store stays the default; leaving
the variable unset renders byte-identical config to before this change.
Selecting redis without a URL fails fast with a clear error before the
engine is launched, both from renderEngineConfig directly and from the
CLI's startEngine() check.
rewriteBundledConfig (src/cli/engine-launch.ts) is the path a real npm
install actually takes; it previously rebuilt its own options object
for renderEngineConfig and silently dropped anything beyond dataDir and
ports, so a live redis: false-negative was live-verified via docker
(iii-state/iii-stream kept rendering to the file adapter under the
bundled config until this was fixed) before landing.
README documents the switch plus the export/import commands for moving
data between backends, since switching starts from an empty store.
* fix(state): fail loudly on unswitchable redis config, harden migration docs and test isolation
renderEngineConfig silently left a worker on its default file/kv adapter
when a custom iii-config.yaml was missing the iii-state/iii-stream worker
block or its adapter block, even though AGENTMEMORY_STATE_BACKEND=redis
had been requested. It now throws so the mismatch surfaces instead of the
engine starting on a different backend than the user selected.
The README migration commands for export/import omitted the bearer token
required when AGENTMEMORY_SECRET is set, used curl -s (which can write an
HTTP error body to backup.json without failing), and never checked that
backup.json was a usable export before telling the user to switch backends
and restart. All three are fixed.
test/state-backend-config.test.ts imported src/config.ts at module scope,
so DATA_DIR/ENV_FILE were computed from the real HOME before beforeEach
sandboxed it, letting a developer's real ~/.agentmemory/.env leak into the
test. It now sandboxes HOME before a fresh dynamic import of the module,
matching the pattern already used in test/env-loader.test.ts.
* fix(state): make a failed redis launch fatal and stop leaking the URL to disk
A failed redis config rewrite in prepareEngineLaunch was caught and
silently discarded, so the CLI would start the engine on the file (or
in-memory) store while the user believed it was on Redis. The catch
now runs resolveLaunchRenderFailure and exits loudly when the state
backend is redis, while the file backend keeps its existing silent
fallback to the bundled config.
The rendered runtime config no longer embeds the literal Redis URL.
iii 0.22.1 expands ${VAR} references in config values at boot (see
engine/src/workers/config.rs expand_env_vars, called from
config_file), so the adapter block now renders
redis_url: '${AGENTMEMORY_REDIS_URL}' and the CLI already forwards
process.env (hydrated from ~/.agentmemory/.env) to the spawned engine
process, so the reference resolves the same way the literal used to.
The engine's own configuration worker still persists the expanded
value under ~/.agentmemory/config/*.yaml (fs.rs writes the resolved
struct, not the source text), so the README documents that residual
exposure and recommends chmod 700 on shared hosts. Every engine
stderr surface the CLI prints (captured startup stderr, docker logs)
now runs through redactCredentialUrls first.
getStateBackend() now throws on an unrecognized AGENTMEMORY_STATE_BACKEND
value instead of silently defaulting to file; startEngine validates it
before touching the engine and exits with the same clear error style
already used for a missing AGENTMEMORY_REDIS_URL. Both Docker paths
(resuming a persisted docker-owned engine, and the AGENTMEMORY_USE_DOCKER
compose path) now warn when redis was requested, since neither renders
iii-config.docker.yaml. /agentmemory/status reports the active
stateBackend so an operator can confirm which store is live without
reading the URL.
README: replaced the AUTH=(...) migration snippet, which produces a
different (and wrong) curl argv under zsh than under bash, with an
if/else that is not shell-dependent; documented plain redis:// only
(the pinned engine has no TLS), one Redis DB per --instance, the
appendonly/noeviction recommendation, and corrected the claim that
only a native start renders the two variables (the one-click deploy
templates overwrite the bundled config and start natively too, so
they render it as well; only this repo's own Docker Compose path
does not). deploy/README.md documents the same two variables.
* fix(redis): reject unloadable URLs and close the file-store fallback
The engine expands ${AGENTMEMORY_REDIS_URL} into the YAML text before it
parses the config, so a single quote or a control character in the URL
breaks the single-quoted scalar and the engine does not start. The render
now rejects those characters with a message to percent-encode them.
A failure to create the engine working directory returned the original
file-backed config before the Redis render ran, so a Redis selection
could silently start on the file store. That path now uses the same
fatal handling as a failed Redis render.
Docker Compose forwards AGENTMEMORY_REDIS_URL to the engine container,
so a hand-edited iii-config.docker.yaml can reference it instead of
holding a literal URL. The README and deploy notes say plain redis://
is unencrypted and must stay on a trusted network or behind an
encrypted tunnel.
* docs(redis): drop em-dashes from the state backend section
* fix(redis): make every state and stream path behave the same on redis
The engine's redis adapter differs from the file store in ways that
broke or degraded agentmemory beyond basic writes:
- state::list is HGETALL collected into a HashMap, so records came back
in arbitrary order (export paging, viewer stream seeding, anything
that slices a list). StateKV now sorts redis listings oldest first by
the creation time in generated ids, then timestamps, then id.
- state::update runs a Lua script whose cjson round trip turns empty
arrays into empty objects (session tags, commitShas). On redis StateKV
applies set/remove/merge itself under a per-key lock and writes with
state::set, which fires the same created/updated triggers.
- The legacy audit migration probed the file store's scope file on disk,
so a fresh redis store was marked unreadable and the audit log showed
as frozen. On redis it sizes the legacy scope from the hash itself and
skips the file flush wait.
- After a redis restart the engine's pub/sub pump ends and live viewer
events stop until restart. The health monitor now sends a probe event
through the stream every 30 seconds on redis and raises
stream_relay_down with a restart fix.
- /status probes the state store and reports state-store-unreachable
with a backend specific fix; the Health page shows the backend in the
State store row and the relay state.
* docs(redis): document redis behaviour, one server per instance and the restart caveat
Redis pub/sub ignores the database index, so separate DB numbers keep
data apart but still cross-feed live viewer events between instances.
List what works unchanged, what agentmemory handles for the engine's
redis differences, and what to do after Redis restarts.
…00#1448) * fix: preserve bounded source evidence and checkpoint fresh vectors * fix: enforce live source budgets and bound replay writes
rohitg00#1449) * fix(governance): count and audit only records that were really deleted mem::forget counted absent sessions, summaries and observation ids as deleted and wrote those fabricated counts into the audit trail. It now checks each record before deleting it, counts only real deletions, and reports absent ids in notFound and failed deletes in failed/failures. mem::governance-delete and mem::governance-bulk follow the same rule: audit rows name only the ids that were deleted, and no audit row is written when nothing was deleted or failed. governance-bulk dry run and live call now accept the same filters. The project filter is applied on both paths, an empty filter is refused on both paths, and unsupported keys such as ids or concepts are refused on both paths instead of silently widening a dry run to the whole store. Existing response fields are unchanged; notFound, failed and failures are new. Fixes rohitg00#1427 Fixes rohitg00#1428 * fix(export): refuse an oversized export before assembling it mem::export listed every collection in parallel, built the whole export object and only then serialized it to check the transport frame limit. On a graph-heavy store that allocated the full payload plus its JSON string just to refuse it. Assembly is now sequential with a running byte budget. Each collection is measured item by item as it is read, and the export returns the same oversized refusal (HTTP 413 through /agentmemory/export) as soon as the budget is exceeded, without listing the remaining collections or session observation scopes. The refusal names the collection where it stopped. The final frame check stays as a backstop, and ?maxSessions and ?offset paging is unchanged. Refs rohitg00#1334 * fix(cli): bound the iii-engine auto-install download The installer piped curl into tar with no timeout, so a slow or stalled GitHub download left the spinner running for minutes and a killed run reported "unknown error" and left a truncated ~/.agentmemory/bin/iii. curl now runs with a 15 s connect timeout and a 300 s overall limit, and the spawn itself is bounded. The archive is downloaded to a temp file in the bin dir and only extracted once complete, so a failed download leaves no partial binary. A failure prints the cause (including curl exit 28 as a timeout) and a copyable manual install command. * fix(cli): keep the live port in the ready hint commands With --port 4321 or --instance N the ready panel still suggested `agentmemory demo` and `agentmemory console`, which target 3111. Both suggestions now carry --port or --instance for the running daemon. * fix(config): print the zero-LLM notice once per process The 30 s health tick builds a status report that called loadConfig() twice, and every call printed the zero-LLM notice, so a keyless daemon wrote it twice every 30 s forever. The status builder now reads the config once, and the provider notice (zero-LLM or agent-sdk fallback) is printed at most once per process. The notice also claimed BM25-only search even when an embedding provider was active; it now says search uses BM25 plus any configured embedding provider. * fix(status): skip the missing-observation check during the keyword rebuild The status routes are registered before the boot BM25 rebuild runs, so the first health tick scanned for unindexed observations while the index was still being filled and cached that partial result for 30 s. /status showed a false "N stored observations are not in the search index" warn for up to 30 s after every boot. The boot marks the keyword rebuild as pending before the routes are registered. While a rebuild is pending or running, the status report skips the scan and shows an info note instead of a warning. Each completed rebuild bumps an epoch, and a cached scan from an older epoch is discarded, so a scan that overlapped a rebuild is never reused. * fix(cli): let the engine flush state before stop terminates it agentmemory stop sent SIGTERM to the engine as soon as the worker exited. The worker's shutdown save writes the vector and BM25 snapshots into the engine, but the engine's file-backed state store only writes dirty scopes to disk on its save interval (5 s by default) and does not flush on shutdown. Stopping the engine inside that window lost the last index save, so vectors written just before a stop were re-embedded on the next boot. stop now gives the worker its full shutdown budget (the worker's own hard-exit time plus a second) instead of SIGKILLing it at 5 s, then waits one engine save interval plus a 1.5 s margin before it stops the engine. The interval is read from the rendered engine config, capped at 15 s, and the wait is skipped for the redis state backend. * fix: address review on forget cleanup, bulk project filter, status and install - mem::forget runs the search and vector index removal in a finally around image-ref cleanup, for memories and observations, and flushes the index save whenever index cleanup ran, so a failed image decrement no longer leaves a deleted record searchable. - mem::governance-bulk rejects a project filter that is empty or not a string, in dry run and live alike. Before, it was ignored and the other filters could delete across every project. - evaluateStatus holds the missing-observations warning while the keyword index rebuild is running, even when a count is passed in. - The manual install command downloads to a mktemp file inside the bin directory instead of a shared /tmp path, removes it after extraction or failure, and keeps the failure status.
…rohitg00#1385) * fix(graph): floor the snapshot-reported total at the nodes it returns paginateFromSnapshot reports stats.totalNodes (or stats.nodesByType[type]) verbatim, with nothing constraining it from below by the candidates it is paging over. When a lossy write leaves the counter understated, the response claims fewer nodes than it ships, and `truncated` - which is gated on that same value - stays false, so the "showing N of M" banner that would surface the discrepancy is off exactly when the counter is least trustworthy. Observed with the empty-body query branch returning 294 nodes alongside totalNodes: 101: `101 > 294` is false, so nothing was reported. Floor the reported total at filteredNodes.length. This is deliberately a floor only, so behaviour above SNAPSHOT_TOP_NODES is unchanged - a healthy 40k counter still reports 40000 with truncated true, and the floor engages only when the counter is smaller than the data actually present. Fixes rohitg00#1382 Signed-off-by: cbsincool <cbsincool@163.com> * chore: drop explanatory comments around the total floor --------- Signed-off-by: cbsincool <cbsincool@163.com> Co-authored-by: Rohit Ghumare <ghumare64@gmail.com>
…0#1446) Hybrid search runs two graph traversals per query: searchByEntities and expandFromChunks. Each lists every node and edge. At AGENTMEMORY_GRAPH_WEIGHT=0 their results were multiplied by zero, but the reads still ran on every search. So the setting people apply to stop the graph's cost saved nothing. At weight 0 both traversals are now skipped. Search results do not change, because the graph term was already zero. Signed-off-by: David Waltermire <davewaltermire@gmail.com>
…d read (rohitg00#1384) * fix(graph): abort the snapshot delta instead of zeroing it on a failed read persistGraphDelta read the snapshot through readSnapshot(), which returns null both when the key is genuinely absent and when the read failed. The caller coalesced that null into emptySnapshot() and wrote the result back, so a single transient state::get failure replaced the snapshot's counters with zeroes. The live graph survived in KV.graphNodes/KV.graphEdges, but /graph/query and /graph/stats serve from the snapshot, so every reported total read near-zero until a full rebuild. A snapshot present under an unrecognised schema version was read the same way - null - so a future schema bump would also have been overwritten by the next extraction. Give the writer path a strict reader: retry a failed read once and let a second failure propagate, and return null only for a genuinely absent key (first run, where emptySnapshot() is correct). A present snapshot whose version is not 1 aborts rather than reading as an empty store. Readers keep the lenient readSnapshot, which falls back safely because it only ever displays. A delta that cannot read the current snapshot is now skipped rather than applied against a zero base; mem::graph-extract reports it as a failed extraction. Fixes rohitg00#1381 Signed-off-by: cbsincool <cbsincool@163.com> * fix(forget): record a deletion before its cleanup runs mem::forget counted a memory or observation as deleted, and added it to the audit targets, only after image-ref and access-log cleanup finished. When that cleanup threw, the record was already gone from the store but was reported as a failed delete and left out of the audit. The deletion is now recorded as soon as kv.delete succeeds, index removal follows, and cleanup errors are reported separately as cleanupFailed / cleanupFailures in the result and the audit entry. --------- Signed-off-by: cbsincool <cbsincool@163.com> Co-authored-by: Rohit Ghumare <ghumare64@gmail.com>
…#1411) (rohitg00#1453) * fix: sweep stale active sessions to abandoned and schedule eviction pass Sessions started by reused runtimes (gateways, cron workers) whose host process never delivers session/end stay active forever: 90% of sessions on our shared daemon were stuck active. The Session type already reserves the abandoned status for this case, and mem::evict already implements the recovery/deletion policy for stale sessions, but nothing ever writes abandoned and nothing invokes mem::evict on a schedule. Add mem::session-sweep (hourly, SESSION_SWEEP_ENABLED gate): mark active sessions with no update for SESSION_SWEEP_STALE_HOURS (default 24) as abandoned with endedAt, then trigger the existing eviction pass. Fall back to startedAt when the session has no heartbeat, because session/start does not initialize updatedAt. Also declare the updatedAt field on Session — mem::observe has been writing it via path-based updates without the field being declared. Fixes rohitg00#1410 Signed-off-by: cristianbdev <132560371+cristianbdev@users.noreply.github.com> * fix: address review on session sweep - Drop the mem::evict trigger: the eviction pass also deletes low-importance observations and memories, so scheduling it hourly by default would turn on data deletion for every install. Marking sessions abandoned stays the sweep's only job; scheduled eviction should be a separate, opt-in change. - Read settings via config.ts getters backed by getMergedEnv() (AGENTMEMORY_SESSION_SWEEP_ENABLED / AGENTMEMORY_SESSION_SWEEP_STALE_HOURS), document both in .env.example, regenerate the skills reference. - Validate the stale-hours value as a plain positive integer instead of parseInt prefix parsing, so 1e2 cannot select 1 hour. - Surface scan failures: a failed kv.list returns success:false with the error instead of an empty-list success. - Reactivate abandoned sessions on new activity: mem::observe flips status back to active and clears endedAt when an abandoned session receives an observation, with tests. - Boot log prints the parsed stale-hours value. - Drop inline rationale comments from code. Fixes rohitg00#1410 Signed-off-by: cristianbdev <132560371+cristianbdev@users.noreply.github.com> * fix(sessions): lock, re-check and audit the stale-session sweep mem::session-sweep now marks each candidate abandoned inside the obs:${sessionId} lock that observe, session start/end and the commit linker share, and re-reads the row there, so activity that lands after the scan keeps the session active instead of being overwritten. A failed per-session update is counted and reported (success false, failed N) rather than hidden behind a successful result. Each run that abandons sessions writes one session_sweep audit entry into the current audit month scope, and the viewer labels it. The sweep never deletes anything and does not touch the project session index, whose entries carry no status. It does not depend on list order, so it is correct on the Redis backend. Reactivating an abandoned session in mem::observe now removes endedAt instead of storing null. The hourly timer reports a failed run through bootLog. Drops the remaining inline comment on Session.updatedAt and the issue number in a test name. * docs(env): drop the issue reference from the session sweep setting --------- Signed-off-by: cristianbdev <132560371+cristianbdev@users.noreply.github.com> Co-authored-by: cristianbdev <132560371+cristianbdev@users.noreply.github.com>
…e (carries rohitg00#994) (rohitg00#1452) * fix(hooks): skip capturing memory_* MCP tools and add capture config env vars Closes rohitg00#993. post-tool-use and post-tool-failure now filter agentmemory's own MCP tools (memory_*, ToolSearch, etc.) so recall/search calls are not written back into the observation store. Adds AGENTMEMORY_CAPTURE_DENY/ALLOW, AGENTMEMORY_CAPTURE_OUTPUT_MAX, and AGENTMEMORY_PRE_COMPACT_BUDGET. * test(hooks): wait on observe signal instead of fixed sleep Address CodeRabbit review on rohitg00#994 — integration test now resolves when the mock server receives /observe (or confirms none arrive after hook exit). * fix(hooks): keep capture truncation within OUTPUT_MAX Reserve space for the truncation marker so AGENTMEMORY_CAPTURE_OUTPUT_MAX is a hard length cap, not max-plus-marker. * chore(hooks): regenerate skills reference for capture env vars Run npm run skills:gen so the agentmemory-config reference lists AGENTMEMORY_CAPTURE_ALLOW, AGENTMEMORY_CAPTURE_DENY, AGENTMEMORY_CAPTURE_OUTPUT_MAX and AGENTMEMORY_PRE_COMPACT_BUDGET, drop the doc comment in the capture filter, and rebuild the hook scripts. * docs(env): list the hook capture settings in .env.example * fix(hooks): match exact capture patterns against the full tool name An exact pattern in AGENTMEMORY_CAPTURE_ALLOW or AGENTMEMORY_CAPTURE_DENY was only compared with the bare tool name, so a full MCP name such as mcp__github__create_issue never matched, while wildcard patterns already checked both forms. Exact patterns now match either the bare or the full name. --------- Co-authored-by: Joyjit Nath <96009+joyjit@users.noreply.github.com>
…itg00#1445) (rohitg00#1454) * fix(graph): bound and attribute node and edge provenance Each merge in persistGraphDelta added every observation id in the batch to every node and edge it touched, with no limit. Hot nodes grew without bound. On one store, 98% of the graph's bytes were these ids: 426 per node and 362 per edge on average. Nodes took 323 MB and edges 398 MB. Every kv.list traversal of the graph carried them. - A merge now adds the incoming record's own sourceObservationIds, not the whole batch. Heuristic extraction knows the exact source, so that attribution is now kept. - Each record keeps its newest 32 ids (MAX_GRAPH_SOURCE_OBSERVATIONS). A re-seen id moves to the newest end. New records are trimmed on first write. Retrieval returns at most 20 results, so 32 covers it. - The temporal-graph node merge uses the same rule. - New mem::graph-compact and POST /agentmemory/graph/compact trim records written before the cap. They walk the name and edge-key indexes and read and write one key at a time, so they never list the nodes or edges scope. An optional scope, offset and limit runs one slice per call, so each call finishes between worker reconnects. Re-running a slice is safe. - Compaction takes the graph:persist lock per record, so a trim cannot overwrite a merge that lands during it. persistGraphDelta no longer takes the batch id list. Measured on a live store: compaction trimmed 18,324 nodes and 32,654 edges and removed 18.9 million ids in 45 s with no retries. Nodes went from 323 to 30 MB and edges from 398 to 42 MB. Fixes rohitg00#1374 Signed-off-by: David Waltermire <davewaltermire@gmail.com> * fix(api): validate graph/compact input and report its real failures Review found two defects in POST /agentmemory/graph/compact. Invalid input returned HTTP 200. mem::graph-compact reports a bad scope, offset or limit as { success: false }, so the route passed that through with 200. The route now rejects an unknown scope, a negative or non-integer offset, and a non-positive or non-integer limit with 400, before invoking compaction. It accepts JSON numbers only, so "5" is rejected. Other success:false results still return 200, because that envelope also carries operational failures. Every failure was reported as "graph disabled". The catch returned graphDisabledResponse(), a 503 with GRAPH_EXTRACTION_ENABLED advice, but compaction is registered whether or not graph extraction is enabled. A timeout, such as an unsliced run on a large graph, now returns 504 and says to run in slices. Any other failure returns 500. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * fix(api): return 500 when graph compaction reports a failure mem::graph-compact catches a storage error and returns { success: false, error } instead of throwing. sdk.trigger therefore resolves, and POST /agentmemory/graph/compact answered 200. The route validates its input first, so a success:false result reaching this point is an operational failure. It now returns 500 with the generic "Graph compaction failed" message. The detailed error stays in the server-side log, which mem::graph-compact already writes. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * docs: add doc comments to the provenance and compaction helpers The explanations above boundSources and compactGraphProvenance are now doc comments, word for word. mergeNode, mergeEdge and the two compaction inner helpers get one-line doc comments, and so do the test helpers. This answers the docstring-coverage pre-merge check. No behavior changes. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * docs: add doc comments to the graph function registrars Each comment names the functions its registrar exposes. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * fix(graph): bound temporal edge provenance before it is written The temporal parser gives every edge the whole batch's observation ids, and the edge write stored them without the cap that nodes already had. A batch of more than 32 observations wrote an edge over the cap. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * chore(graph): drop code comments added with provenance compaction The repo keeps code free of comments; the behaviour is covered by tests and the endpoint is documented in the README and REST reference. * fix(graph): walk compaction slices in a stable order and audit the run Sliced compaction paged over kv.list(index) by offset across calls. The Redis backend returns list entries in a different order on each call, so a full walk could end with nextOffset null while records were never visited. Sort the ids before slicing; ids carry a base36 timestamp, so records inserted during a walk sort to the end. mem::graph-compact now records a graph_compact audit entry with the counts whenever it removes ids, and its registration sits above the graph-reset notes it was splitting. * fix(graph): bound provenance on import, mesh sync and snapshot restore mem::import, mem::mesh-receive / mesh pull and mem::snapshot-restore wrote graph nodes and edges as received. They do not write the name or edge-key index, so compaction cannot reach those rows, and a pre-fix export or peer brought the bloat straight back. Each path now keeps the newest 32 source observation ids, the same bound extraction uses. * fix(graph): accept loosely typed records in boundRecordSources snapshot-restore passes parsed JSON records typed as { id } & Record<string, unknown>, which tsc rejected against the narrower generic bound. * docs: explain POST /agentmemory/graph/compact Add the endpoint to the key endpoints table and describe when to run it, the scope/offset/limit slicing loop, and the final scope "snapshot" call a sliced run needs. --------- Signed-off-by: David Waltermire <davewaltermire@gmail.com> Co-authored-by: David Waltermire <davewaltermire@gmail.com>
…1455) * fix(graph): bound and attribute node and edge provenance Each merge in persistGraphDelta added every observation id in the batch to every node and edge it touched, with no limit. Hot nodes grew without bound. On one store, 98% of the graph's bytes were these ids: 426 per node and 362 per edge on average. Nodes took 323 MB and edges 398 MB. Every kv.list traversal of the graph carried them. - A merge now adds the incoming record's own sourceObservationIds, not the whole batch. Heuristic extraction knows the exact source, so that attribution is now kept. - Each record keeps its newest 32 ids (MAX_GRAPH_SOURCE_OBSERVATIONS). A re-seen id moves to the newest end. New records are trimmed on first write. Retrieval returns at most 20 results, so 32 covers it. - The temporal-graph node merge uses the same rule. - New mem::graph-compact and POST /agentmemory/graph/compact trim records written before the cap. They walk the name and edge-key indexes and read and write one key at a time, so they never list the nodes or edges scope. An optional scope, offset and limit runs one slice per call, so each call finishes between worker reconnects. Re-running a slice is safe. - Compaction takes the graph:persist lock per record, so a trim cannot overwrite a merge that lands during it. persistGraphDelta no longer takes the batch id list. Measured on a live store: compaction trimmed 18,324 nodes and 32,654 edges and removed 18.9 million ids in 45 s with no retries. Nodes went from 323 to 30 MB and edges from 398 to 42 MB. Fixes rohitg00#1374 Signed-off-by: David Waltermire <davewaltermire@gmail.com> * fix(api): validate graph/compact input and report its real failures Review found two defects in POST /agentmemory/graph/compact. Invalid input returned HTTP 200. mem::graph-compact reports a bad scope, offset or limit as { success: false }, so the route passed that through with 200. The route now rejects an unknown scope, a negative or non-integer offset, and a non-positive or non-integer limit with 400, before invoking compaction. It accepts JSON numbers only, so "5" is rejected. Other success:false results still return 200, because that envelope also carries operational failures. Every failure was reported as "graph disabled". The catch returned graphDisabledResponse(), a 503 with GRAPH_EXTRACTION_ENABLED advice, but compaction is registered whether or not graph extraction is enabled. A timeout, such as an unsliced run on a large graph, now returns 504 and says to run in slices. Any other failure returns 500. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * fix(api): return 500 when graph compaction reports a failure mem::graph-compact catches a storage error and returns { success: false, error } instead of throwing. sdk.trigger therefore resolves, and POST /agentmemory/graph/compact answered 200. The route validates its input first, so a success:false result reaching this point is an operational failure. It now returns 500 with the generic "Graph compaction failed" message. The detailed error stays in the server-side log, which mem::graph-compact already writes. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * docs: add doc comments to the provenance and compaction helpers The explanations above boundSources and compactGraphProvenance are now doc comments, word for word. mergeNode, mergeEdge and the two compaction inner helpers get one-line doc comments, and so do the test helpers. This answers the docstring-coverage pre-merge check. No behavior changes. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * docs: add doc comments to the graph function registrars Each comment names the functions its registrar exposes. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * fix(graph): bound temporal edge provenance before it is written The temporal parser gives every edge the whole batch's observation ids, and the edge write stored them without the cap that nodes already had. A batch of more than 32 observations wrote an edge over the cap. Signed-off-by: David Waltermire <davewaltermire@gmail.com> * chore(graph): drop code comments added with provenance compaction The repo keeps code free of comments; the behaviour is covered by tests and the endpoint is documented in the README and REST reference. * fix(graph): walk compaction slices in a stable order and audit the run Sliced compaction paged over kv.list(index) by offset across calls. The Redis backend returns list entries in a different order on each call, so a full walk could end with nextOffset null while records were never visited. Sort the ids before slicing; ids carry a base36 timestamp, so records inserted during a walk sort to the end. mem::graph-compact now records a graph_compact audit entry with the counts whenever it removes ids, and its registration sits above the graph-reset notes it was splitting. * fix(graph): bound provenance on import, mesh sync and snapshot restore mem::import, mem::mesh-receive / mesh pull and mem::snapshot-restore wrote graph nodes and edges as received. They do not write the name or edge-key index, so compaction cannot reach those rows, and a pre-fix export or peer brought the bloat straight back. Each path now keeps the newest 32 source observation ids, the same bound extraction uses. * fix(graph): accept loosely typed records in boundRecordSources snapshot-restore passes parsed JSON records typed as { id } & Record<string, unknown>, which tsc rejected against the narrower generic bound. * docs: explain POST /agentmemory/graph/compact Add the endpoint to the key endpoints table and describe when to run it, the scope/offset/limit slicing loop, and the final scope "snapshot" call a sliced run needs. * feat(graph): compact bloated graph provenance once at boot Stores hit by the unbounded sourceObservationIds bug (rohitg00#1124) crash-loop or time out on graph search, and most of them will never call POST /agentmemory/graph/compact. After the worker is ready and the viewer is up, a background pass now walks nodes, edges and the temporal edge history in sorted slices of 100 with a 200 ms pause, then trims the cached snapshot. It reuses compactGraphProvenance, so each record takes graph:persist once and nothing nests the lock. Progress and the completion marker live in mem:config under graph-compact-on-boot:v1 with the cap, so a restart resumes from the saved offset, a finished store never runs it again, and a future cap change runs it once more. A failing slice is retried three times, then reported in the boot log, /agentmemory/status and the viewer Health page with the manual command; the worker keeps running. AGENTMEMORY_GRAPH_COMPACT_ON_BOOT=false turns it off. The manual endpoint gains a history scope for superseded temporal edges. * fix(graph): keep boot compaction totals exact across crashes Each slice is now counted with a dry run first. When the slice has records to trim, the plan (scanned, trimmed, ids removed, next offset) is saved in the progress record before any record is written, then the slice is applied and the plan is folded into the totals in the same write that advances the offset. A restart that lost the last progress write finds the saved plan at its offset and reuses those counts, so a re-processed slice never adds to the totals twice and never drops the records it already trimmed. The snapshot step follows the same rule. compactGraphProvenance gains a dryRun option that counts without writing. Status, viewer and boot-log fields are unchanged. The new test captures the store before every progress write of a full run, restarts from each of those states, and checks that the final scanned, trimmed and ids-removed totals equal a clean single run. * docs(env): document the boot-time graph compaction setting --------- Signed-off-by: David Waltermire <davewaltermire@gmail.com> Co-authored-by: David Waltermire <davewaltermire@gmail.com>
…mit for the engine (rohitg00#1457) * fix(cli): faster state saves, bounded glibc arenas and a higher fd limit for the engine Lower the iii file-store save interval from the engine default 5000 ms to 2000 ms in every config agentmemory ships or renders, with an AGENTMEMORY_STATE_SAVE_INTERVAL_MS override. The stop wait and the audit migration delete wait now both read the interval from the persisted iii-state entry first, since the engine strips the runtime config block. Set MALLOC_ARENA_MAX=2 for the engine child on Linux and in the Docker and deploy entrypoints, keeping any value the user set. Raise the open-file limit for the Docker engine container and in the deploy entrypoints. The native CLI path already hands the engine a soft limit equal to the hard limit because Node raises its own limit at startup; a test pins that behavior. * docs(env): document the state save interval override * fix(cli): wait for the longest engine save interval on stop The stop wait took the first save_interval_ms it found and capped the result at 15 s. With state and stream stores on different intervals, or a file store that relies on the engine default, stop could stop the engine before the slower store wrote its last changes, and a long configured interval was cut short by the cap. The wait now uses the longest interval across every file-backed store, counting a store without an explicit interval at the engine default of 5000 ms, and the cap is gone. Redis still waits for nothing. * chore: regenerate the config reference after merging main
…t, confine file paths and scrub all writes - REST API and viewer: writes (POST/PUT/PATCH/DELETE) must carry Content-Type application/json when they have a body, and a present Origin header must be a loopback origin for the configured ports, the viewer's own origin, or listed in VIEWER_ALLOWED_ORIGINS. Clients that send no Origin keep working. The Host check is unchanged. - Auth by default: without AGENTMEMORY_SECRET the server generates a secret into ~/.agentmemory/secret (0600). The CLI, viewer, hooks, MCP server and shim, connect configs and bundled integrations read it for loopback URLs. An explicit AGENTMEMORY_SECRET still wins; mesh sync still requires an explicit one. - File paths: compress-file, replay import, graphify import, obsidian export and migrate resolve symlinks and stay inside the data dir or AGENTMEMORY_IMPORT_ROOT. - Scrubbing: write functions run the secret scrubber on their input, and compression, summary, crystal, graph, replay, team share, session title and commit message text is scrubbed before storage. New patterns for PEM private key blocks and credentials embedded in URLs. - Context: stored text is escaped before it goes into the agentmemory-context block and pinned slots; viewer esc() escapes quotes. Supersedes rohitg00#878 and rohitg00#922. Co-authored-by: Dave Cole <dave@site.work> Co-authored-by: siddamsetty srinath <74676797+Srinath279@users.noreply.github.com>
AGENTMEMORY_SECRET is now generated into ~/.agentmemory/secret on first start when unset. VIEWER_ALLOWED_ORIGINS adds origins allowed to write through the viewer, and AGENTMEMORY_IMPORT_ROOT widens the folder that file imports and exports may read from.
…ohitg00#1461) * feat(cli): verify the iii engine download against pinned SHA-256 hashes The automatic engine installer now downloads the release archive, checks its SHA-256 against a hash pinned in the CLI for each platform of iii v0.22.1, and only then extracts it. A mismatch deletes the download and prints the manual install command, which also verifies the hash before extracting. A version or platform without a pinned hash is not auto-installed. The hashes match the .sha256 assets of the iii-hq/iii v0.22.1 release and the archives themselves. * ci: pin actions to commit SHAs and set read-only permissions actions/checkout and actions/setup-node now reference the commit behind the v6 tag they already used, with the version kept next to the pin so Dependabot can update it. The CI workflow declares contents: read. * chore(deploy): pin base images by digest The deploy Dockerfiles pin node:22-slim and the iiidev/iii image by digest. The iii image digest is a build argument next to III_VERSION, and the build stops with a clear message when the binary in the pinned image is not III_VERSION, so the two are overridden together. The busybox init container in docker-compose.yml is pinned the same way. * fix(cli): pass engine installer paths to the shell as arguments The download and extract scripts are now fixed strings, and the release URL, install directory, archive and binary paths reach them as positional arguments, so a home directory containing shell syntax is never evaluated. * chore(ci,deploy): use version tags for actions and base images Digest-pinned base images stop picking up the security patches published under node:22-slim and busybox:1.36 until someone bumps the digest, and the CI workflow runs with read-only permissions and no secrets, so SHA pins on its actions add upkeep without protecting anything that matters. Actions and images go back to version tags. The CI workflow keeps contents: read, and the deploy Dockerfiles keep the check that the iii binary copied from iiidev/iii matches III_VERSION. * chore(scripts): add engine:hashes to refresh the pinned engine checksums npm run engine:hashes [version] downloads every iii release archive the CLI installs, checks each against the .sha256 file published with the release, and rewrites the III_RELEASE_SHA256 table in src/cli/engine-install.ts. Without an argument it uses III_PINNED_VERSION. An engine bump becomes: change III_PINNED_VERSION, run the script, commit.
…ooks
The filesystem watcher, openclaw, opencode, pi and hermes clients now use
AGENTMEMORY_SECRET from the environment first, then ~/.agentmemory/.env,
then the generated secret file. Unexpanded ${...} placeholders count as
unset, and the stored files are only read for loopback URLs (all of
127.0.0.0/8, localhost and ::1). The worker resolves its secret through
ensureServerSecret only, so placeholders follow the same rules there.
Secret store tests also isolate USERPROFILE for Windows.
…d backup writes - mem::compress scrubs the vision model description before storing it. - mem::crystallize scrubs caller supplied metadata such as project. - URL credential redaction also covers an empty username (redis://:password@host). - Obsidian export and compress-file confine every directory and output file to their root and open files without following symlinks, so a pre-existing symlink cannot redirect the memory, MOC or backup writes.
writeConfinedFile opened the destination with O_TRUNC, so a failed write (disk full, interrupted process) left an empty file where the original used to be. It now writes the full content to a temporary file in the same directory, created exclusively without following symbolic links, and renames it over the destination only after the write succeeds. A failed write removes the temporary file and leaves the original intact. A destination that is a symbolic link is still refused.
fix: validate request origin and content type, require auth by default, confine file paths and scrub all writes
… durable backlog A force-kill lost every vector added since the last bucket snapshot (saved at most every 10 minutes), and boot recovery re-embedded at most AGENTMEMORY_VECTOR_BACKFILL_MAX of them. The next snapshot then moved savedAt forward, so the rest were never revisited. - Every vector added to or removed from the live index is also written to a small pending log scope (mem:index:vec-pending), keyed by document id, holding the float32 vector as base64 with session, kind and a monotonic sequence. Removals write a tombstone; a full clear writes one marker entry. - A successful snapshot deletes only the entries it covered (sequence captured at snapshot start). Entries written during the snapshot, or any entry when the snapshot fails, stay. Deletes and writes for the same id run in order. The log triggers an early snapshot once it holds 500 entries, so the scope stays small. - Boot loads the snapshot as before, then replays the log in sequence order (list order is not trusted) with no embedding calls, skipping entries whose dimension does not match the active provider. - Whatever is still missing after replay is re-embedded in batches of AGENTMEMORY_VECTOR_BACKFILL_MAX with a pause between batches until done. A durable backfill marker keeps the recovery window open until the backlog is empty, so a later snapshot cannot skip unrecovered documents and a killed backfill resumes on the next start. The whole-store opt-in (AGENTMEMORY_VECTOR_BACKFILL=all) is unchanged. - /agentmemory/status and the viewer show the pending log size, a pending log write error, and a paused backfill state with plain language fixes. No new settings. Keyless installs write nothing. Existing bucket and legacy snapshots load unchanged.
Explain that vectors written between index saves are replayed from a pending log at start without embedding calls, and that the remaining backlog is re-embedded in batches of AGENTMEMORY_VECTOR_BACKFILL_MAX until done, also across restarts.
The repeated tool call counter belongs to the capture change, so the vector durability status test no longer passes or asserts it.
fix(index): keep vectors across a force-kill with a pending log and a durable backlog
…offline spool (rohitg00#1460) * fix(capture): durable observation acceptance, restart-safe dedup and offline spool Hooks derive a stable event id (host tool_use_id when present, otherwise a hash of session, hook type, tool name, input, output and host timestamp) and send it with every observation. The server writes each event to a capture inbox in the state store before storing it, keeps completed event ids in sharded scopes for AGENTMEMORY_CAPTURE_DEDUP_HOURS, and answers a replay with the existing observation instead of a second one. Failed writes are retried with backoff, also after a restart, and end as dead letters on /status, the viewer Health page and GET /agentmemory/capture. When the server is down, times out or returns a 5xx, hooks append the observation to a bounded private spool under the data dir. It drains at boot and from a detached child of the next hook that reaches the server. New: agentmemory capture [--drain], GET /agentmemory/capture, POST /agentmemory/capture/retry, POST /agentmemory/capture/drain. Fixes rohitg00#1436 Fixes rohitg00#1437 Fixes rohitg00#1438 * fix(index): keep vectors across a force-kill with a pending log and a durable backlog A force-kill lost every vector added since the last bucket snapshot (saved at most every 10 minutes), and boot recovery re-embedded at most AGENTMEMORY_VECTOR_BACKFILL_MAX of them. The next snapshot then moved savedAt forward, so the rest were never revisited. - Every vector added to or removed from the live index is also written to a small pending log scope (mem:index:vec-pending), keyed by document id, holding the float32 vector as base64 with session, kind and a monotonic sequence. Removals write a tombstone; a full clear writes one marker entry. - A successful snapshot deletes only the entries it covered (sequence captured at snapshot start). Entries written during the snapshot, or any entry when the snapshot fails, stay. Deletes and writes for the same id run in order. The log triggers an early snapshot once it holds 500 entries, so the scope stays small. - Boot loads the snapshot as before, then replays the log in sequence order (list order is not trusted) with no embedding calls, skipping entries whose dimension does not match the active provider. - Whatever is still missing after replay is re-embedded in batches of AGENTMEMORY_VECTOR_BACKFILL_MAX with a pause between batches until done. A durable backfill marker keeps the recovery window open until the backlog is empty, so a later snapshot cannot skip unrecovered documents and a killed backfill resumes on the next start. The whole-store opt-in (AGENTMEMORY_VECTOR_BACKFILL=all) is unchanged. - /agentmemory/status and the viewer show the pending log size, a pending log write error, and a paused backfill state with plain language fixes. No new settings. Keyless installs write nothing. Existing bucket and legacy snapshots load unchanged. * fix(observe): keep a repeated tool call when its output changed The time-window dedup for events without an eventId keyed only on session, tool name and tool input, so the same command with a new result was dropped for five minutes. The key now also includes a hash of the tool output (tool_response, tool_output, output or error). The same input with the same output inside the window is still skipped. Status and the viewer report how many captures were skipped this way since start. * docs(env): document the capture settings and the vector backfill batch size Adds the capture inbox, dedup, dead letter and offline spool settings, and describes AGENTMEMORY_VECTOR_BACKFILL_MAX as the batch size for re-embedding vectors missing since the last save, which now continues across restarts. * fix(capture): keep accepted observations until the server saved them An accepted capture lives in the engine's memory until the file store writes it on its save timer. A force-kill inside that window lost observations the hook had already discarded on the 2xx answer. The observe response now carries the server's bootId, acceptedAt and durableAfterMs (engine save interval plus margin for the file store, a short fixed window for redis). Hooks keep each accepted event in the local spool as a sent record named by boot id and due time. Later hook calls release same-boot records once due, without a request, and hand records from an earlier boot to the drain, which re-sends them with the same event id so survivors deduplicate. The server reclaims and re-sends records from earlier boots at start and on its sweep timer, keeps what it drains under its own boot id, and releases its own due records. Older hooks ignore the extra fields. New hooks against a server without a boot id discard on 2xx as before. AGENTMEMORY_CAPTURE_SPOOL= false turns all of it off. Kept records count toward the spool byte limit and give way to unsent records. * perf(capture): skip fsync for kept sent records and document the window A kept record only has to outlive a kill of the service, which the page cache already covers; the fsync cost about 5 ms per hook on macOS. Offline spool appends keep their fsync. The README now explains the boot id, acceptedAt and durableAfterMs fields and how hooks use them. * fix(capture): store a crash-split event exactly once The file state store saves each scope on its own timer, so a force-kill can persist the capture event record without the session observations, or the observations without the event record. - Observation ids now come from the event key and the hook timestamp, so a re-sent event whose event record was lost maps to the same observation, vector and BM25 document instead of a second copy. - When an event record says stored but its observation is missing, the re-sent event is stored again under the recorded id instead of being answered as a duplicate. Crash bench (N=1000, kill at 3 s) before: 976 observations, 1028 vector documents; 25 event records pointed to missing observations, 1 observation was stored twice and 27 vectors were orphaned by re-issued ids. * fix(capture): restore a lost vector when a re-sent event is already stored A force-kill can persist an observation while its vector and the vector pending log are lost. The re-sent event then hits the stored observation (or the stored event record) and returns early, which left the observation without a vector until a full backfill was approved. When a re-sent event finds its observation already stored, add the observation back to the BM25 and vector indexes if either is missing. This embeds only that one observation and does nothing when both entries are present. * fix(capture): scrub secrets before an event is written to the inbox Every write path now redacts credentials before text is stored. The capture inbox keeps the hook payload until mem::observe stores it, and a retrying or dead-lettered event can stay there for days, so the payload data is scrubbed before the inbox write. mem::observe scrubs the same way, so stored observations do not change. * Revert "fix(index): keep vectors across a force-kill with a pending log and a durable backlog" This reverts commit 41f0f83. * fix(status): keep the repeated tool call counter with the capture change The "Repeats skipped" row on /agentmemory/status and the viewer Health page reports the observe dedup counter, which belongs to the capture change. It was added together with the vector durability work, which now lives in its own pull request, so the revert removed it. Add it back with its own status test. * build(hooks): emit the shared capture code once as plugin/scripts/_capture.mjs Each hook is built as its own entry, so the capture and spool code was inlined into all eight capture hook bundles. src/hooks/_capture.ts is now its own hook entry, built to _capture.mjs in plugin/scripts and dist/hooks with a fixed name, and the capture hooks import it through a relative ./_capture.mjs import. Hooks that do not capture are unchanged and stay self-contained. Every consumer runs the hooks from the plugin folder (Claude Code, Cursor, Codex, Copilot, Devin, Droid and Antigravity hooks files, and the Antigravity bridge spawns sibling scripts), and the npm package ships plugin/, so the shared module resolves wherever hooks run. No installer copies single hook files. plugin/scripts goes from 9,280 to 3,133 lines. Cold start and spool output are unchanged. * test(status): move the repeat counter test to its own file Keeps test/status.test.ts as on main apart from the capture line, so this change and the vector durability change merge in either order without a conflict. * fix(capture): keep a deleted observation deleted when its event is replayed Since a re-sent event stores its observation again when the event record exists but the observation does not, an observation deleted on purpose came back when a crash re-send or a kept sent record replayed the same event inside the event id window. - The observation now carries the key of its capture event (captureKey), set by the server for client events and copied through synthetic and LLM compression. A captureKey sent by a client is ignored. - Every path that removes observations (forget by id, session forget and the delete endpoints that use it, auto-forget, eviction by age and by project cap, and import with replace) first marks the event record as deleted (state "deleted", deletedAt), then removes the observation. If the event record was lost in a crash, the marker is created from the key on the observation. If the marker cannot be written, the delete fails instead of leaving a record that a replay could bring back. - A replay of a deleted event is answered as a duplicate with the old observation id and stores and indexes nothing. A stranded inbox entry for a deleted event is dropped by the retry sweep. - Pruning counts a deleted marker's age from the delete, so it lasts the full dedup window after it. Audit records and scrubbing on the delete paths are unchanged.
…henticated recovery commands (rohitg00#1468) - Hooks whose host sends no event id or time (prompt submit, for one) derived the same event id for identical content, so typing the same prompt twice stored it once. The hook timestamp is now part of the id in that case; retries still reuse the stored id, and transcript replays at session end keep a stable id. - When the spool lock cannot be taken in time, an append no longer prunes or evicts, a drain no longer claims the main spool file, and leftovers go to a separate draining file instead of rewriting the main file, so concurrent spool writers cannot overwrite each other. - Printed curl commands (dead letter list and retry, spool drain, graph compact, the status page example) send the bearer token, since auth is on by default. - /status reports when the capture check did not answer instead of implying capture is fine, and the spool drop warning sums drops across every spool and names the latest reason.
…act (rohitg00#1463) * fix(capture): durable observation acceptance, restart-safe dedup and offline spool Hooks derive a stable event id (host tool_use_id when present, otherwise a hash of session, hook type, tool name, input, output and host timestamp) and send it with every observation. The server writes each event to a capture inbox in the state store before storing it, keeps completed event ids in sharded scopes for AGENTMEMORY_CAPTURE_DEDUP_HOURS, and answers a replay with the existing observation instead of a second one. Failed writes are retried with backoff, also after a restart, and end as dead letters on /status, the viewer Health page and GET /agentmemory/capture. When the server is down, times out or returns a 5xx, hooks append the observation to a bounded private spool under the data dir. It drains at boot and from a detached child of the next hook that reaches the server. New: agentmemory capture [--drain], GET /agentmemory/capture, POST /agentmemory/capture/retry, POST /agentmemory/capture/drain. Fixes rohitg00#1436 Fixes rohitg00#1437 Fixes rohitg00#1438 * test: release gate that runs capture and recovery on the packed artifact npm run release:gate builds and packs @agentmemory/agentmemory and @agentmemory/mcp, installs both tarballs into a fresh prefix with a clean HOME and a PATH without the repo, and starts the installed CLI on random free ports against an in-process deterministic embedding server. It runs named scenarios against the installed entrypoints: package and engine identity, capture through the bundled Claude Code hooks, offline capture recovered from the spool on restart, replay dedup after a force kill, dead letters that survive a restart and recover on retry, vectors saved by the first checkpoint surviving a crash, agentmemory stop then start, export and import into a fresh home, the MCP shim and agentmemory mcp, and the viewer plus /agentmemory/status. It prints a pass or fail line per scenario and a JSON summary, archives logs, resolved dependencies and summary.json, and stops only the processes it started. The Release gate workflow runs it on Ubuntu and macOS for pull requests and pushes to main. CONTRIBUTING.md and AGENTS.md ask for a green gate before every publish; --tarball gates the exact file that gets published. Fixes rohitg00#1440 * fix(index): keep vectors across a force-kill with a pending log and a durable backlog A force-kill lost every vector added since the last bucket snapshot (saved at most every 10 minutes), and boot recovery re-embedded at most AGENTMEMORY_VECTOR_BACKFILL_MAX of them. The next snapshot then moved savedAt forward, so the rest were never revisited. - Every vector added to or removed from the live index is also written to a small pending log scope (mem:index:vec-pending), keyed by document id, holding the float32 vector as base64 with session, kind and a monotonic sequence. Removals write a tombstone; a full clear writes one marker entry. - A successful snapshot deletes only the entries it covered (sequence captured at snapshot start). Entries written during the snapshot, or any entry when the snapshot fails, stay. Deletes and writes for the same id run in order. The log triggers an early snapshot once it holds 500 entries, so the scope stays small. - Boot loads the snapshot as before, then replays the log in sequence order (list order is not trusted) with no embedding calls, skipping entries whose dimension does not match the active provider. - Whatever is still missing after replay is re-embedded in batches of AGENTMEMORY_VECTOR_BACKFILL_MAX with a pause between batches until done. A durable backfill marker keeps the recovery window open until the backlog is empty, so a later snapshot cannot skip unrecovered documents and a killed backfill resumes on the next start. The whole-store opt-in (AGENTMEMORY_VECTOR_BACKFILL=all) is unchanged. - /agentmemory/status and the viewer show the pending log size, a pending log write error, and a paused backfill state with plain language fixes. No new settings. Keyless installs write nothing. Existing bucket and legacy snapshots load unchanged. * fix(observe): keep a repeated tool call when its output changed The time-window dedup for events without an eventId keyed only on session, tool name and tool input, so the same command with a new result was dropped for five minutes. The key now also includes a hash of the tool output (tool_response, tool_output, output or error). The same input with the same output inside the window is still skipped. Status and the viewer report how many captures were skipped this way since start. * docs(env): document the capture settings and the vector backfill batch size Adds the capture inbox, dedup, dead letter and offline spool settings, and describes AGENTMEMORY_VECTOR_BACKFILL_MAX as the batch size for re-embedding vectors missing since the last save, which now continues across restarts. * fix(capture): keep accepted observations until the server saved them An accepted capture lives in the engine's memory until the file store writes it on its save timer. A force-kill inside that window lost observations the hook had already discarded on the 2xx answer. The observe response now carries the server's bootId, acceptedAt and durableAfterMs (engine save interval plus margin for the file store, a short fixed window for redis). Hooks keep each accepted event in the local spool as a sent record named by boot id and due time. Later hook calls release same-boot records once due, without a request, and hand records from an earlier boot to the drain, which re-sends them with the same event id so survivors deduplicate. The server reclaims and re-sends records from earlier boots at start and on its sweep timer, keeps what it drains under its own boot id, and releases its own due records. Older hooks ignore the extra fields. New hooks against a server without a boot id discard on 2xx as before. AGENTMEMORY_CAPTURE_SPOOL= false turns all of it off. Kept records count toward the spool byte limit and give way to unsent records. * perf(capture): skip fsync for kept sent records and document the window A kept record only has to outlive a kill of the service, which the page cache already covers; the fsync cost about 5 ms per hook on macOS. Offline spool appends keep their fsync. The README now explains the boot id, acceptedAt and durableAfterMs fields and how hooks use them. * fix(capture): store a crash-split event exactly once The file state store saves each scope on its own timer, so a force-kill can persist the capture event record without the session observations, or the observations without the event record. - Observation ids now come from the event key and the hook timestamp, so a re-sent event whose event record was lost maps to the same observation, vector and BM25 document instead of a second copy. - When an event record says stored but its observation is missing, the re-sent event is stored again under the recorded id instead of being answered as a duplicate. Crash bench (N=1000, kill at 3 s) before: 976 observations, 1028 vector documents; 25 event records pointed to missing observations, 1 observation was stored twice and 27 vectors were orphaned by re-issued ids. * fix(capture): restore a lost vector when a re-sent event is already stored A force-kill can persist an observation while its vector and the vector pending log are lost. The re-sent event then hits the stored observation (or the stored event record) and returns early, which left the observation without a vector until a full backfill was approved. When a re-sent event finds its observation already stored, add the observation back to the BM25 and vector indexes if either is missing. This embeds only that one observation and does nothing when both entries are present. * fix(capture): scrub secrets before an event is written to the inbox Every write path now redacts credentials before text is stored. The capture inbox keeps the hook payload until mem::observe stores it, and a retrying or dead-lettered event can stay there for days, so the payload data is scrubbed before the inbox write. mem::observe scrubs the same way, so stored observations do not change. * Revert "fix(index): keep vectors across a force-kill with a pending log and a durable backlog" This reverts commit 41f0f83. * fix(status): keep the repeated tool call counter with the capture change The "Repeats skipped" row on /agentmemory/status and the viewer Health page reports the observe dedup counter, which belongs to the capture change. It was added together with the vector durability work, which now lives in its own pull request, so the revert removed it. Add it back with its own status test. * build(hooks): emit the shared capture code once as plugin/scripts/_capture.mjs Each hook is built as its own entry, so the capture and spool code was inlined into all eight capture hook bundles. src/hooks/_capture.ts is now its own hook entry, built to _capture.mjs in plugin/scripts and dist/hooks with a fixed name, and the capture hooks import it through a relative ./_capture.mjs import. Hooks that do not capture are unchanged and stay self-contained. Every consumer runs the hooks from the plugin folder (Claude Code, Cursor, Codex, Copilot, Devin, Droid and Antigravity hooks files, and the Antigravity bridge spawns sibling scripts), and the npm package ships plugin/, so the shared module resolves wherever hooks run. No installer copies single hook files. plugin/scripts goes from 9,280 to 3,133 lines. Cold start and spool output are unchanged. * test(status): move the repeat counter test to its own file Keeps test/status.test.ts as on main apart from the capture line, so this change and the vector durability change merge in either order without a conflict. * fix(capture): keep a deleted observation deleted when its event is replayed Since a re-sent event stores its observation again when the event record exists but the observation does not, an observation deleted on purpose came back when a crash re-send or a kept sent record replayed the same event inside the event id window. - The observation now carries the key of its capture event (captureKey), set by the server for client events and copied through synthetic and LLM compression. A captureKey sent by a client is ignored. - Every path that removes observations (forget by id, session forget and the delete endpoints that use it, auto-forget, eviction by age and by project cap, and import with replace) first marks the event record as deleted (state "deleted", deletedAt), then removes the observation. If the event record was lost in a crash, the marker is created from the key on the observation. If the marker cannot be written, the delete fails instead of leaving a record that a replay could bring back. - A replay of a deleted event is answered as a duplicate with the old observation id and stores and indexes nothing. A stranded inbox entry for a deleted event is dropped by the retry sweep. - Pruning counts a deleted marker's age from the delete, so it lasts the full dedup window after it. Audit records and scrubbing on the delete paths are unchanged. * test(release-gate): authenticate requests and check the shared hook module With auth on by default, every gate instance writes a generated secret to its home folder, and unauthenticated requests get 401, so the health check never passed. Gate requests now send that instance's secret as a bearer token. The packed-file check also requires plugin/scripts/_capture.mjs, which the capture hooks import. * test(release-gate): always run install when --only selects scenarios Every scenario needs the installed package, but --only also skipped the install scenario, so a run such as --only capture marked every selected scenario as failed with "install failed". Install now always runs, and the filter still skips the scenarios that were not selected. * test(release-gate): accept vectors held in the pending log before the crash On a slow runner the first vector checkpoint can fire before every captured vector exists. The remaining vectors then wait for the next scheduled save, so the wait for zero pending changes timed out after 60 s on macOS CI. Those vectors are already durable in the pending log and are replayed on boot, so the scenario now waits until every pending change is either saved or in the pending log, then crashes the service and still requires every vector back with no re-embedding. * test(release-gate): drop issue numbers from scenarios and the summary Scenarios carry a name and a description only; the summary no longer reports an issue number per scenario.
) Delete code comments that cite issue or PR numbers, drop the numbers from test names, and strip them from user-facing strings, config comments and the two feature-flag docs links (now pointing at the README environment variables section). Regenerated the agents skill reference from the updated adapter note. No behavior change.
…rohitg00#1464) * feat(bench): measure capture and recovery costs with separate budgets Add benchmark/capture-costs.ts (npm run bench:capture-costs). It runs the built CLI, worker and bundled hooks in an isolated instance per run (fresh HOME and data dir, ports 4900-4903, engine 50100, in-process fake OpenAI-compatible provider on 4950) for keyless, embed and embed-llm profiles, and reports separately: - disk bytes per state scope grouped into source, summary, index, queue, failed delivery, diagnostic, config and stream, fixed versus growth - engine and worker RSS during capture, idle, after recovery and after a warm start, plus CPU seconds - bundled hook latency and stdout bytes, observe latency distribution - provider requests, repeats and returned usage per phase - agent-visible context bytes and server token estimates for search, smart-search, context and hook output with injection off and on - force-kill recovery: time to ready, BM25 rebuild, vectors loaded, backfill, re-embedding requests, logical count and source evidence Invariants fail the run when observations or source go missing or vector documents do not survive recovery. Budgets derived from two repeats at 100 and 1,000 live in benchmark/capture-costs-budgets.json. The corpus generator moves to benchmark/lib/corpus.ts and is shared with load-100k.ts. Results for b3d6cf5 at 100, 1,000 and 10,000 are in benchmark/results and docs/benchmarks/2026-10-01-capture-costs.md. * docs(bench): correct the source share in the capture cost envelope * feat(bench): add Ollama, Redis and crash-offline capture cost runs Extend benchmark/capture-costs.ts with opt-in variants. Defaults are unchanged: fake provider, file state store, standard scenario. - BENCH_PROVIDER=ollama turns the in-process provider into a counting proxy to a local Ollama (BENCH_OLLAMA_URL, BENCH_EMBED_MODEL, BENCH_CHAT_MODEL). It records requests, repeats, returned usage and upstream seconds, and quiesce waits for in-flight upstream calls. - BENCH_STATE_BACKEND=redis with BENCH_REDIS_URL runs the instance on a Redis the bench may flush, and records Redis used memory, key count and server RSS after capture. Budgets are keyed per variant (keyless@redis), apart from the file store. - BENCH_SCENARIO=crash-offline captures through the real post-tool-use hook, SIGKILLs the service mid-capture before the first vector checkpoint (BENCH_KILL_AFTER_MS, optional BENCH_KILL_SETTLE_MS), keeps the hooks running while the service is down so they spool, restarts and checks that every observation and vector comes back with near zero re-embedding. The disk walk now tolerates files removed between readdir and stat, and a failed run stops the RSS sampler so the process can exit. * feat(bench): authenticate bench requests and re-baseline capture cost budgets The bench now sends the generated secret from its own home folder, since auth is on by default. The crash-offline repeat-embedding check allows up to N, because events accepted just before the kill can be embedded again during recovery. Budgets are re-derived on the durable capture and vector durability build: two repeats at 100, 1,000 and 10,000 for keyless, embed and embed-llm on the file store, and at 100 and 1,000 for keyless and embed on Redis. Embedding profiles use more disk than the old baseline because every vector now reaches the index snapshot. The 2026-10-02 report adds the crash-offline, Redis and local model results and supersedes the 2026-10-01 report. * docs(bench): replace committed results with one animated summary figure The raw report JSON and Markdown files (about 38,000 lines) and the two dated result pages are removed: any run of the bench regenerates them, and they go stale with every code change. The bench README now shows one animated SVG with what the runs taught us: vectors and captures survive a force-kill, disk per observation with and without embeddings, and peak memory at 1,000 and 10,000 observations. The figure follows the light or dark color scheme and stays still with reduced motion. Bench output under benchmark/results/ is ignored so runs are not committed by accident. * docs(bench): show what agentmemory costs and what survives a crash The figure now reports what a user gets instead of a comparison with an earlier build: memories and vectors back after a hard kill with no re-embedding, tool calls kept while the service is down, and the cost in disk per memory, RAM, context tokens against the 2,000-token budget, and hook time per tool call. Hook time is shown as the median range with the p95 ceiling across every measured size. The footer states the hardware, engine version, stub embedder, store, date and the limits file. * docs(bench): draw the capture path under a hard kill The summary figure now shows the real capture path instead of stat cards: named tool calls flow through the post-tool-use hook into agentmemory on :3111 and into state, SIGKILL lands on the service, the next calls fall into capture-spool/local-3111.jsonl, and after the restart the spool drains into state. A ledger beside it lists the cost at 10,000 memories on named rows (disk, RAM, context tokens, hook p50 and p95, API keys). Motion plays once with a strong ease-out on transform and opacity only, and reduced motion shows the end state. * chore(bench): stop recording the source commit in the budgets file The budgets file only needs the limits. The derivedFrom field named a local commit, so it is removed from the file and the bench no longer writes it.
Bumps [dotenv](https://github.com/motdotla/dotenv) from 17.4.2 to 18.0.3. - [Changelog](https://github.com/motdotla/dotenv/blob/master/CHANGELOG.md) - [Commits](motdotla/dotenv@v17.4.2...v18.0.3) --- updated-dependencies: - dependency-name: dotenv dependency-version: 18.0.3 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…l replays it (rohitg00#1470) * fix(capture): store a Cursor prompt once when the transcript back-fill replays it Cursor's beforeSubmitPrompt payload carries no prompt id, so the live capture gets a content plus timestamp event id while the session-end transcript back-fill gets a different stable id. The server skips its content dedup whenever an event id is present, so every Cursor prompt was stored twice. The server now keeps a per-session prompt ledger in KV (mem:capture:prompts). Each live prompt_submit observation increments a live count for its normalized text. The session-end hook marks back-fill prompts with backfill: true, and a back-filled prompt is answered as a duplicate while the session has more live captures of that text than back-fill entries already matched against them. Live prompts are never deduped by content, so repeated identical prompts are still both kept, and back-fill entries with no live counterpart are stored. The ledger survives restarts, keeps at most 1000 prompt keys per session, and is pruned with the capture dedup retention window and the capture events cap. Event ids are unchanged, so prompts with a host prompt id behave as before, and hooks without the marker keep the old behavior. * fix(capture): key the prompt ledger by event identity - A failed ledger read now fails the capture instead of being treated as an empty ledger, so the next save cannot overwrite the real counts. - The ledger keeps the identities of live prompts and of claimed back-fill prompts instead of bare counts. Recording or claiming the same event twice is a no-op, so a retried back-fill is still answered as a duplicate without using another live slot. - A ledger update failure on a live prompt is no longer swallowed; the capture retries, and a retry that finds the observation already stored records the live prompt before returning.
…lity (rohitg00#1459) * feat(codex): package local daemon MCP plugin with onboarding * test(codex): verify live daemon flows and published compatibility * fix(codex): secure bridge credentials and validate resource listings * fix(codex): support generated credentials and durable capture packaging
…r engine config (rohitg00#1472) * fix(embedding): let OPENAI_EMBEDDING_API_KEY win over OPENAI_API_KEY The embedding factory passed OPENAI_API_KEY into the OpenAI embedding provider, so the provider's own resolution (embedding key first, then the shared key) never ran. Installs that point embeddings at a separate endpoint with OPENAI_EMBEDDING_BASE_URL sent the chat key there and every embed call failed with 401. - the factory now lets the provider resolve the key from the merged env (process env plus ~/.agentmemory/.env) - blank values are treated as unset - auto-detection also selects the openai provider when only OPENAI_EMBEDDING_API_KEY is set - README env template lists OPENAI_EMBEDDING_BASE_URL and OPENAI_EMBEDDING_API_KEY * fix(hooks): emit structured SessionStart context for hook_event_name payloads With AGENTMEMORY_INJECT_CONTEXT=true the SessionStart hook printed the memory context as plain text. Hosts that add plain SessionStart stdout to the conversation picked it up, but strict hook runners that only read the structured hookSpecificOutput.additionalContext field dropped it. When the payload carries hook_event_name "SessionStart", the hook now prints {"hookSpecificOutput":{"hookEventName":"SessionStart", "additionalContext":...}}, which the documented hook contract accepts for SessionStart (https://code.claude.com/docs/en/hooks), and so does the Codex SessionStart parser. Unchanged: - injection off still prints nothing - the camelCase sessionStart payload keeps the additional_context shape - the Devin path keeps the envelope it already used - payloads without hook_event_name (Copilot CLI) keep plain text - Droid (FACTORY_PROJECT_DIR / DROID_PLUGIN_ROOT set) keeps plain text, which its hook runner documents as SessionStart context pre-tool-use and pre-compact are left as they are: PreCompact has no additionalContext field in the hook contract, and some runners that share these scripts reject additionalContext on PreToolUse. plugin/scripts/session-start.mjs is rebuilt. * docs(engine): document AGENTMEMORY_III_CONFIG and the engine bind address A CLI started inside a container binds the engine to the container's loopback, so published ports reach nothing, and the variable that changes this was only listed by name. The loopback default stays: it is deliberate and covers containers too. This documents how the CLI picks its engine config (AGENTMEMORY_III_CONFIG, ./iii-config.yaml, ~/.agentmemory/iii-config.yaml, bundled), that the chosen file is re-rendered on every start with its host values kept, and that a containerized CLI can opt into 0.0.0.0 by pointing AGENTMEMORY_III_CONFIG at the packaged iii-config.docker.yaml. It also states that docker-compose.yml mounts iii-config.docker.yaml straight into the engine container and never goes through the CLI lookup. - README: new "Engine config and bind address" section - agentmemory-config skill REFERENCE: "Engine config" section - CLI --help: AGENTMEMORY_III_CONFIG under Environment - tests: rendering keeps the bundled config on 127.0.0.1 and keeps the 0.0.0.0 binds of iii-config.docker.yaml * docs(env): document embedding-only OpenAI keys and the engine config path Adds OPENAI_EMBEDDING_BASE_URL and OPENAI_EMBEDDING_API_KEY next to the embedding provider settings, and AGENTMEMORY_III_CONFIG next to the data directory, so .env.example matches the README and the CLI help.
…0#1477) The bucketed index rewrite (rohitg00#1416) replaced test/index-persistence.test.ts and dropped the gating suite that rohitg00#1182 added. Bring it back against the current storage: index_persist rows now come from the legacy migration and its cleanup deletes, and audit rows land in the month scope. The suite clears AGENTMEMORY_AUDIT_INDEX_PERSIST going in and restores it on the way out, then checks that auditing is off by default, that trimmed and case-insensitive "1" and "true" turn it on (with migrate and delete rows), that "0", "false", "yes" and blank values keep it off, and that the index still migrates and reloads from buckets when auditing is off. Co-authored-by: Dmitrii Zhukov <dmitry0983@gmail.com>
…ohitg00#1476) * feat(opencode): support OpenCode 2.x plugin API alongside V1 OpenCode 2 replaced the V1 `Hooks`-object plugin shape. The default export must now carry an `id` plus a `setup(ctx)`, and hooks are registered on the domain that owns the operation. On OpenCode 2.0.20 the existing plugin fails to load entirely: Plugin must export a default definition with an id and an effect or setup function. The plugin now default-exports both implementations from one file: export default { id, setup: v2Setup, server: v1Hooks } V1 calls `server()` and uses the returned hooks; V2 reads `id` and `setup()` and ignores `server()`. The V1 body is unchanged and remains the full 22-hook implementation. The two paths are kept separate on purpose, since sharing an export does not translate V1 hooks into V2 hooks and the payloads differ enough that a shared core would overstate V2 coverage. Ported to V2: event -> ctx.event.subscribe(), tool.execute.before -> ctx.tool.hook, chat.message -> ctx.session.hook("prompt"), and experimental.chat.system.transform -> ctx.session.hook("context"). All 15 consumed event types still exist on the V2 event stream; permission.updated is now permission.asked with a reshaped payload. Three V1 hooks have no faithful V2 equivalent and are dropped on that path rather than approximated, since a wrong-shaped payload reaching agentmemory is worse than a missing one: - config: no mutable global config object and no observing hook. The V2 path takes a one-shot snapshot at setup instead. - chat.params: V2 `context` starts with empty options rather than resolved model settings, so recorded values would be wrong. - experimental.session.compacting: V2's compaction hook can only set `result`, which skips the model call, so context cannot be injected. `@opencode/plugin` is not imported at runtime, so no V2 SDK install is needed. Tested on OpenCode v2.0.20: loads with no errors, and server() still returns all 7 V1 hooks. * fix(opencode): keep the V2 event subscription alive and stop dropping observations Three defects found in the V2 path, all confirmed by reading the code and by running the module under bun. The subscription now wraps its switch body in handleEvent() and invokes it per event. Inline in the for-await loop, the guard `return`s exited the async IIFE and permanently ended the subscription, so the first message.updated for a user role killed the stream and no later session.deleted, /summarize, tool or permission event was ever processed. A throw from a single handler is also isolated now. Verified by driving the real stream: before the fix an event following a guard-return produced 0 observations, after it produces 2. The setup-time config snapshot is parked in pendingConfig when no session is active, which is the normal case, and flushed from the session.created handler after /session/start. It was previously discarded, so config_loaded was almost never sent despite what the README says. The V1 config hook already used this pattern. Tool call dedup now records callID only in the completed and error branches, matching V1. Recording it on the first pending/running update made the terminal update look like a duplicate, so completed tool calls were never observed. * fix(opencode): cast postJson results before reading context postJson returns Promise<unknown | null>, so the three V2 reads of .context produced TS18046. The V1 path already casts at its four read sites; these three were missed during the port. * fix(opencode): port the V2 capture path against the real event stream The V2 path registered and returned a cleanup function, but captured almost nothing. The payload lives in `event.data`, not `event.properties`, so `sid0` was always undefined and every handler returned at its guard. Every event type the handler switched on had been renamed. Rewritten against what v2.0.22 emits. Where the previous port guessed, this one follows the observed runtime: - Events are read from `event.data`; the envelope is `created, data, id, location, type`. Session identity comes from `data.sessionID`, since `location` is process-level. - The V1 event names no longer exist. Mapped to what V2 emits: message text to `session.text.*` and `session.reasoning.*`, `message.part.updated` to `ctx.tool.hook("execute.after")`, `command.executed` to `shell.created`, `session.error` to `session.execution.failed`, `file.edited` to `file.watcher.updated`, `session.status` to `session.step.*`. - `ctx.agent.list()`, `ctx.provider.list()` and `ctx.mcp.list()` resolve to `{ data, location }`. Reading them with `Object.keys()` recorded the literal strings `["location", "data"]` as agent and provider names. - `ctx.model.default()` is a promise. Without `await` it stringifies to `{}`. - There is no `session.created`. A session is registered by the first event carrying its ID, and `session_started` is emitted there, because `session.execution.started` was not observed on the live stream. Tool results come from `execute.after`, which carries `status` and `result.output`, replacing the V1 `message.part.updated` tool branch. The tool name lives on `session.tool.input.started` as `name`, so it is tracked per call ID to label failures. `session.tool.failed` and `shell.exited` are handled as fallbacks, suppressed when the hook already reported the same call. Memory is now injected on every model call instead of once per session: the `context` hook fires per call and exposes `system`, so the previous guard meant only the first prompt carried recalled memory. Validated against payloads captured from a running v2.0.22 rather than hand-built objects, which is how the original shape error survived review. A 36-assertion suite covers every handler, the deduplication paths and the regression where an early `return` ended the subscription permanently. It passes with zero failures, and a live run produced real observations for `post_tool_use`, `config_loaded`, `step_start`, `step_finish`, `assistant_message`, `command_executed`, `reasoning`, `text_started`, `text_ended` and `notification`. The V1 path is untouched: the diff starts at the V2 block. * docs(opencode): document the V2 runtime as observed, not as generated The README claimed all fifteen event types still existed on the V2 stream and that memory could not be attached to the compaction prompt. Both statements were wrong. The V1 event names are gone, the compaction hook does expose system, and the stale SDK types are what made the original port look correct. Each claim is now marked observed or inferred, and the verification section states that the suite runs against payloads captured from v2.0.22. * docs(opencode): explain the ctx:any signature and the double-load risk Both are open questions from review, and both were answered only in PR comments, which is not where someone installing the plugin will look. The `any` is deliberate: typing it from @opencode/plugin would import generated types that are stale for V2, so the compiler would reject correct code while accepting the shape that broke the previous port. The README now states the three risks that follow from that trade, and the comment above v2Setup repeats them for anyone reading the source. The install section previously told people to copy the file into the plugins directory AND list it under "plugins", which can register the plugin twice and capture every event twice. It now presents the two routes as alternatives with an explicit warning against combining them. * docs(opencode): correct the assertion count to 35 The suite has 35 assertions, not 36. Off by one from counting the summary line as an assertion. * fix(opencode): bound per-call state and register every session Three of the five review findings are real leaks or correctness gaps; the other two are documentation that contradicted the code. Registration was gated on `activeSessionId` being unset, so the first session claimed it and every later one skipped registration entirely. A subagent child session running alongside its parent never reached `/session/start` and never got `session_started`, while the handlers below still emitted observations for that ID, so the server received activity for a session it had never seen. Registration is now tracked per session ID, with the global kept only as a fallback for events that carry no ID, and the set is pruned on `session.deleted`. V1 does not have this problem because it registers on its own `session.created` event. `toolNames`, `toolCallInputs` and `reportedToolCalls` were never released on the success path. `toolCallInputs` holds the full `input` of every call, which for `write` and `edit` is the entire file body, so a long-running process accumulated every file it had ever written. Entries are now dropped when `session.tool.success` reports the result, and a session's dedupe keys are cleared on `session.deleted`. The file watcher and filesystem handlers wrote to the stash without applying `MAX_STASHED_FILES`, unlike every other write. Those fire for any workspace change, so the stash could grow without bound and newer files could sit queued behind older entries that never got enriched. All writes now go through one `stashAdd` helper that enforces the cap. The duplicated `reportedToolCalls.add` lines and the duplicated comment in the `execute.after` error branch were leftovers from editing the same block twice. Removed. The install section told people to copy the file into the auto-loaded directory and then, under a "Route A" heading, to list that same copy under `"plugins"` — which is exactly the double registration the section warned about. The copy is now the whole install, the config key is presented only for installing elsewhere, and the warning says which entry to remove. The compaction row in the model table said "not injectable" while the text described it as wired. It is wired but was never observed firing, so it now reads "inferred, unobserved" in both places, and the prose says the same. Suite is now 40 assertions, up from 35, with new cases for multi-session registration, release of per-call state, and the stash cap under a burst of watcher events. Zero failures. The V1 body is untouched: the code diff starts inside `v2Setup`. * fix(opencode): only register a session once /session/start returns The session was added to `registeredSessions` before the request, so a start that failed while agentmemory was restarting left the ID marked as registered and every later event skipped registration for the rest of the process. The session could never be recovered without restarting OpenCode. `postJson` resolves to null on any failure, so the ID is now added only after a non-null response, and a failed attempt drops the per-session state it created so the retry starts clean. Registration is then retried on the next event. Two assertions added, bringing the suite to 42. Both fail against the previous code and pass against this one: a start that fails twice and then succeeds is retried rather than abandoned, and the subscription stays live through the failure. * chore(scripts): detect drift between repo and installed OpenCode plugin The file in ~/.config/opencode/plugins/ is an install artifact and must not be edited directly. On 2026-10-02 it was still the pre-fix build, reading event.properties and capturing nothing, while the PR had moved on eight fixes. The script compares SHA-256, reports drift, and names the V2 blocker explicitly when it finds the installed copy still reading event.properties. It only writes when given -Apply, because silent sync would hide a regression arriving from the PR. Verified in all three states: in sync, drifted, and drifted-then-applied. * fix(opencode): capture compaction from events, drop the dead hook A re-probe against v2.0.22 that creates a session and forces a compaction shows three things the earlier pass got wrong. `session.created` exists and carries sessionID, projectID, location, title, version, subpath and slug. This file claimed it did not. That came from observing an already-open session and never creating one: absence in a sample rather than absence in the API. Registration already ran off the first event carrying an ID, so `session.created` needs no code change, only the correction and a comment on why the fallback still exists. `ctx.session.hook("compaction")` registers a handler that is never invoked, even when `ctx.session.compact()` is called directly and returns a compaction message. The loader validates hook names at registration and not against invocation, so the dead handler is removed. Compaction is captured from `session.compaction.started` and `session.compaction.failed` instead, which means compaction is now observed for real. Memory still cannot be attached to the compaction prompt, because no compaction event carries a system array. `parentID` is accepted by `session.create` and appears in no event payload, so what `/session/start` sends stays null. The field is kept for a future version and documented as unused. `session.execution.succeeded` carries only `{ sessionID }`, already present on every other observation, and `session.step.ended` covers the signal, so it is observed but deliberately not recorded. The V1 body is untouched. * test(opencode): cover the V2 capture path against captured payloads The V2 port shipped dead because its tests agreed with its assumptions: it read `event.properties`, always `{}` on the runtime, and every handler returned at its guard while the suite stayed green. These tests drive the handlers with payloads captured from v2.0.22 and run under the repo's existing vitest, so `npm test` covers the plugin with no new tooling. 33 cases across session lifecycle, tool hooks, memory injection, compaction, the event switch and resilience. Two behaviours they lock in: - the compaction hook is asserted absent, not merely untested, because it registers without error and never fires - the subscription survives events it does not recognise, the regression where a guard `return` inside the for-await ended it permanently Each new assertion was checked against the previous commit: the three compaction cases fail on 7a06239 and pass here. One harness rule is worth stating because it cost an hour: every assertion waits for its own target. An earlier draft waited for `/session/start` and asserted on `session_started`, which is emitted after that await resolves, so it read an empty array and 19 cases failed for a reason that had nothing to do with the plugin. * docs(opencode): correct the session.created and compaction claims Removes the "there is no session.created" claim, corrects the compaction row from "ported, inferred" to the observed reality (the hook never fires, compaction is captured from events and is not injectable), and points Verification at the vitest file that now runs under the repo own test command. Also drops the 42-assertion count, which referred to the bun harness that no longer exists. The suite is 33 vitest cases. * fix(scripts): scope the V2 blocker check so it stops crying wolf The drift detector reported "reads event.properties, capture dead" on a correctly synced file. It grepped the whole file, and the V1 path legitimately reads event.properties because in V1 the payload does live there. A healthy install looked broken. The check now scopes to the v2Setup block, falls back to the whole file for the older V2-only format that predates the dual export, and ignores comments. An earlier attempt still missed the real case because its pattern expected event.properties while the code reads (event as any).properties; matching the word properties in non-comment code within the V2 scope is both simpler and correct. Verified both ways: the 30/09 pre-fix file is flagged, and the synced file is not. * fix(opencode): read the tool error from event.error The failure branch read `result.metadata.error`, but `metadata` carries only exit/status/truncated. The message came out empty, and since the call was then marked as reported the `session.tool.failed` fallback was skipped too, so the error was lost twice over. `event.error` is now in the chain ahead of the metadata fallback. The test covers the documented shape and was checked against the previous code, where it reads an empty string. * fix(opencode): fetch recalled context once per prompt The context hook fires on every model call, tool continuations included, so requesting /context per call multiplied the requests without adding anything: within a prompt the recalled set does not change. The result is now cached for the turn and cleared by the prompt hook, so each new prompt refreshes it. Memory still reaches the system prompt of every call. * fix(opencode): restore the compaction hook and drop redundant observations The compaction hook was removed on the claim that it never fires. That claim was not established: `ctx.session.compact()` only queues a compaction request, and the run that produced the evidence failed immediately with "Nothing to compact yet" because the session was empty. The hook never got a real compaction to run, so it is restored and shares the turn's cached context. `session.text.started`, `session.text.ended` and `session.usage.updated` are no longer recorded. The first two carry no content and the third duplicates the token and cost figures already in `step_finish`, while every observation goes through the capture pipeline. * style(opencode): drop code comments Repo convention keeps the plugin sources comment-free; the runtime notes that explain the V2 shape live in README.md instead of beside the code, so the behaviour is documented once, in the place a reviewer reads first. The V1 body is untouched: the diff starts inside v2Setup. * chore(opencode): remove the local plugin sync script The script compared the repo plugin against one machine's install path to catch drift. That is local setup, not something the repository should carry, and the install path it hardcodes does not exist for anyone else. * docs(opencode): correct the V1 and compaction claims Three claims were wrong or unproven. `session.created` was said not to exist. It does; the earlier reading came from observing an already-open session and never creating one, which is absence in a sample rather than in the API. The compaction hook was said to never fire, and it was removed. That was not established: `compact()` only queues the request and the run behind the claim failed immediately on an empty session, so the hook never had a real compaction to run. The compatibility line named "OpenCode 1.x" with "1.18.29+". OpenCode has no 1.x CLI: `@opencode/cli` goes from 0.0.0 beta straight to 2.0.x, and the V1 plugin API ships in `opencode-ai`, tested here at 1.18.34. A plugin declaring both a `server()` on its default export and a named export is loaded once by that version: the default is used and the named export is ignored. Also records that `session.text.*` and `session.usage.updated` are read but not recorded, and that /context is fetched once per prompt. * fix(opencode): close the V2 capture gaps found in live sessions Checked against OpenCode 2.0.24, 1.18.34 and 1.17.10 in real sessions. - V1 releases call setup() as well as server(), with a context that has no tool, session or event. setup() now returns when those are missing, so V1 no longer runs a second config snapshot and a silently failing V2 registration. - session.execution.succeeded requests /summarize, the V2 equivalent of the V1 idle summary. V2 sessions were never summarized. - session.text.ended is recorded as assistant_message, so the agent's answer reaches memory. - session.instructions.updated records the changed instruction sources. The payload field is delta, a map of source to hash, not text. - Drop observations with no content: reasoning.started, inbox.delivered, and the shell.created entry that duplicated the tool observation. - Skip a config snapshot equal to the previous one. - Remove the comment blocks added with the V2 path. * docs(opencode): record verified V1 and V2 runtime behavior The V1 load is now verified on 1.17.10 and 1.18.34, including the setup() call. Correct the session.created fields, the summarize and instructions behavior, the text and shell event rows, and the standalone delete gap. --------- Co-authored-by: Tyxiel <128870773+Tyxiel@users.noreply.github.com>
* chore: bump version to 0.10.0 Bump every manifest and the export-import contract to 0.10.0: package.json, packages/mcp (and its @agentmemory/agentmemory range to ~0.10.0 so the shim resolves the new release), the Claude, Codex, Cursor, Devin and OpenCode plugin manifests, the hermes and openclaw integrations, src/version.ts, the ExportData.version union and supportedVersions. The deploy templates (fly, railway, render, coolify) now pin AGENTMEMORY_VERSION 0.10.0 with III_VERSION and III_SDK_VERSION 0.22.1, the engine and iii-sdk pair this release is built against. * ci: sync documented numbers and versions automatically Counts in the READMEs, the 11 translations, AGENTS.md, plugin manifests and stat badges were kept by hand and had drifted: most translations still said 124 REST endpoints, the test counts ranged from 596 to 1,674 against about 1,760, and the OpenClaw README said 43 MCP tools. scripts/docs-sync.ts computes each fact from the source (MCP tools, core and local-fallback tools, REST endpoints, skills, hooks, tests, iii functions, source files, lines of code, KV scopes) and rewrites the previously synced value wherever it is followed by that fact's noun in any of the documented languages, so deliberate subset numbers are left alone. scripts/docs-sync.state.json holds the last synced values. A leftover report names any old value it could not place. A version bump in package.json carries into src/version.ts, the export format unions, every manifest that shared the old version, the deploy templates, the AGENTS.md stats heading and CHANGELOG.md, where the Unreleased section becomes the new release with its compare link. npm run docs:sync applies it and npm run docs:check fails on drift; CI runs the check. The Docs sync workflow runs the sync on each pull request and commits the result to the branch, then re-dispatches CI because pushes made with the workflow token start no runs. Fork pull requests get a failing check with instructions. A drift on main opens a sync pull request when a DOCS_SYNC_TOKEN secret is configured and fails with instructions otherwise. This commit also applies the first sync. * chore: run docs sync locally before each commit instead of in CI Drops the Docs sync workflow and the CI docs check. The sync stays a local step: AGENTS.md now tells contributors and agents to run `npm run docs:sync` before every commit, as part of the same build, skills and test sequence CI runs, and to change the source rather than hand-editing documented numbers or versions. * docs: document the 0.10.0 features and sync the documented numbers README: the viewer section now describes the snapshot plus live stream model, the 12 tabs, the Memories, Sessions, Graph and Health pages and the empty states. The environment block gains the storage and durability settings (state backend, Redis URL, state save interval, index save interval, boot graph compaction), the session sweep, the hook capture filters and the audit log settings. The key endpoints table lists GET /agentmemory/status and GET /agentmemory/viewer/snapshot. npm run docs:sync updates the version, REST endpoint count, test count, function count, source file and LOC counts and KV scope count across README, the translated READMEs, AGENTS.md and the stat badges. * docs(changelog): 0.10.0 release notes Turn the Unreleased section into 0.10.0 and cover everything merged since v0.9.29: upgrade notes (engine 0.22.1, boot graph compaction, monthly audit scopes, bucketed index migration, opt-in Redis, 2 s state saves, stop waiting for the engine flush, the session sweep, capture filters and the changed response shapes), then Security, Added, Changed, Fixed and Infrastructure with PR numbers and credit to the external contributors. Adds the compare link. * docs(env): document AGENTMEMORY_AUDIT_INDEX_PERSIST * docs: sync documented numbers after merging main docs:sync after the main merge: 138 REST endpoints, 2,400+ tests, 311 iii functions, 218 source files, 52,000 lines and 60 KV scopes across README, the translated READMEs, AGENTS.md and the test badge. * docs(changelog): finalize the 0.10.0 release notes Rewrite the 0.10.0 entry to cover everything merged since 0.9.29, including durable capture, the offline hook spool, the vector pending log, auth by default, request hardening, the verified engine download, the release gate and the capture cost bench. Group it into breaking changes, upgrade notes, first start migrations, added, fixed, security, performance and docs. Date 2026-10-03. * chore: release as 0.9.30 instead of 0.10.0 Sets the package, plugin manifests, bridge, deploy templates, export format list and tests to 0.9.30, dates the changelog entry to the release day, and adds the embedding key precedence, structured SessionStart context and engine config docs merged from main. * feat(website): rebuild agent-memory.dev around use cases and sourced proof The site leads with what changes for the user instead of how the project is tested, and every number on it links to where it came from. - Monochrome design with Geist, a 1.25 type scale and shared motion tokens; light and dark themes with a toggle. - Hero with a rotating headline across memory types and agents, the install command, a copyable setup prompt for coding agents, and a canvas of the local memory store that answers recall queries. - Live numbers fetched at build time: GitHub stars, contributors, releases, and all-time npm installs of the CLI plus the MCP shim. - Use cases rendered as agent sessions: one memory across Claude Code, Cursor and Codex, session start context, handoff, lessons, commit context and transcript import. - "What your agent gets": a month of history reduced to the 1,571 tokens the next session needs, with sourced recall, token and latency results. - Scroll walkthrough from tool call to recall, LongMemEval-S question matrix, agents grid with connect commands, local-first boundary, viewer screenshots, quotes, FAQ with structured data, and install. - New pages: /benchmarks, /security (including the release gate), /privacy, /changelog generated from CHANGELOG.md, and /vs comparison pages linked from the footer. - robots.txt, sitemap.xml, llms.txt and llms-full.txt for agents and search, and a monochrome Open Graph image. - metadataBase now points at agent-memory.dev instead of the parked agentmemory.dev domain, which broke link previews. - Checked at 13 widths from 320 to 1920 pixels with visible scrollbars, in light and dark themes, with reduced motion respected throughout. * feat(website): brand accent, star CTA and mobile hero hierarchy Bring the docs orange into the monochrome site as the memory color: recall hits, selected history squares, injected context slots, the logo's hot tier and retrieval lines, eyebrows and the hero rank. Buttons and agent logos stay ink and grayscale. Add a Star on GitHub call to action with the live star count, fix the mobile hero so the heading outranks the lede, and stop the outgoing rotating word from peeking into the slot. * feat(website): pill setup-prompt button with agent glyphs Restyle the hero setup-prompt button as a mono pill carrying the Claude, Codex and Cursor glyphs. It lifts on hover with a staggered glyph rise, and on copy turns the brand orange and reads "Copied, paste it into your agent". Hero action buttons share the pill radius. * fix(website,docs): stop scroll jumps and address review findings Website: - How it works: step titles form a fixed index and the active step's text sits in one stacked slot, so the pinned panel no longer re-centers or changes height while scrolling. Short laptop screens get compact figure spacing so nothing clips inside the pin. - Use cases: every scene is sized in one grid cell above 640px, so switching tabs no longer changes the section height. - Reduced motion paints the walkthrough's end state, so Tuesday's session and the recall line are visible. - Footer links Use cases, which the mobile nav hides. - LongMemEval per-type hits use exact counts from the results file. Docs: - docs-sync inventories untracked files and fails check mode when a fact cannot be computed. - OpenClaw README badge shows 54 tools. - The rohitg00#1415 changelog entry no longer describes a BM25 save. * style(design): FINDING-001, agent cards show their connect command The copy command was invisible until hover, so cards looked empty on desktop and hid the command from keyboard and touch users. Show it always, keep the slug unbroken, and use a compact two-column name grid on phones, where the section was about 4000px tall. * style(design): FINDING-002, footer divider sits on the content grid The rule under the ask block was a border on the padded wrap, so it ran 40px past the content edges on both sides. * style(design): FINDING-003, first stat starts on the content grid Every cell carried the same left padding, so the first number sat 24px inside the edge that every other section aligns to. * style(design): FINDING-004, hero keeps two actions and a docs link Three bordered buttons did not fit the copy column, so Read the docs wrapped onto its own row and looked like a third primary action. It is now a text link under the setup prompt and GitHub star. * style(design): FINDING-005, use-case scenes sit centered in the stage The stage keeps the tallest scene height so tab switches do not move the page; shorter scenes were pinned to the top with a band of empty space below. Center them vertically instead. * style(design): FINDING-006, featured badges share the card left edge On phones the two badge cards centered their image while the outlet cards above were left aligned, so the strip had two alignments. * style(design): FINDING-007, footer file link stays on one line INSTALL_FOR_AGENTS.md broke before .md in the narrow last column. That column now sizes to its content. * style(website): show agent logos in color in the hero headline The rotating agent logo next to the headline was forced to grayscale. Show each logo in its real colors. * docs(release): fold the OpenCode 2 and audit test PRs into the 0.9.30 notes - CHANGELOG: add OpenCode 2 support (rohitg00#1476, building on rohitg00#1451 by Tyxiel), cite rohitg00#1477 for the restored audit gating suite, credit Tyxiel, and date the release 2026-10-06. - CONTRIBUTING: the release process now matches AGENTS.md. Bump package.json and let docs:sync carry the version. - Test badges: the aria-label still read 1648+. docs:sync now rewrites the aria-label along with the visible number. * fix(website): date the benchmark sources and align the data dir copy - The 14 ms recall figure comes from coding-agent-life-v1 on v0.9.26 and the token figures from the v0.6.0 evaluation. Say so next to each number on the home page, the benchmarks page and llms.txt. - The security page now lists $XDG_DATA_HOME for Linux, matching llms-full.txt and the CLI. - Regenerate the build-time meta and changelog data for 0.9.30. * fix(docs): auth note in translations and docs-sync version rules - The translated READMEs still said protected endpoints need a bearer token only when AGENTMEMORY_SECRET is set. Auth is on by default; match the English README in all eleven. - docs-sync resolves its root from import.meta.url, so it runs on every Node 20 release the package supports, not only 20.11 and later. - A version bump now also moves the @agentmemory/agentmemory dependency in packages/mcp/package.json, and every matching version rule applies to a file instead of only the first. * fix(copilot): validate bundled MCP and lifecycle capture * fix(antigravity): validate native capture and shared configuration
* docs: translate README into 32 languages Adds 21 new translations (Dutch, Polish, Czech, Romanian, Hungarian, Greek, Swedish, Danish, Norwegian, Finnish, Ukrainian, Hebrew, Arabic, Bengali, Urdu, Thai, Vietnamese, Indonesian, Tagalog, Traditional Chinese, European Portuguese) and refreshes the existing 11 against the current English README. Every README now carries the same flag language bar. Hebrew, Arabic and Urdu render right to left. * docs: sync translations with the 0.9.30 README and teach docs-sync their nouns Carries the Codex hook trust, Copilot VS Code and Antigravity 2.0 wording from the merged English README into all 32 translations, and extends the docs-sync noun lists so the documented counts are found in every language.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upgrades the fork to upstream v0.9.30 and iii-engine 0.22.1. Capture now uses the durable inbox and offline spool, API clients share the generated local secret, and the viewer uses the upstream snapshot and streaming protocol.
Preserves the fork's durable recall and context selection, expansion-only search, environment-file loading, injection gates, data directory support, synthetic fallback indexing, graph sharding, journal and side indexes, and bounded persistence retries. Packages and plugin manifests identify the fork as 0.9.30-codex.1.
Fixes three issues caught during integration: semantic/procedural list responses now paginate below the engine frame limit, hook environment loading keeps file credentials out of remote URL overrides, and capped graph provenance retains every reverse observation link.
Validation includes 2,681 passing unit tests, build, generated skills and docs checks, all ten packed release recovery scenarios, and live capture tests for three hosts. The rendered viewer was tested against a private cold copy of the installed store, including its 23,240 semantic facts. It now loads without worker disconnects. Review evidence is bound to the final head in the local evidence directory.
The upstream tag has existing standalone TypeScript diagnostics; integration diagnostics were reconciled against that baseline. Historical legacy audit storage is retained when the upstream bounded migration freezes it, although those old rows are omitted from current audit queries.