Skip to content

Captured conversation re-sent after a client write timeout → duplicate extracted memories #106

Description

@lehneres

Summary

When the plugin's capture write (ingestConversation / addMemory) times out on the client — particularly during an extraction backlog — the captured batch is retried on every later event and again at session end, re-POSTing the same customId. Each re-POST makes the server run document extraction and chunking again, appending a fresh set of memory rows for content that was already captured.

We measured this on our self-hosted setup (local/supermemory-server:0.0.8, opencode-supermemory 2.0.15, OpenCode 2.0.24):

  • Fact "The System bugcheck 0x116 (VIDEO_TDR_FAILURE) occurred on September 4 and September 6, 2026" was stored 55×.
  • Three stepper-layout facts were stored 42× / 42× / 41×.
  • Every duplicate family traces to exactly one source document customId — i.e. the same document was re-ingested, not the facts re-derived from different sources.
  • Mean number of distinct fact texts was only ~3,900 of ~7,700 rows.

The client log captured the cause directly: a session.execution.interrupted / session_end fired captureSessionEnd, which re-ran captureCadence over all complete batches and re-saved each with the same customId; three identical [capture] conversation batch saved lines for the same ses_...:1-3 window appeared 1 ms apart.

Root cause

completedCaptureIds in the V2 runtime is an in-memory Set, populated only after a successful save. A client timeout leaves the batch unguarded, so it is re-POSTed on each subsequent event. At session end captureSessionEnd re-runs over every complete-cadence batch and POSTs each again.

Proposed fix (we applied this locally)

Make capture idempotent rather than retrying blind:

  1. Check before write. Before POSTing, GET /v3/documents/<customId> and treat done | indexing | queued | failed as already captured (a failed document must not be re-POSTed — re-POSTing it is precisely what duplicates the memories).
  2. Persist attempt state. Write a small capture-ledger.json (0600) alongside the plugin, so the guard survives reload, service restart, and Mac sleep/wake.
  3. Back off, don't storm. Only retry within a short window (~30 s) and only if the server shows the document is absent/unknown — do not re-POST unboundedly.

With that change, re-evaluating the same session at session_end produces batch already on server, not re-POSTing and creates zero new documents. Patch is available and can be shared.

Separate from #104 (per-location tool tools disappearing) and from #99 (per-location ownership PR); that is why we are filing this distinctly.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions