Skip to content

tests: prompt-injection attack corpus wired to the memory data boundary - #268

Closed
raymondginger2018-sudo wants to merge 1 commit into
HKUDS:mainfrom
raymondginger2018-sudo:pr/injection-corpus
Closed

raymondginger2018-sudo wants to merge 1 commit into
HKUDS:mainfrom
raymondginger2018-sudo:pr/injection-corpus

Conversation

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor

Summary

Add a self-contained prompt-injection regression corpus (lesson 13's #1
threat) as a single test file, with the memory_note attack samples pushed
through the real data boundary in core/harness/memory.py.

Changes

  • tests/test_injection_corpus.py (new, tests-only, +190):
    • 9 attack samples across the four injection surfaces (spawn prompt, tool
      output, memory note, MCP content), each tagged with the guard it must
      satisfy;
    • drift guard (assert_surface_coverage) plus corpus shape/filter checks;
    • parametrized regression test that runs every memory_note sample through
      _frame_data_block and asserts the boundary markers, the restrict
      clause, and verbatim payload containment.

Tests

  • pytest tests/test_injection_corpus.py tests/test_memory.py -> 30
    passed
    (5 new cases + 25 existing memory-boundary cases) on the branch;
    ruff check + ruff format green (pre-commit pinned rev, hooks passed at
    commit).
  • No production files touched: the corpus is tests-only by design — it has
    no runtime consumer, so it lives with its tests instead of adding an
    uncalled core/ module.

Dependency note

  • Imports only boundary helpers that exist on main (core.harness.memory:
    _frame_data_block, _BOUNDARY_OPEN, _BOUNDARY_CLOSE, _RESTRICT_CLAUSE).
    No new dependencies.

Add a self-contained corpus of lesson-13 prompt-injection attack samples across DeepCode's four surfaces (spawn prompt, tool output, memory note, MCP content) and push the memory-note samples through the real _frame_data_block boundary, so a future change that drops the framing fails the suite instead of silently regressing. Corpus is tests-only: it has no runtime consumer, so it lives with its tests instead of adding an uncalled core/ module.
@Zongwei9888

Copy link
Copy Markdown
Collaborator

Thank you, @raymondginger2018-sudo. We're closing this one. Only the two memory_note samples reach real code; the other six entries, and three of the four tests, check the corpus itself. The memory boundary already has stricter tests in tests/test_memory.py (lines 297 and 314 on main), which check that a note can't close <untrusted-data> in any spelling or forge a frame. The new test asserts that the payload stays verbatim inside the boundary, which holds only because neither sample tries to close it.

A test we'd take: a payload sent through a real code path that the existing tests don't cover yet, with an assertion on what that code returns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants