Repository navigation
tests: prompt-injection attack corpus wired to the memory data boundary - #268
Closed
raymondginger2018-sudo wants to merge 1 commit into
Closed
raymondginger2018-sudo wants to merge 1 commit into
raymondginger2018-sudo wants to merge 1 commit into
Conversation
Add a self-contained corpus of lesson-13 prompt-injection attack samples across DeepCode's four surfaces (spawn prompt, tool output, memory note, MCP content) and push the memory-note samples through the real _frame_data_block boundary, so a future change that drops the framing fails the suite instead of silently regressing. Corpus is tests-only: it has no runtime consumer, so it lives with its tests instead of adding an uncalled core/ module.
Collaborator
|
Thank you, @raymondginger2018-sudo. We're closing this one. Only the two A test we'd take: a payload sent through a real code path that the existing tests don't cover yet, with an assertion on what that code returns. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a self-contained prompt-injection regression corpus (lesson 13's #1
threat) as a single test file, with the
memory_noteattack samples pushedthrough the real data boundary in
core/harness/memory.py.Changes
tests/test_injection_corpus.py(new, tests-only, +190):output, memory note, MCP content), each tagged with the guard it must
satisfy;
assert_surface_coverage) plus corpus shape/filter checks;memory_notesample through_frame_data_blockand asserts the boundary markers, the restrictclause, and verbatim payload containment.
Tests
pytest tests/test_injection_corpus.py tests/test_memory.py-> 30passed (5 new cases + 25 existing memory-boundary cases) on the branch;
ruff check+ruff formatgreen (pre-commit pinned rev, hooks passed atcommit).
no runtime consumer, so it lives with its tests instead of adding an
uncalled
core/module.Dependency note
core.harness.memory:_frame_data_block,_BOUNDARY_OPEN,_BOUNDARY_CLOSE,_RESTRICT_CLAUSE).No new dependencies.