Skip to content

refactor(tools): enforce disallowed_tools only through the rule layer - #8747

Open
CodeGhost21 wants to merge 1 commit into
tinyhumansai:mainfrom
CodeGhost21:chore/364-disallowed-tools-rule
Open

CodeGhost21 wants to merge 1 commit into
tinyhumansai:mainfrom
CodeGhost21:chore/364-disallowed-tools-rule

Conversation

@CodeGhost21

@CodeGhost21 CodeGhost21 commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Draft: needs a maintainer decision on one behaviour change (Option A/B/C below) before review.

Summary

  • Step 1 of [core-shrink] reimplement: fold tool visibility layers onto ToolRuleSet senamakel/openhuman#364: disallowed_tools is now enforced only through the harness ToolRuleSet deny rule (agent_rule_layer). Before, the same list was also re-filtered by glob_list_matches at five session, sub-agent and definition call sites.
  • Deletes those re-filters and the glob_list_matches wrapper. The agent-library RPC listing keeps its filter, because the rule gate never sees it.
  • Adds regression tests: a disallowed tool is refused by name, absent from tool_search, and never executed, for both sessions and sub-agents.

Problem

Tool visibility was decided twice for the same list. That's two places to keep in sync, and the hand filters only removed tools from the session's tool list. A guessed name or tool_search relied on the rule layer anyway.

Solution

Call site Result Why the rule layer covers it
session_host/builder/factory/build_session.rs (Wildcard and Named) removed scope_def keeps disallowed_tools. Each turn installs session_rule_set (ensure.rs → turn_tool_rules → turn_run_policy → install_turn_rules), and the system prompt drops those tools (rule_withheld_tools).
subagent_host/tool_prep.rs (first pass, the recovery-tool exception, and extra_tools) removed for_subagent(definition) → child_rule_policy adds agent_rule_layer, which reaches run_subagent_via_graph → TurnSpec. rule_listed_indices keeps denied tools off the child's prompt.
tinyagents/host/definition_registry.rs (Named and Wildcard) removed project() sets tool_rules: agent_rule_layer(def), and the harness applies it in ToolGate::new. Hosted children inherit it through inherited_rules.
agent_registry/library/ops.rs kept An RPC display (the #8681 fix) that the rule gate never sees. It now calls tinytools::glob_matches directly.

The issue's ops/runner.rs and spawn_parallel/staging.rs call sites no longer exist on main.

Glob semantics are identical. glob_list_matches was patterns.iter().any(|p| tinytools::glob_matches(p, name)). ToolRule::names(Deny, …) matches through Patterns::matches, which is the same glob_matches (*/?, ASCII case-insensitive). With an empty on list it applies on every surface.

Behaviour changes: decision needed

  1. A sub-agent that calls a disallowed tool by name now stops.
    • Before, the tool was never registered, so the call got a recoverable "unknown tool" error and the child kept going.
    • Now it gets the rule refusal, which repeated_failure / ClassifiedFailureTracker classifies as permission. The sub-agent halts after one attempt ("Stopping after 1 attempt(s): failure class permission …") and ends Incomplete. Root sessions very likely behave the same, since they use the same middleware.
    • This is what a config [tool_rules] deny already does today. Models do guess tool names, though, so runs that used to recover can now end early.
    • Option A: accept it (this PR as written).
    • Option B (recommended): treat a deny refusal as recoverable the first time, so the model can correct course, while the rule stays the only check. That's a harness and middleware change, and it also affects [tool_rules] denies.
    • Option C: keep the duplicate filters and drop step 1.
  2. A wildcard agent with disallowed_tools now gets the normal wildcard tool list. The old Wildcard branch turned the list into "all tools minus denied", which disabled tool_search, listed Deferred tools directly and kept Hidden todo_* tools visible. Now tool_search is on and Deferred and Hidden tools are stripped. Only custom agents with tool_allowlist = ["*"] plus a denylist are affected; every built-in agent has a Named scope.
  3. The refusal text differs from the old "unknown tool" wording.

Submission Checklist

  • Tests added or updated:
    • tools/rules/ops_tests.rs runs a real AgentHarness with rules installed: a_session_refuses_disallowed_tools_by_name_and_in_tool_search, a_sub_agent_refuses_its_disallowed_tools_by_name_and_in_tool_search and a_sub_agent_refuses_its_own_disallowed_tools_under_a_restricted_parent. They cover exact names and globs, direct and deferred tools, and check that an allowed tool still runs.
    • typed_mode_refuses_disallowed_tools_called_by_name is an end-to-end typed-mode run.
    • Existing tests are rewritten to assert through the rule layer.
  • Diff coverage ≥ 80%: the new tests exercise the changed paths; the CI gate measures it.
  • Coverage matrix updated: N/A, no feature rows change
  • All affected feature IDs listed: N/A
  • No new external network dependencies introduced
  • Manual smoke checklist: N/A
  • Linked issue: N/A, this is step 1 of [core-shrink] reimplement: fold tool visibility layers onto ToolRuleSet senamakel/openhuman#364, which stays open for steps 2 and 3

Impact

  • Agents and sub-agents: see "Behaviour changes" above. Nothing changes for agents without disallowed_tools; the change is a no-op for an empty list.

Related


AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: chore/364-disallowed-tools-rule
  • Commit SHA: 6aa6a51ced

Validation Run

  • pnpm --filter openhuman-app format:check: N/A
  • pnpm typecheck: N/A
  • Focused tests: typed_mode_, tool_prep, tools::rules::, definition_registry and agent_registry::library pass 85/85. A wider run of agent::session_host:: and agent::subagent_host:::
    • 481 passed, 61 failed;
    • 57 of the failures are secrets-dir, auth-store and security-module environment errors;
    • the other 4 (typed_transcript_compat ×2, a_detached_driver_uses_its_carried_actor_for_lifecycle_commits, resolve_subagent_source_hint_with_config_routes_via_factory) were rerun on origin/main dbbaf7f8db and fail identically there.
  • Rust fmt/check:
    • cargo test --no-run passes with product features, and gates-off passes with --lib --tests.
    • Touched files are rustfmt-clean.
    • module-boundaries, process-statics, agent-runtime-boundary and storage-bypass all pass.
  • Tauri fmt/check: N/A

Validation Blocked

  • command: check-test-ownership, check-security-module-dependencies
  • error: test-ownership flags three files this PR doesn't touch; security-module-deps can't run without local node_modules
  • impact: none from this branch

Behavior Changes

  • Intended behavior change: one enforcement point for disallowed_tools; see the decision above
  • User-visible effect: a sub-agent that calls a denied tool stops (Option A) instead of getting "unknown tool"

Parity Contract

  • Legacy behavior preserved: the same tools are hidden and refused, with identical glob semantics
  • Guard/fallback/dispatch parity checks: name call, tool_search and catalogue are all covered by the new tests

Duplicate / Superseded PR Handling

  • Duplicate PR(s): none
  • Canonical PR: this
  • Resolution: N/A

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Behavior Changes
    • Disallowed tools are now consistently hidden from tool catalogues and search results, and refused when called directly—including in sub-agents and restricted sessions.
    • Tool scopes may retain disallowed tools, but deny rules prevent their use. Exact names and wildcard patterns are supported.
    • Allowed tools remain available and continue to work; recovery tools are also subject to deny rules.

`disallowed_tools` already becomes a deny rule (`agent_rule_layer`) that the
harness tool gate applies to the catalogue, `tool_search` and every call, on
session turns (`session_rule_set`), sub-agent turns (`child_rule_policy`) and
hosted runs (the projected definition's `tool_rules`). Drop the duplicate
`glob_list_matches` re-filters in the session builder, sub-agent tool
preparation and the hosted definition projection, and delete the wrapper.

The agent-library listing keeps its own filter (an RPC display the rule gate
never sees), now on `tinytools::glob_matches` directly.

Regression tests drive a real harness turn with no allowlist and check that a
disallowed tool, exact or globbed, direct or deferred, is refused by name and
never surfaced by `tool_search`, for both session and sub-agent rules, plus a
typed-mode sub-agent run that calls a disallowed tool by name.

Refs senamakel#364

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Tool scopes and sub-agent preparation no longer remove tools matched by disallowed_tools. Rule policies enforce denials across catalogues, tool_search, and calls. Agent registry listings continue to filter disallowed names.

Changes: Tool denial policy

Layer / File(s) Summary
Scope projection and sub-agent preparation
crates/openhuman-core/src/agent/tinyagents/host/definition_registry.rs, crates/openhuman-core/src/agent/tinyagents/host/definition_registry_tests.rs, crates/openhuman-core/src/agent/subagent_host/tool_prep.rs, crates/openhuman-core/src/agent/subagent_host/*tests*.rs, crates/openhuman-core/src/agent/subagent_host/README.md, crates/openhuman-core/src/agent/subagent_host/mod.rs
Named and wildcard scopes retain names matched by disallowed_tools. Sub-agent filtering no longer applies the denylist. Tests check that child rule policies refuse denied tools, including recovery tools.
Rule enforcement and runtime coverage
crates/openhuman-core/src/tools/rules/*, crates/openhuman-core/src/agent_registry/library/ops.rs, crates/openhuman-core/src/agent/session_host/builder/factory/build_session.rs
Tests cover denial from catalogues and tool_search, refusal of calls, and non-execution of denied tools. Agent registry listings filter disallowed names with glob matching. Comments describe the denial behavior and recovery-tool visibility.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant DefinitionRegistry
  participant SubagentToolPrep
  participant ChildRulePolicy
  participant Harness
  DefinitionRegistry->>SubagentToolPrep: projected tool names
  SubagentToolPrep->>ChildRulePolicy: prepared tools and disallow rules
  ChildRulePolicy->>Harness: catalogue and tool_search denials
  Harness->>ChildRulePolicy: tool call by name
  ChildRulePolicy->>Harness: permission refusal for denied tool
Loading

Possibly related PRs

Suggested labels: rust-core, agent

Suggested reviewers: senamakel, m3ga-mind

Merge Risk: Minimal · up to 6aa6a

This refactor moves disallowed_tools enforcement into the rule layer, and the new tests exercise it for sessions and sub-agents. A sub-agent that calls a denied tool by name now gets a permission refusal instead of an "unknown tool" error. The author flagged that behavior change for a maintainer decision. No concrete defect was found, but the failing CI coverage lane should be confirmed unrelated before merging.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: disallowed_tools is now enforced only through the rule layer. The changeset removes duplicate disallowed_tools filtering from session, sub-agent, and definition tool p…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: disallowed_tools is now enforced only through the rule layer. The changeset removes duplicate disallowed_tools filtering from session, sub-agent, and definition tool preparation, and the rule layer becomes the single enforcement point. The title is specific, concise, and uses a conventional commit prefix.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

I’m a rabbit with a rulebook in my paws,
Denied tools meet a careful pause.
Names stay listed, calls get checked,
Search and catalogues show respect.
I hop along as tests inspect.

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8645e9e4-1ab5-49e7-955e-55e71a468860
Commits

Reviewing files that changed from the base of the PR and between dbbaf7f and 6aa6a51.

Files selected for processing: 13

  • crates/openhuman-core/src/agent/session_host/builder/factory/build_session.rs
  • crates/openhuman-core/src/agent/subagent_host/README.md
  • crates/openhuman-core/src/agent/subagent_host/mod.rs
  • crates/openhuman-core/src/agent/subagent_host/ops_tests_slug_filter_typed_mode_tests.rs
  • crates/openhuman-core/src/agent/subagent_host/tool_prep.rs
  • crates/openhuman-core/src/agent/subagent_host/tool_prep_recovery_visibility_tests_tests.rs
  • crates/openhuman-core/src/agent/subagent_host/tool_prep_tests.rs
  • crates/openhuman-core/src/agent/tinyagents/host/definition_registry.rs
  • crates/openhuman-core/src/agent/tinyagents/host/definition_registry_tests.rs
  • crates/openhuman-core/src/agent_registry/library/ops.rs
  • crates/openhuman-core/src/tools/rules/mod.rs
  • crates/openhuman-core/src/tools/rules/ops.rs
  • crates/openhuman-core/src/tools/rules/ops_tests.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

@CodeGhost21 CodeGhost21 changed the title refactor(tools): enforce disallowed_tools only through the rule layer (#364 step 1) refactor(tools): enforce disallowed_tools only through the rule layer Nov 16, 2026
@CodeGhost21
CodeGhost21 marked this pull request as ready for review November 16, 2026 18:21
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T18:29:44.204974Z 6aa6a51 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai coderabbitai Bot added agent Built-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/. rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure. labels Nov 16, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6aa6a51ced

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +330 to +333
// "no filter", so it needs nothing. An agent that disallows it still
// loses it: `disallowed_tools` is a deny rule in the session's rule
// set (`tools::rules::agent_rule_layer`), which the prompt catalogue,
// the harness catalogue, `tool_search` and every call obey.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep wildcard belts empty before builder expansion

With a custom ToolScope::Wildcard definition that has a nonempty denylist and a { skills = "*" } entry yielding deferred Composio tools, removing the Wildcard branch leaves visible containing only delegation_tools (lines 316–322). SessionHostBuilder::build then sees a nonempty set and sets belt_is_wildcard false (builder_build.rs:128), so it never expands the durable registry: all ordinary allowed tools and tool_search disappear, rather than just the names denied by the rule. Keep the wildcard represented as empty through this path, or materialize the full allowed set before applying the rule.

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Built-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/. rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant