Skip to content

feat: add tool script safety guard#118

Open
Spock12138 wants to merge 5 commits into
trpc-group:mainfrom
Spock12138:fix/tool-safety-guard
Open

feat: add tool script safety guard#118
Spock12138 wants to merge 5 commits into
trpc-group:mainfrom
Spock12138:fix/tool-safety-guard

Conversation

@Spock12138

@Spock12138 Spock12138 commented Jul 4, 2026

Copy link
Copy Markdown

Related Issue

Resolves #90

Summary

Add Tool Script Safety Guard with static scanning, policy configuration, Tool Filter integration, CodeExecutor wrapper, audit events, examples, CLI, and documentation.

What Changed

  • Added trpc_agent_sdk.tools.safety with policy loading, redaction, rule matching, Python/Bash scanning, audit helpers, Tool Filter integration, and CodeExecutor wrapper.
  • Added scripts/tool_safety_check.py for standalone scanning.
  • Added examples/tool_safety_guard/ with policy, 12 samples, generated report/audit fixtures, runner, and README.
  • Added safety tests covering policy, redaction, matchers, rules, scanner behavior, audit, filter integration, CodeExecutor wrapper, and CLI/examples.

Validation

  • 12 public samples scan successfully with structured reports
  • 12/12 sample decisions correct
  • Dangerous delete / secret file read / non-allowlisted egress detected
  • 500-line Python and Bash scans complete under 1s
  • Safety tests pass: 114 passed
  • ToolSafetyFilter blocks before execution and records audit
  • SafetyGuardedCodeExecutor blocks before delegate execution and records audit
  • Report includes decision / risk_level / rule_id / evidence / recommendation
  • Audit JSONL includes tool_name / decision / risk_level / rule_ids / elapsed_ms / redacted / blocked
  • No dependency changes
  • Secret scan and whitespace checks pass

Known Non-Blocking Items

  • mempalace and e2b_code_interpreter optional extras are not installed locally, so related full-suite collection failures are environment-related.
  • Some existing Windows path/POSIX permission tests fail outside the safety code path.
  • subprocess_call may produce an extra same-rule finding; it does not affect final decision or acceptance criteria.

- Apply yapf formatting to all changed SDK safety files (_scanner, _code_executor, _cli_helpers, _redaction, _rules)

- Add logging.disable(NOTSET) to _enable_caplog_logger helper to undo global logging disable from cross-test pollution
@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.21317% with 278 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@099b571). Learn more about missing BASE report.

Files with missing lines Patch % Lines
trpc_agent_sdk/tools/safety/_cli_helpers.py 0.00000% 127 Missing ⚠️
trpc_agent_sdk/tools/safety/_scanner.py 78.57143% 93 Missing ⚠️
trpc_agent_sdk/tools/safety/_filter.py 81.16883% 29 Missing ⚠️
trpc_agent_sdk/tools/safety/_code_executor.py 90.27778% 14 Missing ⚠️
trpc_agent_sdk/tools/safety/_audit.py 89.47368% 4 Missing ⚠️
trpc_agent_sdk/tools/safety/_matchers.py 95.06173% 4 Missing ⚠️
trpc_agent_sdk/tools/safety/_redaction.py 89.47368% 4 Missing ⚠️
trpc_agent_sdk/tools/safety/_policy.py 96.87500% 2 Missing ⚠️
trpc_agent_sdk/tools/safety/_rules.py 98.52941% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main        #118   +/-   ##
==========================================
  Coverage        ?   87.25293%           
==========================================
  Files           ?         478           
  Lines           ?       45281           
  Branches        ?           0           
==========================================
  Hits            ?       39509           
  Misses          ?        5772           
  Partials        ?           0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

构建 Tool 执行脚本安全扫描、Filter 拦截与监控机制

1 participant