Skip to content

Commit ce8e5ea

Browse files
feat(sandbox): generate CLI recording shims via record_cli
cli_called reads a JSON Lines invocation log, but nothing produced one: every suite had to hand-write a recording mock and get the record shape right, making the format a contract between the harness and each consumer repository. That is how contracts drift — and it drifted inside a single downstream suite, where two of five mock templates were copies that never gained the log. record_cli closes the loop. Declaring a tool generates a self-contained shim into cli_mocks/, PATH-prepended through the existing mock_path_dirs machinery, appending records to cli_mocks/calls.jsonl — which cli_called now reads by default, so a task sets neither mock_path_dirs nor log:. sandbox: record_cli: - {tool: uip, exit_code: 1, stderr: "not connected\n"} - {tool: curl} The shim records the invocation, writes the configured output, and exits. Nothing is executed: no network, no auth, no side effects. It stubs a tool; it does not proxy one, and it serves no per-invocation responses. Recording a REAL executable on the way through depends on the tool being installed, on PATH order, and usually on live credentials — state the harness cannot guarantee — so that stays a hand-written wrapper under mock_path_dirs, as does anything needing a fixture set. Keeping the generated shim to the case that is always well-defined is what lets every record carry a real exit code and keeps the shim free of platform-conditional code. Decisions worth noting: - A .cmd twin ships beside each shim so a bare `uip` also resolves through Windows PATHEXT lookup. - The log is seeded empty: a correct run that calls nothing must satisfy max_count: 0, while a MISSING log (mock never ran) must still fail. - stdin is never read — it would block whenever the sandbox leaves it on an open pipe, hanging the task. - A name collision with a mock_path_dirs entry raises instead of letting directory order silently decide which executable runs. - The rendered shim imports nothing from coder_eval and is pure ASCII: it runs inside a sandbox where this package is not installed. 21 new tests, including the round trip that matters — generate, execute, then grade the produced log with cli_called and no log: configured. Full suite, ruff, pyright and all 166 custom lint rules pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 04dbb05 commit ce8e5ea

7 files changed

Lines changed: 631 additions & 3 deletions

File tree

docs/TASK_DEFINITION_GUIDE.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ Complete reference for defining evaluation tasks in Coder Eval.
1717
- [Agent Configuration](#agent-configuration)
1818
- [Run Limits](#run-limits)
1919
- [Sandbox Configuration](#sandbox-configuration)
20+
- [Recording CLI Invocations](#recording-cli-invocations)
2021
- [Template Sources](#template-sources)
2122
- [Success Criteria](#success-criteria)
2223
- [Continuous Scoring](#continuous-scoring)
@@ -430,6 +431,31 @@ Under `driver: tempdir` only `timeout` is enforced — the agent can consume
430431
arbitrary host memory, CPU, and PIDs. Use `driver: docker` when you need the
431432
container limits above to actually bind.
432433

434+
### Recording CLI Invocations
435+
436+
`record_cli` shadows executables with generated recording shims, so a task can assert on **what the agent actually ran** without hand-writing a mock:
437+
438+
```yaml
439+
sandbox:
440+
record_cli:
441+
- tool: uip
442+
exit_code: 1
443+
stderr: "uip: not connected to a tenant in this sandbox.\n"
444+
- tool: curl # so a disobedient agent cannot reach the network
445+
```
446+
447+
Each shim records the invocation, writes the configured `stdout`/`stderr`, and exits with `exit_code`. **Nothing is executed** — no network, no auth, no side effects.
448+
449+
The sandbox writes the shims into `cli_mocks/` and PATH-prepends that directory, then appends one JSON record per invocation to `cli_mocks/calls.jsonl` — the log [`cli_called`](#cli_called) reads by default. Nothing else to wire: no `mock_path_dirs`, no `template_sources`, no `log:` on the criterion.
450+
451+
Notes:
452+
453+
- **A `.cmd` twin** is generated beside each shim so a bare `uip` also resolves through Windows PATHEXT lookup.
454+
- **The log is seeded empty**, so a correct run that legitimately calls nothing still satisfies a `max_count: 0` guard — while a *missing* log (mock never ran, or wrote elsewhere) still fails.
455+
- **stdin is never read** by the shim: reading it would block whenever the sandbox leaves stdin attached to an open pipe, hanging the task.
456+
- **Collisions are rejected.** If a `mock_path_dirs` entry already provides an executable of the same name, setup raises rather than letting directory order decide which one runs.
457+
- **It stubs a tool; it does not proxy one, and it does not serve per-invocation responses.** Recording a *real* executable on the way through, or returning different output per invocation, stays a hand-written mock under `mock_path_dirs` — both depend on state the harness cannot guarantee (the tool being installed, PATH order, live credentials, a fixture set).
458+
433459
## Template Sources
434460

435461
Tasks can start with preset files instead of an empty sandbox. Multiple sources are applied sequentially (last wins for conflicts).
@@ -780,6 +806,8 @@ Use this instead of `command_executed` or `file_matches_regex` when a test shado
780806
ignore_flags: ["output"] # Flags dropped before matching (default: ["output"])
781807
```
782808

809+
`log` defaults to `cli_mocks/calls.jsonl`, where [`sandbox.record_cli`](#recording-cli-invocations) writes — so a task using generated recorders never sets it. Point it elsewhere only when supplying your own mock.
810+
783811
**Log format.** One JSON object per line. Only `argv` is required; `tool` lets one log serve several shadowed executables, and `exit`/`ts` are recorded for reporting rather than matched. Unknown keys are ignored, so a mock may record more.
784812

785813
```json

src/coder_eval/cli_recorder.py

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
"""Source template for the CLI recording shims that ``SandboxConfig.record_cli`` generates.
2+
3+
The rendered script runs INSIDE the sandbox, where ``coder_eval`` is not
4+
installed, so it imports nothing from this package: its configuration arrives as
5+
embedded literals and everything else comes from the standard library.
6+
7+
Keeping the template here rather than inline in :mod:`coder_eval.sandbox` lets
8+
:func:`render_recorder` be exercised directly (render, execute, read the log)
9+
without standing up a sandbox.
10+
"""
11+
12+
import json
13+
14+
from coder_eval.models import RecordedCli
15+
16+
17+
# Written beside the shims, inside the generated recorder directory, so the log
18+
# travels with them if the sandbox root moves.
19+
LOG_FILENAME = "calls.jsonl"
20+
21+
_TEMPLATE = '''\
22+
#!/usr/bin/env python3
23+
"""Recording shim for `{tool}` - generated by coder_eval SandboxConfig.record_cli.
24+
25+
Appends one JSON record per invocation to {log_filename} beside this script, in
26+
the format the `cli_called` success criterion reads. Do not edit: regenerated on
27+
every sandbox setup.
28+
"""
29+
30+
import json
31+
import os
32+
import sys
33+
import time
34+
35+
TOOL = {tool!r}
36+
EXIT_CODE = {exit_code!r}
37+
STDOUT_TEXT = {stdout!r}
38+
STDERR_TEXT = {stderr!r}
39+
40+
SHIM_DIR = os.path.dirname(os.path.abspath(__file__))
41+
LOG_PATH = os.path.join(SHIM_DIR, {log_filename!r})
42+
43+
44+
def record(argv, exit_code):
45+
"""Append this invocation to the log.
46+
47+
Best-effort: a logging failure must never break the command the agent ran,
48+
which would turn an evidence problem into a behaviour problem.
49+
50+
argv is stored as a LIST. A space-joined string cannot distinguish
51+
`--flag "two words"` from two arguments, which is the whole reason this log
52+
exists instead of a flattened command line. stdin is deliberately never
53+
read: it would block whenever the sandbox leaves it on an open pipe, and in
54+
passthrough mode it would consume the payload the real tool needs.
55+
"""
56+
entry = {{
57+
"ts": round(time.time(), 3),
58+
"tool": TOOL,
59+
"argv": list(argv),
60+
"exit": exit_code,
61+
}}
62+
try:
63+
# ensure_ascii escapes non-ASCII and any stray surrogate from
64+
# undecodable argv bytes, so an exotic argument cannot make this write
65+
# raise and silently drop the record.
66+
with open(LOG_PATH, "a", encoding="utf-8", newline="\\n") as handle:
67+
handle.write(json.dumps(entry) + "\\n")
68+
except OSError:
69+
pass
70+
71+
72+
def main(argv):
73+
"""Record the invocation, then fail like the tool would with nothing behind it.
74+
75+
Nothing is executed: no network, no auth, no side effects. A test that needs
76+
the real tool's behavior recorded instead should supply its own wrapper under
77+
mock_path_dirs -- proxying a live executable is a different job from stubbing
78+
one, and this shim deliberately does only the second.
79+
"""
80+
record(argv[1:], EXIT_CODE)
81+
if STDOUT_TEXT:
82+
sys.stdout.write(STDOUT_TEXT)
83+
if STDERR_TEXT:
84+
sys.stderr.write(STDERR_TEXT)
85+
return EXIT_CODE
86+
87+
88+
if __name__ == "__main__":
89+
sys.exit(main(sys.argv))
90+
'''
91+
92+
93+
def render_recorder(spec: RecordedCli) -> str:
94+
"""Render the shim source for one ``record_cli`` entry."""
95+
return _TEMPLATE.format(
96+
tool=spec.tool,
97+
exit_code=spec.exit_code,
98+
stdout=spec.stdout,
99+
stderr=spec.stderr,
100+
log_filename=LOG_FILENAME,
101+
)
102+
103+
104+
def parse_log(text: str) -> list[dict[str, object]]:
105+
"""Parse recorder-log text into records, skipping unparseable lines.
106+
107+
Shared with tests and any caller that wants the log without duplicating the
108+
JSON-Lines handling in :mod:`coder_eval.criteria.cli_called`.
109+
"""
110+
records: list[dict[str, object]] = []
111+
for line in text.splitlines():
112+
stripped = line.strip()
113+
if not stripped:
114+
continue
115+
try:
116+
parsed = json.loads(stripped)
117+
except ValueError:
118+
continue
119+
if isinstance(parsed, dict):
120+
records.append(parsed)
121+
return records

src/coder_eval/models/__init__.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,10 +152,13 @@
152152

153153
# Sandbox
154154
from coder_eval.models.sandbox import (
155+
RECORD_CLI_DIR,
156+
RECORD_CLI_LOG,
155157
DockerBuildConfig,
156158
DockerDriverConfig,
157159
NodeEnvConfig,
158160
PythonEnvConfig,
161+
RecordedCli,
159162
ResourceLimits,
160163
SandboxConfig,
161164
validate_template_sources_list,
@@ -265,6 +268,9 @@
265268
"NodeEnvConfig",
266269
"PythonEnvConfig",
267270
"SandboxConfig",
271+
"RecordedCli",
272+
"RECORD_CLI_DIR",
273+
"RECORD_CLI_LOG",
268274
"ResourceLimits",
269275
"validate_template_sources_list",
270276
# Telemetry

src/coder_eval/models/criteria.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
from coder_eval.models.agent_config import AgentConfig, ClaudeCodeAgentConfig, parse_agent_config
1717
from coder_eval.models.enums import AgentKind
1818
from coder_eval.models.judge_defaults import DEFAULT_JUDGE_MODEL
19+
from coder_eval.models.sandbox import RECORD_CLI_LOG
1920

2021

2122
# SECURITY: ignore_patterns floor. The judge's working directory is a copy of
@@ -398,7 +399,14 @@ class CliCalledCriterion(BaseSuccessCriterion):
398399
"""
399400

400401
type: Literal["cli_called"] = "cli_called"
401-
log: str = Field(description="Path to the JSON Lines invocation log, relative to the sandbox working directory")
402+
log: str = Field(
403+
default=RECORD_CLI_LOG,
404+
description=(
405+
"Path to the JSON Lines invocation log, relative to the sandbox working directory. "
406+
f"Defaults to '{RECORD_CLI_LOG}', where SandboxConfig.record_cli writes, so a task using "
407+
"generated recorders never repeats it"
408+
),
409+
)
402410
verb: str | None = Field(
403411
default=None,
404412
description=(

src/coder_eval/models/sandbox.py

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -307,6 +307,65 @@ def _validate_working_dir(cls, v: str | None) -> str | None:
307307
return v
308308

309309

310+
# Sandbox-relative location of the generated CLI recorders and their shared log.
311+
# Not dot-prefixed on purpose: CI artifact upload (actions/upload-artifact) skips
312+
# hidden files, and the log is primary evidence for every `cli_called` criterion,
313+
# so it must survive into the run artifact.
314+
RECORD_CLI_DIR = "cli_mocks"
315+
RECORD_CLI_LOG = f"{RECORD_CLI_DIR}/calls.jsonl"
316+
317+
318+
class RecordedCli(BaseModel):
319+
"""One executable to shadow with a generated recording shim.
320+
321+
The shim records the invocation, writes the configured output, and exits —
322+
nothing is executed, so there is no network, no auth, and no side effect. Each
323+
invocation becomes a JSON Lines record in :data:`RECORD_CLI_LOG`, the log the
324+
``cli_called`` criterion reads by default, so a task asserts on what actually
325+
ran without hand-rolling a mock and without the record shape being a contract
326+
between two repositories.
327+
328+
It stubs a tool; it does not proxy one. A test that needs a REAL executable's
329+
behavior recorded on the way through still supplies its own wrapper under
330+
``mock_path_dirs`` — that depends on the tool being installed, on PATH order,
331+
and usually on live credentials, which is a different problem with different
332+
failure modes.
333+
"""
334+
335+
model_config = ConfigDict(extra="forbid")
336+
337+
tool: str = Field(description="Executable name to shadow on PATH (e.g. 'uip', 'curl', 'git')")
338+
exit_code: int = Field(
339+
default=1,
340+
description=(
341+
"Exit status the shim returns. Defaults to 1 so an unconfigured tool looks like a failing "
342+
"one rather than silently succeeding"
343+
),
344+
)
345+
stdout: str = Field(default="", description="Text the shim writes to stdout")
346+
stderr: str = Field(
347+
default="",
348+
description=(
349+
"Text the shim writes to stderr. Use it to explain the failure the way the real tool "
350+
"would, so an agent reads a plausible error rather than silence"
351+
),
352+
)
353+
354+
@field_validator("tool")
355+
@classmethod
356+
def validate_tool_name(cls, v: str) -> str:
357+
"""Reject names that are not a bare filename.
358+
359+
The shim is written as ``<RECORD_CLI_DIR>/<tool>``; a separator or a
360+
traversal segment would place it outside the managed directory.
361+
"""
362+
if not v or v != v.strip():
363+
raise ValueError("record_cli tool must be a non-empty name without surrounding whitespace")
364+
if "/" in v or "\\" in v or v in {".", ".."}:
365+
raise ValueError(f"record_cli tool {v!r} must be a bare executable name, not a path")
366+
return v
367+
368+
310369
class SandboxConfig(BaseModel):
311370
"""Configuration for the sandboxed execution environment.
312371
@@ -360,6 +419,20 @@ class SandboxConfig(BaseModel):
360419
),
361420
)
362421

422+
record_cli: list[RecordedCli] | None = MergeField(
423+
strategy="replace",
424+
default=None,
425+
description=(
426+
"Executables to shadow with a generated recording shim. The sandbox writes each shim "
427+
f"into '{RECORD_CLI_DIR}/' and PATH-prepends that directory, so the agent's calls are "
428+
f"recorded as JSON Lines in '{RECORD_CLI_LOG}' — the log a 'cli_called' criterion reads "
429+
"by default. Use instead of hand-writing a mock under mock_path_dirs when all the test "
430+
"needs is a faithful record of what ran plus a canned exit status and message. It does "
431+
"NOT serve per-invocation responses and does NOT proxy the real executable; supply your "
432+
"own mock for either. Replaced (not merged) across config layers, like mock_path_dirs."
433+
),
434+
)
435+
363436
# Customizable ignore patterns
364437
ignore_patterns: list[str] = MergeField(
365438
strategy="replace",

0 commit comments

Comments
 (0)