[burtenshaw]

Summary

Clarify the merged posttrain_env README's “drop such a rollout” wording, as flagged by the RFC 013 author in #1256 (comment).

- Drop an unscored rollout.
+ Exclude it from reward-based training and reward estimates; retain its episode and spend.
+ Under the RFC 013 proposal, pass None plus spend to the sampler's update first.

RFC 013 remains a proposal until accepted, and this integration follows its eventual accepted semantics. This docs-only clarification preserves RFC 014's contingency and the prototype's existing accounting; it adds no core behavior or trainer integration guarantee. Refresh the generated documentation copy with the same text.

Evidence

• Before: “Trainers should drop such a rollout” could be read as discarding the accounting that the same README requires retaining.
  After: Reward exclusion and accounting/update are separate, and acceptance remains explicitly pending.
• Test plan: python3 scripts/sync_env_docs.py --check, git diff --check, and changed README formatting check pass.
• PYTHONPATH=.:src:envs uv run pytest tests/core/test_harness_runtime.py tests/envs/test_posttrain_environment.py -q: 120 passed, 1 skipped. HF Jobs is faked; no jobs, GPU use or spend.
• uv run ruff check src/ tests/: passes. Full lint hook reports existing import-sort failures in two unchanged tests and formatting failures in 47 unchanged files. The standard test hook cannot start because timeout is unavailable; the tests above were run directly.
• Based on latest main aca76d95347bbbfb6017709dfa9bc666ca893489; open PR search found no duplicate.

Merge Danger

Door: two-way

Blast Radius: documentation

Only the README and its generated documentation copy change.

[cursor]

Alignment Review Report

Automated Checks

• Lint: N/A — bash .claude/hooks/lint.sh can't run in this sandbox (uv isn't installed), but the diff touches only two Markdown files, so ruff/usort over src//tests/ would not apply regardless.
• Debug code: CLEAN — bash .claude/hooks/check-debug.sh only surfaces pre-existing print/console.print calls elsewhere in src/, none introduced by this diff (which is docs-only).

Open RFCs Context

• RFC 013 ("Task sampling and in-training evaluation", PR #1256) is still open/In Review (not yet a merged file under rfcs/). This PR's wording change is the direct fix for a concern @tiagoperrupato flagged on that PR: the old "Trainers should drop such a rollout" phrasing in posttrain_env's README read as discarding the accounting the same doc requires retaining.
• RFC 014 ("Environment Services", rfcs/014-environment-services.md) is In Review. It's the RFC that actually governs posttrain_env's spend/accounting contract ("an episode that ends unscored still reports its spend"), and this change keeps the README consistent with it.
• Both referenced RFCs are correctly caveated in the new text ("RFC 013 remains a proposal until accepted; this integration is subject to its acceptance and final unscored-rollout semantics"), so the doc doesn't overstate settled behavior.

Tier 1: Fixes Required

None. No lint, debug-code, or mechanical issues found in the diff.

Tier 2: Alignment Discussion

Principle Conflicts

None identified. The change only clarifies doc wording about trainer-side accounting of None-reward rollouts; it doesn't move reward computation out of the environment ("rewards in environment" principle intact — the environment still decides reward=None/score_failed; the trainer only records spend/accounting for a rollout it didn't score) and doesn't touch the Gym-like/MCP boundary or client-server separation.

RFC Conflicts

None identified. The change doesn't pre-empt or implement RFC 013 differently from the proposal — it explicitly defers to "final unscored-rollout semantics" pending acceptance, and it's consistent with RFC 014's already-documented spend-accounting contract. If anything, this is the kind of RFC-driven doc correction the review process is meant to produce: a reviewer on the RFC's own PR found a documentation inconsistency and it's being resolved here. scripts/sync_env_docs.py --check also confirms the generated docs/source/environments/posttrain.md copy stays in sync with envs/posttrain_env/README.md (remaining diffs are only the expected link-rewriting the sync script performs).

Summary

• 0 mechanical issues to fix
• 0 alignment points for human review
• 0 RFC conflicts to discuss

This is a clean, well-scoped documentation clarification — the wording now correctly separates "exclude from reward-based training/estimates" from "retain episode and spend for accounting," and keeps the RFC 013 dependency explicitly conditional on acceptance.

  

_{Sent by Cursor Automation: Pre-review}