Skip to content

Commit ba85374

Browse files
cyyehclaude
andauthored
Let the sidebar hand the agent loop to pi, which can ask the page's own model (#8)
* Let the sidebar hand the agent loop to pi, behind one Harness interface What runs the loop is now a choice, like what runs the model: the built-in harness, or pi, a coding agent that asks a model it has credentials for (Claude, GPT…). The page draws pi's steps in the same diagram, step cards and event list, from the events pi reports, without tokens or a prompt. Harness - harness/harnesses/harness.py: Harness, what the page asks of whatever runs the loop (run, stop, decide), and which events every harness sends and which only one that knows them. - harness/agent.py moves to harness/harnesses/simple_harness.py, and Run becomes SimpleHarness, beside SimpleBackend; run_start says which harness ran the turn ("simple" or "pi"). - LiveRun takes a function that makes the harness (harness_for()). pi - harness/harnesses/pi_harness.py: PiHarness starts pi in RPC mode for each question, in the conversation's folder and on its session, without the user's extensions, skills or context files; it sets the model and the thinking level and turns pi's retries off before the question goes, and turns pi's records into the page's events. One of pi's turns is a step; its streamed text is a new event, delta. pi has no step limit, so the page aborts it past Max steps. - harness/harnesses/pi_approve.ts: a pi extension that asks before every tool call; its dialog becomes the page's Allow and Reject, and a refusal reaches the model in the turn's language. - The sidebar: What runs the loop, then pi's models (pi.probe), its tools, its thinking level, Ask me before running a tool and Max steps. The Context tab says pi shows no prompt; the Model tab shows pi's model, settings and tools, which run on this computer without a sandbox. - render.py draws a run without a family, a context or tokens: the streamed text by channel, the provider's token counts, and pi's own wording for its stops and its diagram. Tests and docs - tests/fake_pi.py speaks pi 1.4's RPC protocol from a script, so the default suite needs neither Node nor a model; test_pi_harness.py covers PiHarness, and with RUN_PI_TESTS=1 runs the real pi against a scripted model on 127.0.0.1 (passes with pi 1.4.0). test_render.py and test_app.py cover pi's runs and the page with pi picked. - How it works gets a section on pi and the delta event; the READMEs, AGENTS.md and the guide's excerpts follow the move and the rename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Let pi ask the page's own model and see every token, and show the request pi sends With pi picked under What runs the loop, The model pi asks now picks either this page's model (the default) or one of pi's own. On the page's model, Gemma 4 E2B or Qwen3.5 0.8B on transformers or vLLM, pi's steps show the string the model read and every token, as the built-in harness's do; on pi's own models, each step's context section shows the request pi sent. The page's model for pi - harness/harnesses/model_endpoint.py serves the model in memory as an OpenAI-compatible chat completions endpoint on 127.0.0.1, behind a token, only while a turn is served. Each request becomes the conversation SimpleHarness keeps, is rendered with the chat template, encoded and generated; its context, prefill, token and model_end events go into pi's step, and the reply streams back to pi (thinking as reasoning_content, the tool calls the family parses as tool_calls). - A request waits for the step pi opened for it, since pi's records and its HTTP request reach the page by different ways. - The sidebar keeps the model menu, What runs the model and the decoding settings for it; Thinking mode reaches the model as pi's thinking level. The request pi sends - pi_approve.ts becomes pi_extension.ts, loaded into every pi: it reports each request pi sends the model (before_provider_request) as a notification, registers the page's model as the provider agent-loop, and, with Ask me before running a tool, asks before each call. - A new event, request: the request as JSON, image bytes left out, with what the step added marked. It is the context section of a step on a provider's model, and folded under the context on the page's model. Screenshots and docs - docs/screenshot-en.png and docs/screenshot-zh.png are retaken from a real Gemma run (on MPS), with What runs the loop at the top of the sidebar. - How it works: the pi section in tabs, one per kind of model pi asks, with the endpoint's code; the request event in the event table. The READMEs and AGENTS.md follow. Tests - test_model_endpoint.py asks the endpoint over HTTP with FakeLLM behind it; fake_pi.py asks it as pi does, so test_pi_harness.py, test_render.py and test_app.py cover pi on the page's model in the default suite. - With RUN_PI_TESTS=1 the real pi 1.4.0 asks the endpoint too, and with RUN_MODEL_TESTS=1 it asks real Gemma 4 E2B and Qwen3.5 0.8B. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Tell the built-in harness and pi apart in How it works with tabs How it works now treats what runs the loop as it treats what runs the model and the model: a part of a section that differs by harness is told once for each, in tabs, and picking one switches every set on the page. - A new tab kind, What runs the loop: Built-in harness (SimpleHarness) and pi (PiHarness), with pi-page and pi-provider as narrower kinds of pi, as vllm-docker is of vllm, where pi asking this page's model and pi asking one of its own differ. guide.js already syncs narrower kinds. - The overview, sending a question, preparing a run, context, model, parse, stop, writing the call back, checks, environment, observation and the page each get harness tabs, with pi's code in its tabs. The separate section on pi is spread into them, so the guide has 18 sections again. - Tabs do not nest, so what the built-in harness and pi on this page's model share (the chat template, the parsers, the backends) stays outside the harness tabs, saying it is for both. - The READMEs and AGENTS.md say the guide's tabs cover the harness too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Free the page's model as soon as pi is aborted, and say what shows of the model pi asks Review of #8. Aborting pi: - `_abort()` now wakes a request that waits for its step (`_serving_step`). pi's request for the turn past Max steps can reach the endpoint before the page has read that turn's start; it then waited out WAIT_FOR_STEP (10 s) holding the endpoint's lock, so the next question's first request waited too. - The endpoint turns away a request that belongs to no step once the turn is being stopped, instead of rendering, encoding and prefilling it for a pi that has given up on it. Text that predated pi asking the page's model: - The intro on the Loop tab now depends on the model pi asks: on this page's model it says every token and the string the model reads show; on pi's own models, that the Context tab holds pi's request and that no probabilities show. - What runs the loop's help, page.py's docstring, pi's model card and its tools card no longer say the prompt and the tools' schemas stay out of view. Tests: - pi on the page's model at the step limit, with the request arriving before the page reads the turn's start: the turn ends at once, the request is turned away and the endpoint is free; with RUN_PI_TESTS=1, the same with the real pi, asking first. - Ask me before running a tool on the page's model: Allow and Reject, and what the model reads of each. - The intro says what shows for each model pi asks. - FakePi ends a message in an HTTP error, as pi's provider does. How it works §18 says the abort wakes the waiting request; tests/guide_excerpts.txt was rewritten after rereading it. Both READMEs' test tables name the new tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rename the harness package to agent, with model_backends and model_families The project's two parts are now app/ and agent/. The agent is everything that runs a turn: the harness that runs its loop, the model's family, what runs the model, and the tools. The harness is one part of it, beside the model, so the package that holds all of them is no longer named after it. - harness/ is agent/, harness/backends/ is agent/model_backends/ and harness/families/ is agent/model_families/; harnesses/, tools.py, media.py, i18n.py and sandbox/ keep their names. Imports follow (`from agent import model_families`, `from agent.model_backends.backend import Backend`), and so does every `families.X` (`model_families.REVISIONS`, `model_families.VLLM_SPECULATIVE`). - tests/test_families.py is tests/test_model_families.py. - The sandbox builds with `docker build -t agent-loop-sandbox agent/sandbox`, which the sidebar now says too. Only a comment in its Dockerfile changed, so an image built before still works and SANDBOX_VERSION stays. - AGENTS.md, both READMEs, How it works and the package docstrings describe the two parts as app and agent ("The agent never knows the app"). All the guide's code references point at the new paths. In tests/guide_excerpts.txt every changed line differs only in a path or module name, which the guide's text was reread and fixed for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check every push with ruff, mypy and the tests, and test that the agent never imports the app The rename to agent/ left 22 files' imports out of order (agent.model_* now sorts before agent.harnesses), nothing checked the rule that agent/ never imports app/ or Streamlit, and there was no linter or type checker at all. Now: Lint - Ruff, set up in pyproject.toml: pyflakes and pycodestyle's errors, import order, bugbear and pyupgrade. No formatter, so it never rewrites a line it has nothing to say about; zip()'s strict= and named lambdas are left off, with the reason beside them. - Imports sorted everywhere (tests/'s helpers sort with agent and app, as src), three unused imports gone, Callable and Iterator from collections.abc, two loop variables marked unused. - An f-string in test_render.py reused its quotes, which only Python 3.12 parses, though requires-python says 3.11. Types - mypy, at its default strictness, with the bodies of agent/'s and app/'s unannotated functions checked too (SimpleHarness.run, the page); the tests only where annotated. Stubs for PyYAML, Pygments and openpyxl in the dev group. - No behavior changes. SimpleBackend, VLLMBackend and FakeLLM were not Backends to a type checker: each typed its drafter (and VLLMBackend its drafter_id) as None alone. Encoded.ids is Any, the form each backend's generate() takes; pick() is generic, as the guide already called it with tuples. The rest is annotations on empty dicts and lists, Sequence where a default is (), asserts where a value is known not to be None (a tool call without errors has a tool, generate() ends with done, Popen's pipes), PiHarness.pi for the pi run() started, and format_func=names.__getitem__, which returns a str where dict.get may not. CI - .github/workflows/checks.yml runs ruff check, mypy and uv run pytest on Ubuntu on every push and pull request, each even when the one before failed. The agent never knows the app - tests/test_layout.py reads every file of agent/ for an import of app or streamlit, at the top or inside a function, and checks that the check itself sees each way of importing. Also - .gitignore: .mypy_cache/, .ruff_cache/ and .impeccable/. - .nvmrc picks Node 22 for pi with nvm (`nvm use`), which the READMEs' pi section and AGENTS.md now say; nvm keeps global packages per version, so pi is installed under it. - The READMEs tell a clone from before the rename that the empty-looking harness/ it keeps is only the __pycache__ folders git ignores, which `git clean -fdX harness` removes. - AGENTS.md and both READMEs list the commands, the workflow and the new test. - How it works: the guide's text was reread for every changed excerpt (annotations, a renamed local, the drafting condition now `mtp is not None`); none of it became untrue, and tests/guide_excerpts.txt is rewritten. One reference follows the line it points at. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Catch a heavy import in How it works on the pull request, and check each push once - checks.yml has a guide job that runs what pages.yml runs before it publishes How it works: tests/test_guide.py and the build, with pygments and pytest alone. The check job installs every package, so a pull request that had app/guide.py, app/render.py, agent/i18n.py or agent/model_families/ import torch, transformers, streamlit or httpx passed it, and only the Pages build on main failed, after the merge. - Both workflows run those commands with --isolated. Without it, uv reuses a .venv that already has pygments, and with it the whole app, so the same commands run by hand in a synced checkout passed with `import httpx` in agent/i18n.py. A fresh checkout in CI has no .venv; the flag makes a run by hand mean the same. AGENTS.md lists the command. - checks.yml runs on pushes to main only, besides pull requests. A push to a pull request's branch ran every job twice, once per event (the concurrency group differs by the event, so neither cancelled the other), and installed torch twice. A push to a branch without a pull request is no longer checked. - The READMEs say `nvm install` instead of `nvm use`: .nvmrc names 22, and nvm use picks the newest 22 already installed, which can be older than the 22.19 pi 1.4 needs; nvm install installs the newest 22 and switches to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 3279b0e commit ba85374

64 files changed

Lines changed: 5292 additions & 1110 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/checks.yml‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Checks every pull request and every push to main: ruff's lint (the order of imports with it), mypy's types, and the tests that
2+
# load no weights and touch no network (the ones that need the models, Docker's images, a vLLM server or pi skip).
3+
# mypy and the tests import the whole app, so the job first installs every package uv.lock pins, torch among them.
4+
# A second job builds How it works with pygments alone, as pages.yml does after the merge.
5+
name: Checks
6+
7+
# A push to a pull request's branch is checked once, as the pull request: on a push to any branch as well, both ran,
8+
# neither cancelled the other (their concurrency groups differ by the event), and torch was installed twice.
9+
on:
10+
push:
11+
branches: [main]
12+
pull_request:
13+
14+
permissions:
15+
contents: read
16+
17+
concurrency:
18+
group: checks-${{ github.event_name }}-${{ github.ref }}
19+
cancel-in-progress: true
20+
21+
jobs:
22+
check:
23+
runs-on: ubuntu-latest
24+
timeout-minutes: 30
25+
steps:
26+
- uses: actions/checkout@v7
27+
- uses: astral-sh/setup-uv@v10.3.0
28+
with:
29+
version-file: pyproject.toml # its required-version: no uv older than the fix for the revoked emSign certificates
30+
- id: sync
31+
run: uv sync --locked
32+
# Each runs even when the one before failed, so that one push reports all three.
33+
- run: uv run ruff check
34+
- if: ${{ !cancelled() && steps.sync.outcome == 'success' }}
35+
run: uv run mypy
36+
- if: ${{ !cancelled() && steps.sync.outcome == 'success' }}
37+
run: uv run pytest
38+
39+
# What pages.yml runs before it publishes How it works, word for word: keep the two the same. The job above has
40+
# every package installed, so it would not notice app/guide.py, app/render.py, agent/i18n.py, agent/model_families/
41+
# or a package's __init__.py importing torch, transformers, streamlit or httpx; only the Pages build on main would.
42+
# --isolated keeps uv from reusing a .venv that already has pygments, and with it the whole app: there is none in a
43+
# fresh checkout, but there is in a checkout where these are run by hand.
44+
guide:
45+
runs-on: ubuntu-latest
46+
timeout-minutes: 10
47+
steps:
48+
- uses: actions/checkout@v7
49+
- uses: astral-sh/setup-uv@v10.3.0
50+
with:
51+
version-file: pyproject.toml # its required-version: no uv older than the fix for the revoked emSign certificates
52+
- run: uv run --isolated --no-project --with pygments --with pytest pytest tests/test_guide.py
53+
# Runs even when the tests failed, so that one push reports both.
54+
- if: ${{ !cancelled() }}
55+
run: uv run --isolated --no-project --with pygments python -m app.guide _site

‎.github/workflows/pages.yml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,9 @@ jobs:
3232
# The page needs only pygments, not torch and the rest of the app, and its tests only pytest
3333
# besides. They fail when an edit moved or removed a line the page points at, or changed code it
3434
# quotes since its text was last reread (tests/guide_excerpts.txt); the page already up then stays.
35-
- run: uv run --no-project --with pygments --with pytest pytest tests/test_guide.py
36-
- run: uv run --no-project --with pygments python -m app.guide _site
35+
# checks.yml's guide job runs these two on every pull request, so keep the two the same.
36+
- run: uv run --isolated --no-project --with pygments --with pytest pytest tests/test_guide.py
37+
- run: uv run --isolated --no-project --with pygments python -m app.guide _site
3738
- uses: actions/configure-pages@v6
3839
- uses: actions/upload-pages-artifact@v5
3940
with:

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
.venv/
22
__pycache__/
33
.pytest_cache/
4+
.mypy_cache/
5+
.ruff_cache/
6+
.impeccable/
47
_site/
58
/models/
69
.DS_Store

‎.nvmrc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
22

‎AGENTS.md‎

Lines changed: 66 additions & 29 deletions
Large diffs are not rendered by default.

‎README.md‎

Lines changed: 74 additions & 31 deletions
Large diffs are not rendered by default.

‎README.zh-TW.md‎

Lines changed: 74 additions & 31 deletions
Large diffs are not rendered by default.

‎agent/__init__.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
"""The agent: everything that runs a turn, and nothing a person sees.
2+
3+
harnesses/ is what runs the loop, behind one interface: simple_harness.py is this project's own loop (SimpleHarness):
4+
context, model, parse, tool, observation, stop, with one event per stage for the app to draw; pi_harness.py hands the
5+
loop to pi, a coding agent, and turns what pi reports into the same events. tools.py has the tools SimpleHarness runs,
6+
with run_python's Docker sandbox built from sandbox/. media.py turns the files attached to a question into what the
7+
model reads. model_families/ is everything model-specific, and model_backends/ is what runs the model, behind one interface.
8+
i18n.py has the page's two languages, which the errors the model reads use too.
9+
10+
Nothing here imports Streamlit, and importing the package loads nothing heavy: the app's guide imports model_families
11+
and i18n with pygments alone.
12+
"""

‎agent/harnesses/__init__.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
"""What runs the agent loop, behind the one interface the app knows.
2+
3+
harness.py is that interface (Harness) and the events every harness sends. simple_harness.py is this project's own
4+
harness (SimpleHarness), which runs the model through a Backend (model_backends/); pi_harness.py hands the loop to pi, a
5+
coding agent that runs its own (PiHarness). pi_extension.ts is the pi extension through which the page sees pi's
6+
requests and asks before pi runs a tool, and model_endpoint.py serves the page's own model to pi over HTTP.
7+
"""

‎agent/harnesses/harness.py‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
"""The interface between the page and whatever runs the agent loop.
2+
3+
The page (app/page.py) runs one turn at a time and reaches the loop only through a Harness: it makes one with a
4+
function that appends each event to the turn's list, calls run() in a background thread, and passes the user's
5+
clicks on with stop() and decide(). render.py draws the list. This project's own harness is
6+
simple_harness.SimpleHarness, which runs the model through a Backend (model_backends/backend.py), so it knows every token;
7+
pi_harness.PiHarness hands the loop to pi, a coding agent that runs its own and says what it does as JSON. Another
8+
agent that reports its loop as events, such as the Claude Agent SDK or Codex, would be one more; this module says
9+
what it has to send.
10+
11+
A harness that runs the model through a Backend knows the prompt the model reads and each token it writes; one that
12+
asks a model's API sees only the text that comes back. So, like a token event's details (backend.TOKEN_DETAILS), most
13+
events are sent only by a harness that knows what they say, and render.py draws a step from whichever it has: the page
14+
never shows a prompt or a number the harness did not see.
15+
16+
Nothing here imports anything, so a harness can be written, and the interface read, without the rest.
17+
"""
18+
19+
from typing import Protocol, runtime_checkable
20+
21+
# The events every harness sends: run_start first, a step_start each time around the loop, and run_end last (after
22+
# an error event when something failed). run_start carries which harness ran the turn (`harness`: "simple", "pi"), the
23+
# model, its family (a key of model_families.FAMILIES, or None when the harness does not know how the model writes its turns),
24+
# the turn's settings and the tools the model gets; run_end the stop reason, the final reply and the conversation.
25+
ALWAYS = ("run_start", "step_start", "run_end")
26+
# The events a harness sends in a step when it knows what they say, in the order they come:
27+
# request the request sent to the model's API, as JSON, from a harness that asks one (pi)
28+
# context the prompt as the model reads it, from a harness that renders it (SimpleHarness, or pi's model endpoint)
29+
# prefill, token the prompt read, then each token (backend.TOKEN_FIELDS), from a harness that runs the model itself
30+
# delta a piece of the model's output as it streams, with its channel, from one that sees text, not tokens (pi)
31+
# model_end the model's turn ended: why, and how many tokens, with their speed when it was measured
32+
# parsed what the harness read in it: the thinking, the reply, the tool calls
33+
# tool_call a call it will run (or reports as failed), with its environment
34+
# approval_request, approval the call held for the user, and the user's answer
35+
# tool_start, tool_result the call running in its environment, and what came back
36+
# error what went wrong, before run_end
37+
SOMETIMES = ("request", "context", "prefill", "token", "delta", "model_end", "parsed", "tool_call", "approval_request", "approval",
38+
"tool_start", "tool_result", "error")
39+
40+
41+
@runtime_checkable
42+
class Harness(Protocol):
43+
"""What runs one turn. The page makes one per question (app.page.LiveRun) and runs one at a time."""
44+
45+
id: str
46+
47+
def run(self) -> None:
48+
"""Run the turn to its end, sending its events. It returns after run_end and never raises: a failure is an
49+
error event followed by run_end."""
50+
51+
def stop(self) -> None:
52+
"""Stop the turn as soon as it can: the user pressed Stop. It is called from another thread, and run() still
53+
ends with run_end."""
54+
55+
def decide(self, call_id: str, approved: bool) -> None:
56+
"""The user's answer to the approval_request for the call `call_id`. It is called from another thread."""

0 commit comments

Comments
 (0)