Skip to content

Commit b85f361

Browse files
kevin9327claudedavidmckayv
authored
Name the Bot on a hop that was refused, retried or lost, not only on one that landed (#306)
The Audit screen's Bot column reads `payload.bot` and nothing else, and renders a dash without it. `agent.handoff_offered` says so in its own comment, added when the accepted pair was fixed: without this the two handoff rows are the only Bot actions on a screen headed "Every action a Bot took" that name no Bot. The pair got the key. The four rows either side of it did not: `agent.handoff_refused`, both `agent.handoff_retried` rows, and `agent.handoff_failed` carry `from` and no `bot`, so every one of them draws a dash. Those are the rows the trail exists for. `server/src/audit.ts` says it outright of the first one — "a hop that happened is visible in the transcript anyway; a hop that was refused is invisible everywhere else" — and the same holds for one that ran out of attempts, which is where a person's unanswered question ends. So the screen named the Bot on the outcome you could already see, and drew a dash on the three you could not. `agent.escalated` and `agent.escalation_failed` one file over share a single payload and get this right on both outcomes, which is the shape being restored here. Pairs with #302, which stops that same screen calling `agent.handoff_refused` "Allowed". Together a refused hop reads as a refusal and says whose it was; either alone leaves half the row wrong. Co-authored-by: kevin9327 <kevin9327@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: David McKay <davidmckayv@users.noreply.github.com>
1 parent 698d52e commit b85f361

5 files changed

Lines changed: 106 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,14 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.
88

99
## Unreleased
1010

11+
### A hop the boundary refused now names the Bot that was refused
12+
13+
The audit page renders its Bot column from `payload.bot` and nothing else. `agent.handoff_offered`
14+
and `agent.handoff_delivered` were given that key; the four rows either side of them — a hop
15+
refused, a hop retried, and a hop that failed for good — were not, so they showed a dash where the
16+
Bot belongs. Those are the rows somebody actually opens the trail for: a hop that happened is visible
17+
in the transcript anyway, and a refused or lost one is visible nowhere else. All four now name the
18+
asking Bot, exactly as the accepted pair and `agent.escalated` already did.
1119
### A failed tool refresh no longer leaves a connector offering nothing
1220

1321
Refreshing a connector's tools replaced the list with a delete and then an insert, as two separate

server/src/agents/handoff-runner.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -300,6 +300,9 @@ export function createHandoffRunner(options: {
300300
targetId: work.toBotId,
301301
...(work.actorId ? { actorUserId: work.actorId } : {}),
302302
payload: {
303+
// See the same key on `agent.handoff_delivered` below: the Audit screen's Bot
304+
// column reads `payload.bot`, so a row without it names no Bot.
305+
bot: work.fromBotId,
303306
from: work.fromBotId,
304307
to: work.toBotId,
305308
run: work.runId,
@@ -374,6 +377,8 @@ export function createHandoffRunner(options: {
374377
targetId: work.toBotId,
375378
...(work.actorId ? { actorUserId: work.actorId } : {}),
376379
payload: {
380+
// See the same key on `agent.handoff_delivered` below.
381+
bot: work.fromBotId,
377382
from: work.fromBotId,
378383
to: work.toBotId,
379384
run: work.runId,
@@ -456,6 +461,10 @@ export function createHandoffRunner(options: {
456461
targetId: work.toBotId,
457462
...(work.actorId ? { actorUserId: work.actorId } : {}),
458463
payload: {
464+
// See the same key on `agent.handoff_delivered` above. This row is the one a
465+
// person's unanswered question ends on, so a Bot column showing a dash on it is
466+
// the worst place in the set to have one.
467+
bot: work.fromBotId,
459468
from: work.fromBotId,
460469
to: work.toBotId,
461470
run: work.runId,

server/src/agents/handoff.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,12 @@ export function createHandoffDesk(options: {
110110
targetId: from.botId,
111111
...(from.actorId ? { actorUserId: from.actorId } : {}),
112112
payload: {
113+
// The same key `agent.handoff_offered` sets below, and for the same reason: the Audit
114+
// screen renders `payload.bot` and nothing else in its Bot column, so a row without it
115+
// shows a dash. The accepted row was given this and its refusal was not, which left the
116+
// refusal — the one the trail says matters more, because a hop that happened is visible in
117+
// the transcript and a refused one is invisible everywhere else — naming no Bot at all.
118+
bot: from.botId,
113119
from: from.botId,
114120
// As the model named it, capped: untrusted input, kept because "who did it reach for" is the
115121
// useful half of the question.

server/tests/agent-handoff-runner.test.ts

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,40 @@ describe("delivering a hop", () => {
199199
});
200200
});
201201

202+
/*
203+
* And so do the rows either side of it, which is the half the assertion above did not reach.
204+
*
205+
* A delivery is the outcome that is also visible in the transcript. A hop that was retried or
206+
* that failed is visible nowhere else at all, so those are the rows somebody actually comes to
207+
* this screen for — and they were the ones rendering a dash where the Bot's name belongs.
208+
*/
209+
test("a hop that was retried or that failed names the Bot too", async () => {
210+
const retried = runner({
211+
claimed: [
212+
{ kind: "bot.message", key: "run-1:abc", payload: WORK, attempts: 2 },
213+
],
214+
});
215+
await retried.runner.sweep();
216+
217+
expect(
218+
retried.written.find(
219+
(event) => event.eventType === "agent.handoff_retried",
220+
)?.payload,
221+
).toMatchObject({ bot: WORK.fromBotId, from: WORK.fromBotId });
222+
223+
const failed = runner({
224+
deliver: async () => {
225+
throw new Error("the gateway was unreachable");
226+
},
227+
});
228+
await failed.runner.sweep();
229+
230+
expect(
231+
failed.written.find((event) => event.eventType === "agent.handoff_failed")
232+
?.payload,
233+
).toMatchObject({ bot: WORK.fromBotId, from: WORK.fromBotId });
234+
});
235+
202236
/* Releasing an unusable row would put it back on the queue for ever. */
203237
test("a row that is not a hop is finished rather than released", async () => {
204238
const { runner: sweep, calls } = runner({

server/tests/agent-handoff.test.ts

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -338,8 +338,57 @@ describe("handing work to another Bot", () => {
338338
expect(refused.events[0]?.payload).toMatchObject({
339339
reason: "not_granted",
340340
run: "run-1",
341+
/*
342+
* And the refusal names the Bot too, which is the half this was missing.
343+
*
344+
* The accepted row above was given `bot` and its refusal was not, so the pair the trail calls
345+
* "both outcomes" rendered one Bot and one dash. On the row the notes call the more important
346+
* of the two: a hop that happened is visible in the transcript, and a refused one is
347+
* invisible everywhere except here.
348+
*/
349+
bot: "assistant",
341350
});
342351
});
352+
353+
/*
354+
* Every way a hop can be refused, not only the one the pair above happens to use.
355+
*
356+
* `refuse` is one function and all five reasons go through it, so this could not drift per reason
357+
* — but that is the argument for asserting it once across all of them rather than trusting it.
358+
*/
359+
test("every refusal names the Bot that was refused", async () => {
360+
const cases: Array<[string, ReturnType<typeof desk>]> = [
361+
["no_task", desk()],
362+
["not_granted", desk({ granted: false })],
363+
["unknown_bot", desk()],
364+
["depth", desk({ caps: { maxDepth: 0, maxPerRun: 3 } })],
365+
["fan_out", desk({ caps: { maxDepth: 2, maxPerRun: 0 } })],
366+
];
367+
const envelopes: Record<string, { target: string; task: string }> = {
368+
no_task: { target: "researcher", task: "" },
369+
not_granted: { target: "researcher", task: "t" },
370+
unknown_bot: { target: "nobody-by-that-name", task: "t" },
371+
depth: { target: "researcher", task: "t" },
372+
fan_out: { target: "researcher", task: "t" },
373+
};
374+
375+
for (const [name, harness] of cases) {
376+
const envelope = envelopes[name] as { target: string; task: string };
377+
const outcome = await harness.desk.send({
378+
from: FROM,
379+
target: envelope.target,
380+
envelope: { task: envelope.task },
381+
});
382+
383+
expect(outcome.ok).toBe(false);
384+
expect(harness.events.map((event) => event.eventType)).toEqual([
385+
"agent.handoff_refused",
386+
]);
387+
// The asking Bot, the same one `agent.handoff_offered` records, so the two rows of a pair
388+
// read as one Bot's two possible outcomes rather than as one Bot and a dash.
389+
expect(harness.events[0]?.payload).toMatchObject({ bot: "assistant" });
390+
}
391+
});
343392
});
344393

345394
/*

0 commit comments

Comments
 (0)