Skip to content

feat(tools): add per-call llm_profile override to the task tool - #4510

Open
georgeglarson wants to merge 9 commits into
OpenHands:mainfrom
georgeglarson:feat/task-tool-llm-profile
Open

feat(tools): add per-call llm_profile override to the task tool#4510
georgeglarson wants to merge 9 commits into
OpenHands:mainfrom
georgeglarson:feat/task-tool-llm-profile

Conversation

@georgeglarson

@georgeglarson georgeglarson commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

HUMAN:

Enforced sub-agent delegation is the objective here.


AGENT:

Why

A parent agent cannot currently choose a different saved LLM profile for one delegated task. It must either switch its own model or use a subagent definition with a fixed model: value.

That makes mixed-model delegation awkward. The parent should be able to keep its model while assigning an individual task to a cheaper, faster, or specialized worker.

Summary

  • Add an optional llm_profile field to the task tool and advertise saved profile names so a parent can select a worker model per call.
  • Use one cipher-aware profile-loading primitive while preserving definition model, per-call profile, and parent-inheritance precedence across creation and resume.
  • Keep the worker model, context, and metrics isolated from the parent; subscription-backed workers retain the factory-created condenser supported by subscription completion dispatch.

The selection order is:

  1. A model profile specified by the subagent definition.
  2. The task call's llm_profile.
  3. The parent model.

A bare resume keeps the task's effective profile. Passing a different profile replaces it for that resume and later resumes. If the resumed subagent definition supplies its own model, the ignored per-call profile is cleared.

Unknown profiles return an error before task state is registered. There is no silent fallback to the parent model.

Issue Number

Fixes #4654.

How to Test

uv run pytest -q \
  tests/tools/task \
  tests/sdk/subagent \
  tests/sdk/tool/test_switch_llm.py \
  tests/sdk/conversation/test_switch_model.py

Result on the current branch:

235 passed, 142 warnings in 2.38s

All touched-file pre-commit hooks pass, including Ruff, Pyright, import dependency checks, and tool registration checks.

I also ran the delegation path against real saved profiles. The parent used MiniMax M3 and delegated a task through an encrypted MiMo v2.5 Pro profile:

task_status=completed
task_result=encrypted worker succeeded
parent_model_before=openai/MiniMax-M3
parent_model_after=openai/MiniMax-M3
worker_model=openai/mimo-v2.5-pro
parent_registry_ids=[]
parent_metric_keys=['task:task_00000001']
resume_profile=encrypted-worker
resume_worker_model=openai/mimo-v2.5-pro

The persisted profile did not contain the plaintext API key.

Video/Screenshots

Not applicable. This change affects SDK task-tool behavior and is covered by the live output above.

Design Doc

Per-call llm_profile design

Companion documentation: OpenHands/docs#761

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

The task tool description includes saved profile names from the default profile store so the parent model can select one. It does not include profile configuration, model IDs, provider URLs, or credentials. Profile names remain visible when enable_switch_llm_tool=False; that setting removes the conversation-level switch tool but does not disable task-level worker selection.

A file-defined subagent may use a custom profile_store_dir. The task resolves profiles from that directory, but the advertised list still comes from the default store. Valid custom-store names may therefore be absent from the list.

Once a task stores a per-call profile, resume can replace it with another profile but cannot explicitly return it to parent inheritance. A definition-owned model clears the stored override. Restoring task IDs across a new process remains outside the current in-memory TaskManager contract.

Older SDK versions reject events containing the new field because action schemas forbid unknown fields. Mixed-version deployments should upgrade the agent server before emitting llm_profile.

Subscription-backed workers retain their factory-created condenser, matching current top-level agent creation and profile switching. Definition-level model: loading still has its pre-existing encrypted-profile limitation; the new per-call path is cipher-aware.

@github-actions

Copy link
Copy Markdown
Contributor

📁 PR Artifacts Notice

This PR contains a .pr/ directory with temporary PR-specific documents. Because this is a fork PR, the directory will be automatically removed from main immediately after merge.

@georgeglarson
georgeglarson marked this pull request as draft August 16, 2026 20:53
@georgeglarson
georgeglarson force-pushed the feat/task-tool-llm-profile branch from 1735617 to 60752f7 Compare August 16, 2026 21:13
Comment thread openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py
Comment thread openhands-sdk/openhands/sdk/subagent/AGENTS.md Outdated
Comment thread openhands-tools/openhands/tools/task/definition.py Outdated
Comment thread openhands-tools/openhands/tools/task/definition.py
georgeglarson and others added 4 commits August 26, 2026 06:56
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
@georgeglarson
georgeglarson marked this pull request as ready for review August 26, 2026 19:43
@all-hands-bot

Copy link
Copy Markdown
Collaborator

🚦 CI is currently failing on this PR's latest commit.

Please fix the failing checks before OpenHands reviews it - this is re-checked automatically once you push a new commit. (A maintainer can also request @all-hands-bot as a reviewer to have it reviewed regardless of CI status.)

This is an automated check - no AI was used to generate this comment.

@all-hands-bot

Copy link
Copy Markdown
Collaborator

🤖 OpenHands is reviewing this PR.

Head commit: d99d9f08b58142b1d341e61a8c97524e41d0cdba
View the conversation: https://oss-agent-canvas.ngrok.dev/conversations/4c732d30-00b2-4238-b9ee-d16fda989ec8

This comment was posted by an AI agent (OpenHands).

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review was created by an AI agent (OpenHands) on behalf of the repository maintainers.

Verdict

No material bugs or security issues found. The change is well-scoped and correctly implements the documented precedence (definition model → per-call llm_profile → parent model), reuses a single cipher-aware loading primitive (local_conversation.load_profile_llm) instead of duplicating store/decryption logic, preserves worker metrics and condenser isolation, and has solid test coverage. I ran the focused suite locally (tests/tools/task/test_task_manager.py -k "llm_profile or profile") plus the conversation/task tool-set tests — all passing — and CI at the head SHA is green across pre-commit, pyright, and the full test matrix.

Eval-risk flag (blocking approval)

This PR changes the task tool's description/prompt template (TASK_TOOL_DESCRIPTION) and adds new delegation behavior (llm_profile) — the "agent behavior that could plausibly affect benchmark/evaluation performance" category. There is no openhands-eval-monitor.vercel.app link and no human confirmation of benchmark results in the PR description or comments, so per repo policy I'm leaving this as COMMENT rather than APPROVE. A human maintainer should decide after running lightweight evals on the delegation path.

Risk assessment

🟡 MEDIUM — behavior-affecting tool/prompt change with no eval evidence; the implementation itself is low-risk, well-factored, and thoroughly tested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Select an LLM profile for an individual task-tool worker

3 participants