Skip to content

fix(client): do not leak a step's input: mapping into later calls of the same tool - #585

Open
goranai wants to merge 1 commit into
OpenBMB:mainfrom
goranai:fix/input-mapping-leak
Open

goranai wants to merge 1 commit into
OpenBMB:mainfrom
goranai:fix/input-mapping-leak

Conversation

@goranai

@goranai goranai commented Nov 17, 2026

Copy link
Copy Markdown

Opened together with #584 so the change can be discussed on the actual diff, as docs/CONTRIBUTING.md asks for discussion before a change. Happy to adjust it or close it if you'd rather fix it another way.

Summary

UltraData.get_data merges a step's input: mapping into self.io[concated]["input"] in place. self.io is keyed by tool, not by step and is built once in _extract_io, so the mapping outlives its step: every later call of the same tool without an input: block of its own silently receives the earlier step's variable instead of the variable the tool declares. _extract_io already merges the override into a copy before validating it; the runtime line contradicted that.

Merge into a copy, one line:

-        input_items = self.io[concated]["input"]
+        input_items = dict(self.io[concated]["input"])

Closes #584: a step's mapping no longer affects any later step.

Reproduction

End to end, with ultrarag build pipeline.yaml --no-mcp then ultrarag run pipeline.yaml --no-mcp on the three-file fixture in the linked issue: gen emits q_ls = ["Ada"]; echo.answer declares prompt_ls with default ["Default", "Value"] and logs what it receives; pipeline gen.emit, {"echo.answer": {"input": {"prompt_ls": "q_ls"}}}, echo.answer. Both commands exit 0; echo/calls.jsonl holds one line per call.

Unmodified 5f48fff03c74:

$ cat echo/calls.jsonl
["Ada"]
["Ada"]

With this PR:

$ cat echo/calls.jsonl
["Ada"]
["Default", "Value"]

The regression test drives the same three-step shape through build(..., no_mcp=True) / run(..., no_mcp=True, return_all=True) and asserts both final_result and the logged call arguments.

Behaviour change

Exactly one, plus a pre-existing error that becomes visible:

  1. A call with no input: mapping of its own no longer inherits an earlier call's mapping; it gets its tool's declared inputs as the pipeline author wrote them.
  2. Disclosed consequence: inside a branch, a step whose declared default has a different cardinality from the routed items previously had that mismatch masked by the leaked list (which had the right length). With the fix the call gets the declared default and save_data's pre-existing Router ... length > global_vars error surfaces. That is a misconfigured branch being reported, not a valid pipeline being rejected; the mapping step itself still works in that pipeline.

No shipped example changes behaviour: all 42 pipelines in examples/experiments/ and the documented DeepResearch pipeline in docs/llms.txt have no unmapped call after a mapped call (loops expanded two iterations, branches enumerated as paths), so they resolve as before. ultrarag build + ultrarag run examples/experiments/sayhello.yaml --no-mcp still exit 0.

Tests

New test test_step_input_mapping_does_not_leak_into_later_calls, appended to tests/test_no_mcp_execution.py (reuses that file's _server helper): builds the same three-step shape with the single-element default [Default] and asserts both final_result and the logged call arguments.

On Windows 11 (Python 3.12.16, editable install from source with uv sync --frozen, no mcp extra, one worker — pytest-xdist is not installed):

  • new test on unmodified 5f48fff03c74 (with only the test added): pytest tests/test_no_mcp_execution.py -k step_input_mapping -q → 1 failed, 12 deselected, failing with AssertionError: assert {'ans_ls': ['Ada']} == {'ans_ls': ['Default']}
  • new test with this PR: pytest tests/test_no_mcp_execution.py -k step_input_mapping -q → 1 passed, 12 deselected
  • wider selection with this PR: pytest tests/test_no_mcp_execution.py -k "leak or local_pipeline" -q -rs → 2 passed, 1 skipped, 10 deselected, the skip being tests/test_no_mcp_execution.py:521: MCP extra not installed (the pre-existing test_mcp_and_local_pipeline_agree needs the mcp extra, so that selection's count is environment-dependent; the added test is the step_input_mapping one, and it runs either way)
  • CI's pytest command pytest tests servers/generation/tests servers/retriever/tests -q, with this PR: 3 failed, 203 passed, 2 skipped in 335.60s
  • same command on unmodified main: 3 failed, 202 passed, 2 skipped in 344.53s
  • the 3 failures are identical on both trees: OSError: [WinError 1314] A required privilege is not held by the client where tests create symlinks (test_builder_agent.py::test_prompt_tools_block_traversal_and_symlinks, ...::test_pipeline_reads_reject_symlink_escape_before_loading, test_workspace_access.py::test_admin_access_and_prompt_writes_are_preserved) — environment, in files this patch does not touch
  • the only count difference is 203 vs 202 passed, the new regression test

CI's other guards on the patched tree: uv lock --check, uv sync --frozen, the no-MCP import assert, sayhello build+run --no-mcp, ruff check src/ultrarag tests/test_no_mcp_execution.py (All checks passed!) and ruff format --check (17 files already formatted) all exit 0. No .pre-commit-config.yaml and no gitleaks step exist in the repo.

Not run, not claimed: CI itself (Python 3.11 and 3.12 on ubuntu-latest); only Python 3.12.16 on Windows here. MCP-over-stdio behaviour was not exercised separately. servers/*/tests outside the CI command were not run. Real-world impact count (how many live pipelines hit the leak) is unknown.

Found and validated with AI assistance (DeepSeek V4.1 Flash via Oh My Pi).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Step-level input: mapping leaks into later calls of the same tool

1 participant