Skip to content

fix: prevent response_model from being passed in ReAct flow when LLM lacks function calling#4698

Open
devin-ai-integration[bot] wants to merge 3 commits intomainfrom
devin/1772626609-fix-output-pydantic-no-function-calling
Open

fix: prevent response_model from being passed in ReAct flow when LLM lacks function calling#4698
devin-ai-integration[bot] wants to merge 3 commits intomainfrom
devin/1772626609-fix-output-pydantic-no-function-calling

Conversation

@devin-ai-integration
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot commented Mar 4, 2026

fix: don't pass response_model in ReAct flow for non-FC LLMs

Summary

Fixes #4695. When an LLM does not support function calling (supports_function_calling() returns False), the executor falls back to the ReAct text-based pattern. Previously, response_model (derived from task.output_pydantic) was still passed to get_llm_response in this path, which caused InternalInstructor to force structured output via instructor's TOOLS mode — before the agent could reason through Action/Observation cycles.

The fix sets response_model=None in both _invoke_loop_react() and _ainvoke_loop_react(). The output schema is already embedded in the prompt text for guidance (via build_task_prompt_with_schema), and the final conversion to pydantic/json happens downstream in task._export_output().

This also removes the now-dead if self.response_model is not None early-exit block from both methods, simplifying the ReAct response handling to always go through process_llm_response.

Changes to _invoke_loop / _ainvoke_loop routing

The routing logic was refactored to handle three cases explicitly:

Condition Route Rationale
FC-capable + user-defined tools _invoke_loop_native_tools Unchanged — native tool calling
FC-capable + no tools at all (self.tools empty) + response_model _invoke_loop_native_no_tools Single-shot structured output via instructor; no tools to iterate over
Everything else (non-FC, or FC with internal tools, or no response_model) _invoke_loop_react ReAct text-based loop; response_model=None to avoid forcing structured output

The check uses self.tools (which includes internal tools like delegation/human input) rather than self.original_tools (user-defined only), so agents with delegation enabled still use the ReAct loop even if they have no user-defined tools.

Updates since last revision

  • Used self.tools instead of self.original_tools for no-tools routing check (flagged by Bugbot): self.original_tools only includes user-defined tools, but self.tools also includes internal tools (delegation, human input). Using self.tools ensures agents with internal tools still use the ReAct loop for Action/Observation cycles.
  • Added test for FC+internal-tools scenario (total: 15 new tests).

Review & Testing Checklist for Human

  • Verify task._export_output() reliably converts ReAct string output to pydantic: The fix relies on downstream convert_to_model (which may re-call the LLM) to produce the pydantic output. The old code had an early-exit that accepted raw valid JSON matching the schema directly — this path is now removed. Test with a real non-FC LLM (e.g., Ollama) to confirm end-to-end pydantic output works.
  • Verify ReAct parser handles JSON-like final answers: Now all answers go through process_llm_response which expects the Final Answer: prefix. Confirm the prompt instructs the LLM to use this format, and that models don't produce raw JSON without the prefix.
  • Verify the self.tools vs self.original_tools distinction is correct: The routing uses not self.tools to check for truly no tools. Confirm that self.tools reliably includes all internal tools (delegation, human input, memory) and that there's no edge case where self.tools is empty but internal tools are still expected.
  • Verify the FC+no-tools+response_model path: _invoke_loop_native_no_tools does a single LLM call (no iteration loop). Confirm this is acceptable when an FC-capable LLM has output_pydantic but no tools.

Suggested manual test: Run a crew with output_pydantic set on a task, using an Ollama model (or any model where supports_function_calling() returns False), and confirm the agent completes the ReAct loop and produces valid pydantic output.

Notes

  • 15 new tests added covering sync, async, routing (including FC+no-tools+internal-tools edge cases), tool usage, and crew-level integration
  • All existing executor tests (sync + async) pass with no regressions (55 total tests passing)
  • Link to Devin session: https://app.devin.ai/sessions/fad74341a5d2401dab83634287fc10fd
  • Requested by: João

Note

Cursor Bugbot is generating a summary for commit c5d4384. Configure here.

…lacks function calling

When an LLM does not support function calling (supports_function_calling()
returns False), the executor falls back to the ReAct text-based pattern.
Previously, response_model (set from task.output_pydantic) was still passed
to get_llm_response in the ReAct path, which caused InternalInstructor to
force structured output via instructor's TOOLS mode before the agent could
reason through Action/Observation cycles.

This fix sets response_model=None in both _invoke_loop_react and
_ainvoke_loop_react, allowing the ReAct loop to work normally. The output
schema is already embedded in the prompt text for guidance, and the final
conversion to pydantic/json happens in task._export_output() after the
agent finishes.

Fixes #4695

Co-Authored-By: João <joao@crewai.com>
@devin-ai-integration
Copy link
Contributor Author

Prompt hidden (unlisted session)

@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

- Remove unused asyncio import from test file
- Route FC-capable LLMs with no tools + response_model to
  _invoke_loop_native_no_tools (preserves structured output)
- FC-capable LLMs with no tools and no response_model still
  fall through to ReAct path (no regression)
- Add tests for both FC+no-tools routing scenarios

Co-Authored-By: João <joao@crewai.com>
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Address Bugbot concern: self.tools includes internal tools (delegation,
human input) while self.original_tools only has user-defined tools.
Only route to native_no_tools when there are truly no tools at all,
so agents with internal tools still use the ReAct loop.

Add test for FC+internal-tools scenario.

Co-Authored-By: João <joao@crewai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] output_pydantic model injected as native tool even when supports_function_calling() is False

0 participants