[FixBug] Bedrock anthropic max iterations exceeded error

[noproblem520]

Related Issues or Context

I saw 2068 has been partially fixed this issue, we still need to tell LLM to summarize the information gathering from previous tool calls, otherwise it will still call __no_more_tools_available__.

Fix langgenius/dify#27428
Fix langgenius/dify#27215
Fix langgenius/dify#17545

--- Old: 2068 has fixed partial problems based on this PR---

Issues
When we use bedrock anthropic models, there's a limitation「The toolConfig field must be defined when using toolUse and toolResult content blocks.」. function_calling.py will empty the tools that agents can use in the last round, this logic is fine but violates the limitation of bedrock anthropic models.

Reproduce
We can easily reproduce the error by setting max_iterations to 2. If agent tries to think more than twice, the error occurred.

This PR contains Changes to Non-Plugin

• Documentation
• Other

This PR contains Changes to Non-LLM Models Plugin

• I have Run Comprehensive Tests Relevant to My Changes

This PR contains Changes to LLM Models Plugin

• My Changes Affect Message Flow Handling (System Messages and User→Assistant Turn-Taking)

• My Changes Affect Tool Interaction Flow (Multi-Round Usage and Output Handling, for both Agent App and Agent Node)

• My Changes Affect Multimodal Input Handling (Images, PDFs, Audio, Video, etc.)

• My Changes Affect Multimodal Output Generation (Images, Audio, Video, etc.)

• My Changes Affect Structured Output Format (JSON, XML, etc.)

• My Changes Affect Token Consumption Metrics

• My Changes Affect Other LLM Functionalities (Reasoning Process, Grounding, Prompt Caching, etc.)

• Other Changes (Add New Models, Fix Model Parameters etc.)

Version Control (Any Changes to the Plugin Will Require Bumping the Version)

• I have Bumped Up the Version in Manifest.yaml (Top-Level Version Field, Not in Meta Section)

Dify Plugin SDK Version

• I have Ensured dify_plugin>=0.3.0,<0.6.0 is in requirements.txt (SDK docs)

Environment Verification (If Any Code Changes)

Local Deployment Environment

• Dify Version is: , I have Tested My Changes on Local Deployment Dify with a Clean Environment That Matches the Production Configuration.

SaaS Environment

• I have Tested My Changes on cloud.dify.ai with a Clean Environment That Matches the Production Configuration

[gemini-code-assist]

Summary of Changes

Hello @noproblem520, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical issue with Bedrock Anthropic models that caused 'max iterations exceeded' errors during agent execution. The problem stemmed from the model's strict requirement for a toolConfig field when toolUse or toolResult blocks are present in the prompt, even if no actual tools are available. The solution involves strategically injecting a dummy tool configuration and a direct instruction into the prompt during the final stages of an agent's reasoning process, ensuring compliance with the model's API while guiding it to conclude its task without further tool calls.

Highlights

• Bedrock Anthropic Tooling Fix: Addresses a specific error in Bedrock Anthropic models where the toolConfig field is required even when no tools are available, particularly when toolUse or toolResult content blocks are present in the prompt history.
• Dummy Tool Implementation: Introduces a mechanism to add a dummy toolConfig with a 'no_tools_available' specification when the agent's function calling strategy clears all tools in the last round, satisfying the Bedrock Claude model's requirement.
• Final Iteration Instruction: Adds a crucial instruction to the prompt in the final iteration of function calling, explicitly telling the model 'IMPORTANT: DO NOT call any more tools' and to summarize gathered information, preventing further tool attempts.
• Version Bump: The bedrock plugin version has been updated from 0.0.50 to 0.0.51 in manifest.yaml.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses a max_iterations error with Bedrock Anthropic models by ensuring a toolConfig is always present when the message history contains tool usage, even in the final iteration when no tools are available. The approach of adding a dummy tool and a final instruction to the model is a clever workaround for this specific API requirement. My review includes suggestions to make the detection logic for tool usage history more robust and less prone to errors, improving both correctness and maintainability.

[ybalbert001]

This final prompt maybe not meet the requirement for all situation. And it's not transparent to the end user.

[noproblem520]

The Dify workflow agent node fails to generate a final summary in the last round if the LLM isn't explicitly prompted to summarize. The last round in the agent returns an message like There is not a tool named __no_more_tools_available__.. This results in the workflow returning an truncated message, e.g. "Hi, I will help you figure out the problem:", based on the user's question

Our logic dictates that the final prompt is only added when toolUse and toolResult are present and the flow is in the is_final_round. This implementation can prevent any other issues, I guess.

I've validated this code in production, and it's working perfectly. If you have any concerns, please let me know.