fix: remove duplicate tool processing from 7 additional providers #4601
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Following PR #4531 which fixed duplicate tool processing in OpenAI-compatible providers, this change applies the same fix to 7 additional providers that had the same issue. The problem: ToolCallAccumulator was processing tool calls in the streaming loop, but the code was also manually emitting tool_call_partial chunks, causing duplicates. The fix: Remove ToolCallAccumulator usage and rely solely on emitting raw tool_call_partial chunks, which are properly handled by NativeToolCallParser. Affected providers: - inception - lm-studio - deepinfra - xai - lite-llm - qwen-code - chutes Also removed unused ToolCallAccumulator import from base-openai-compatible-provider. Fixes the same issue as #4529 but for additional providers.
🦋 Changeset detectedLatest commit: 550f7ef The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![kiloconnect[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
Contributor
Author
There was a problem hiding this comment.
✅ No Issues Found
9 files reviewed | Confidence: 95% | Recommendation: Merge
Review Details
Files:
.changeset/fix-duplicate-tool-processing-all-providers.md- Changeset documentationsrc/api/providers/base-openai-compatible-provider.ts- Removed unused importsrc/api/providers/chutes.ts- Removed ToolCallAccumulator, kept tool_call_partial emissionsrc/api/providers/deepinfra.ts- Removed ToolCallAccumulator, kept tool_call_partial emissionsrc/api/providers/inception.ts- Removed ToolCallAccumulator, added tool_call_partial emissionsrc/api/providers/lite-llm.ts- Removed ToolCallAccumulator, kept tool_call_partial emissionsrc/api/providers/lm-studio.ts- Removed ToolCallAccumulator, added tool_call_partial emissionsrc/api/providers/qwen-code.ts- Removed ToolCallAccumulator, added tool_call_partial emissionsrc/api/providers/xai.ts- Removed ToolCallAccumulator, kept tool_call_partial emission and processFinishReason
Checked: Security, bugs, code consistency, duplicate processing
Analysis:
- The fix correctly removes
ToolCallAccumulatorusage which was causing duplicate tool call events - All providers now consistently emit
tool_call_partialchunks directly - The
NativeToolCallParserhandles state management for these chunks - The pattern matches the established fix from PR #4531
- The
xai.tsprovider correctly retains itsprocessFinishReasoncall (which was already present before this PR) - Other providers don't need
processFinishReasonas they follow the same pattern asopenai.tsandbase-openai-compatible-provider.ts
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Following PR #4531 which fixed duplicate tool processing in OpenAI-compatible providers, this PR applies the same fix to 7 additional providers that had the same issue.
Problem
The
ToolCallAccumulatorwas processing tool calls in the streaming loop, but the code was also manually emittingtool_call_partialchunks, causing duplicate tool call events to be emitted. This resulted in tools being called twice.Solution
Remove
ToolCallAccumulatorusage from the streaming loops and rely solely on emitting rawtool_call_partialchunks, which are properly handled byNativeToolCallParser.Affected Providers
inception.tslm-studio.tsdeepinfra.tsxai.tslite-llm.tsqwen-code.tschutes.tsAlso removed unused
ToolCallAccumulatorimport frombase-openai-compatible-provider.ts.Changes Made
For each affected provider:
ToolCallAccumulatorimportToolCallAccumulatorinstantiation and usage in streaming looptool_call_partialchunks with proper commentRelated Issues
Fixes the same issue as #4529 but for additional providers not covered by PR #4531.
Testing
The fix follows the exact same pattern as PR #4531, which was already tested and merged. Each provider now emits tool call chunks in a consistent way that prevents duplication.