[Agent] Gemini INVALID_ARGUMENT in Agent

[pr-maia]

Self Checks

• I have read the Contributing Guide and Language Policy.
• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.9.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Gemini Plugin Version: 0.5.5

A workflow that was running just fine until this morning started throwing this error:

Run failed: Failed to transform agent message: req_id: 9d46b2ab18 PluginInvokeError: {"args":{},"error_type":"Exception","message":"read llm model failed: request failed: [google] Error: req_id: 75678b1c8a PluginInvokeError: {"args":{},"error_type":"ClientError","message":"400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': 'The input token count (947054) exceeds the maximum number of tokens allowed (131072).', 'status': 'INVALID_ARGUMENT'}}"}"}

Two things caught my attention:

(1) The input token count (947k) is just below the limit for the Gemini 2.5 Pro model in the Paid tier but i'm receiving an error mentioning the 131k limit from the free trial.

(2) The mention of INVALID_ARGUMENT

I've wrote a small script that takes an equivalent amount of input tokens and a standalone evaluation in python and the model answered just fine with no errors.

Maybe there was some changes in the Gemini API that need to be reflected in the Dify plugin.

✔️ Expected Behavior

The model should answer just like it does when running it outside of dify (python)

❌ Actual Behavior

Run failed: Failed to transform agent message: req_id: 9d46b2ab18 PluginInvokeError: {"args":{},"error_type":"Exception","message":"read llm model failed: request failed: [google] Error: req_id: 75678b1c8a PluginInvokeError: {"args":{},"error_type":"ClientError","message":"400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': 'The input token count (947054) exceeds the maximum number of tokens allowed (131072).', 'status': 'INVALID_ARGUMENT'}}"}"}

[pr-maia]

Same API Key with a similar amount of tokens (770k here) and i get an answer just fine in python just like i used to in the dify workflow

Small code i've used to debug:

genai.configure(api_key=API_KEY)
model = genai.GenerativeModel("gemini-2.5-pro")

# Ask Gemini to summarize the JSON content
prompt = (
    "Summarize the following economic reports in a concise way, with the main points and key information:\n\n"
    f"{json_str}"
)

response = model.generate_content(prompt)
print("Summary:\n", response.text)

[...]
              }
            ],
            "role": "model"
          },
          "finish_reason": "STOP",
          "index": 0
        }
      ],
      "usage_metadata": {
        "prompt_token_count": 772441,
        "candidates_token_count": 530,
        "total_token_count": 773963
      },
      "model_version": "gemini-2.5-pro"
    }),
)

[dosubot]

The Gemini plugin in Dify is known to have a token counting bug that can massively overcount tokens, especially when the Google Vertex AI client is unavailable or misconfigured. In these cases, the plugin falls back to a generic counting method (like counting characters), which is incompatible with Gemini models and can inflate token counts by an order of magnitude or more. This leads to errors like INVALID_ARGUMENT, even when your actual input is below the paid tier limit and works fine with the official Python client using the same API key and model source.

Recent plugin updates have improved token counting by using Gemini's usage metadata when available, but fallback logic can still trigger if the metadata is missing or zero, causing overcounting source. There was also a bug where completion tokens were incorrectly accumulated for every chunk, leading to duplicate counting and inflated totals, which has been fixed in newer plugin versions source.

To mitigate this, you can:

• Patch the fallback logic in tokenizer.py to use a conservative estimate (e.g., len(text)//4) and log a warning when the official Google method is unavailable.
• Make sure your deployment always has access to the official Google token counting endpoint.
• If you are seeing inflated token counts, consider downgrading the Gemini plugin to a version known to have correct token calculation (e.g., 0.1.5), as some users have reported this resolves the issue source.

This issue affects both free and paid API tiers and is especially severe for Gemini Pro 2.5 models. Monitoring for upstream fixes and ensuring your environment is correctly configured for the official token counting method is recommended for long-term reliability.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[pr-maia]

So this is really a bug in the plugin? That's why it works in my standalone code?

[dosubot]

Yes, this is a known bug in the Gemini plugin for Dify. The plugin can massively overcount tokens—especially if it can't access Google's official token counting endpoint—by falling back to a generic method that's not compatible with Gemini models. This leads to inflated token counts and INVALID_ARGUMENT errors, even when your input is within the actual model limits. Your standalone Python code works because it uses the official client, which counts tokens accurately and respects your paid tier limits. For more details, see the discussion and technical explanation here: issue #1363 and issue #1350.

Recent plugin updates (like PR #1491) have improved token counting, but the latest 0.5.5 release mainly adds new models and does not mention a fix for this bug. To work around it, make sure your deployment can always reach the official Google token counting endpoint, or patch the fallback logic to use a more conservative estimate if needed.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[pr-maia]

@crazywoola doesn't seem the PR dosubot mentioned is related to my issue here

[dosubot]

Hi, @pr-maia. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

• You reported that the Gemini plugin (v0.5.5) in Dify 1.9.1 was overcounting input tokens, causing 400 INVALID_ARGUMENT errors despite inputs being valid.
• The issue was traced to the plugin falling back to a generic token counting method when Google's official endpoint was unavailable.
• This fallback method inflated token counts inaccurately, triggering errors.
• Recent plugin updates have improved token counting accuracy but may not fully resolve the problem.
• Users are advised to ensure access to the official token counting service or patch the fallback logic.

Next Steps:

• Please confirm if this issue is still relevant with the latest version of Dify and the Gemini plugin.
• If it is, feel free to keep the discussion open by commenting; otherwise, I will automatically close this issue in 15 days.

Thank you for your understanding and contribution!