Render image URLs in tool output details as inline images#2331
Conversation
MCP tool results normally carry images as base64 image content blocks, which ToolUpdate already renders. Some proxies strip those blocks and leave only a text block containing the image URL — e.g. hf.co/mcp with the account-level "Skip Gradio Images" flag reduces Gradio results to "Image URL: https://...image.webp" — so the tool detail showed only raw text where an image used to appear. Fall back to extracting image URLs (png/jpg/webp/gif/avif/svg) from the output text and rendering them inline when no base64 image blocks are present in that output. https://claude.ai/code/session_01EvvtbhEB86c56RNjf9sPZx
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bda56587ae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| <img | ||
| alt={`Tool result image ${imageIndex + 1}`} | ||
| class="max-h-60 cursor-pointer rounded border border-gray-200 dark:border-gray-700" | ||
| src={imageUrl} |
There was a problem hiding this comment.
When a custom or otherwise untrusted MCP tool returns text containing an image-looking URL, expanding the tool details now makes the user's browser fetch that URL automatically. That turns previously inert tool text into a network request that can leak the client IP/referrer origin and can also hit localhost, intranet, or authenticated same-origin GET endpoints if such a URL is present; please gate this on the exact trusted Image URL: field or proxy/sanitize the URL instead of loading every regex match from arbitrary output text.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
This doesn't add a new exposure class in this app's threat model: the chat markdown renderer already auto-renders remote <img> (and <video>/<audio>) from model-authored output (marked.ts image: renderer; sanitizeHref only blocks javascript:/data:text/html), with no user interaction. Model output is influenced by the same tool text — any URL a tool wants fetched can already be (and for image tools, routinely is) embedded in the answer as markdown and fetched on render. The tool-details pane is strictly narrower: it only loads after the user expands the block, and tool outputs come from admin-configured MCP_SERVERS, not arbitrary user-supplied servers. The localhost/intranet GET concern applies identically to markdown images today; <img> requests are opaque no-cors GETs either way.
Gating on the literal Image URL: label adds little: a malicious tool can emit the label trivially, so it only filters incidental URLs, while breaking the fallback for MCP servers that return bare image URLs. Proxying server-side would trade a client-side fetch for an SSRF surface plus caching/abuse concerns, which isn't warranted given the above.
Applied the free part: referrerpolicy="no-referrer" on the URL-based images (9fcb5f6), so the page origin isn't disclosed to image hosts — which makes these images leak less than markdown-rendered ones do today.
Generated by Claude Code
2 participants
Context
Images stopped showing in the tool-call output details (e.g.
gr1_qwen_image_fast_inferresults showed onlyImage URL: https://...image.webpas raw text).Investigation traced it end to end:
{type: "image", data, mimeType}content blocks — that path is unchanged and works.multimodalart/Qwen-Image-Fast, pinned Gradio 5.39) still returns the base64 image block + theImage URL:text block (verified by calling its MCP endpoint directly).hf.co/mcp(@huggingface/mcp-servicesv0.3.19) strips image content blocks for accounts whose MCP settings include theNO_GRADIO_IMAGE_CONTENT("Skip Gradio Images") flag —GET /api/settings/mcpconfirmed the flag is set on the affected account, and an authenticated A/B call confirmed only text blocks survive the proxy.So the base64 block never reaches chat-ui in that configuration, and the only image reference left is the URL inside the text.
Change
ToolUpdate.sveltenow falls back to extracting image URLs (png/jpg/jpeg/webp/gif/avif/svg, query strings allowed) from the output text and renders them inline — only when the output carries no base64 image blocks, so nothing is shown twice in the normal case.Testing
Image URL: https://...hf.space/--replicas/.../image.webp+ seed line), markdown-wrapped URLs, query strings, and non-image URLs (ignored).npm run check: 0 errors (1 pre-existing warning inChatMessage.svelte).server(310 tests) andssrworkspaces pass. Client browser workspace is opt-in (VITEST_BROWSER=true) and was not run.https://claude.ai/code/session_01EvvtbhEB86c56RNjf9sPZx
Generated by Claude Code