Agentic BodyMaps AI assistant: tool loop, self-capture, and document reading

[Awpoon123]

Summary

Extends the BodyMaps AI sidebar from a single-shot chatbot into an assistant that investigates a case before answering, can see the viewer, and reads attached documents. Everything runs on the locally hosted Ollama models, so no case data leaves the machine.

What's new

Agentic investigation. For questions about the open case, the assistant runs a bounded tool loop instead of answering from a static payload. It can fetch an organ's measured volume, mean HU and reference percentile, list segmented structures, read patient demographics, isolate organs in the viewer, and change the CT window — observing each result before deciding its next step. Progress is streamed to the UI, viewer actions fire immediately, and rounds are capped.

Self-capture. When a question is about visual appearance, the assistant requests screenshots of the CT views; the browser captures them, attaches them to the conversation for transparency, and the turn continues through the vision model. Limited to one capture per turn.

Document reading. Attached PDFs are parsed in the browser and their text travels with the message, so the assistant can respond to document content and ask informed follow-ups.

Answer quality. Model reasoning can no longer leak into replies (including the orphan-</think> shape some local models emit); answers are length calibrated; multi-turn context is preserved so follow-ups continue the conversation instead of being misrouted to case data; multi-part requests are answered part by part; the color-legend shortcut no longer hijacks long image prompts. The system prompt was also compressed ~46% with all behavior preserved, which measurably cuts response latency on CPU.

Reliability. The dev server now runs threaded so long-running work cannot starve other requests, and the default CORS allowlist covers both localhost spellings.

Notes for reviewers

• The assistant no longer requires sign-in: _ai_gate() is open and usage is recorded only for signed-in users. Flagging explicitly in case plan limits should be reinstated before production.
• New frontend dependency: pdfjs-dist.
• services/mesh_generation.py is included because api_blueprint.py imports bake_case_meshes from it; the 3D viewer changes themselves are not part of this PR.
• Every failure path degrades gracefully: agent loop → plain streaming → non-streaming endpoint → rule-based replies. With Ollama stopped entirely, viewer commands and measurements still answer.

Testing

• Frontend: npm run build, npx tsc -b, and npm test (193 tests) pass.
• Backend: python -m pytest tests/unit passes with no new failures relative to main.
• Manual: verified the Ollama-unavailable fallback path and the assistant end to end against local models.