Problem / Problema
EN: Each conversation turn currently takes ~5–11s end-to-end (perceptible on
WhatsApp). Most of it is model + tool round-trips. We already send a "typing…"
indicator, but the wall-clock is still high.
ES: Cada turno de conversación tarda ~5–11s de punta a punta (perceptible en
WhatsApp). La mayor parte es el modelo + los round-trips de tools. Ya enviamos el
indicador de "escribiendo…", pero el tiempo total sigue siendo alto.
Ideas to explore / Ideas a explorar
- Measure where the time goes (add per-phase timings to the structured logs in
src/infrastructure/observability/).
- Reduce tool round-trips (batch/parallelize independent tool calls where safe).
- Consider streaming partial responses where the channel supports it.
- Cache stable lookups within a turn (e.g. emergency by slug).
Acceptance / Aceptación
- A short write-up of where the latency is spent, backed by real timings.
- At least one measurable improvement to median turn time.
Good starting point: src/application/conversation-service.ts and the agent
tools in src/agent/tools.ts.
Problem / Problema
EN: Each conversation turn currently takes ~5–11s end-to-end (perceptible on
WhatsApp). Most of it is model + tool round-trips. We already send a "typing…"
indicator, but the wall-clock is still high.
ES: Cada turno de conversación tarda ~5–11s de punta a punta (perceptible en
WhatsApp). La mayor parte es el modelo + los round-trips de tools. Ya enviamos el
indicador de "escribiendo…", pero el tiempo total sigue siendo alto.
Ideas to explore / Ideas a explorar
src/infrastructure/observability/).Acceptance / Aceptación
Good starting point:
src/application/conversation-service.tsand the agenttools in
src/agent/tools.ts.