Repository navigation
feat(telemetry): profiled network latency, capacity, and real-time bandwidth usage in UI - #2471
Open
presentyourlove wants to merge 1 commit into
Open
presentyourlove wants to merge 1 commit into
presentyourlove wants to merge 1 commit into
Conversation
…ndwidth usage in UI
cdvankammen
pushed a commit
to cdvankammen/exo
that referenced
this pull request
Oct 23, 2026
…ge in UI (exo#2471) Ports upstream exo-explore#2471 (9a2b47a) onto the fork chain head ecef3ad (the exo-explore#2470 connection-types-viz port). Resolves TODO.md items 4 and 5. Resolved 5 conflict blocks across 4 files. The fork's exo-explore#2470 port added hardware connection-type badges (TB5/Ethernet/WiFi/RDMA) to the same topology graph regions upstream's exo-explore#2471 extends with telemetry badges, so every conflict was a "keep both" merge rather than a take-one-side. Conflict resolution summary --------------------------- - channels.py: fork already had the anyio MemoryObjectStreamState try/except fallback from an earlier port. Upstream's hunk was a no-op whitespace change. Kept fork side entirely (skill §9d: already satisfied). - worker/main.py: merged imports — kept fork's full GatheredInfo import list (MacThunderboltIdentifiers, MiscData, etc.) AND added upstream's NodeLinkTelemetryData + link_telemetry_registry. Also applied upstream's minor cleanups (timezone.utc -> UTC, set literal -> set comprehension). - app.svelte.ts: kept fork's full connection-type classification block (RDMA check, IP heuristic fallback, tb5/tb/wifi/ethernet/virtual routing) AND added upstream's telemetry lookup (nodeLinkTelemetry match per edge). Both are independent additions to the same edge transform. - TopologyGraph.svelte (5 blocks): merged ConnectionInfo type fields (connectionType/connectionCategory + latencyMs/bandwidthCapacityMbps/ usedBandwidthMbps), merged entry.connections.push fields, assigned the themed link to `baseLine` so upstream's tooltip append works, kept upstream's telemetry extraction + tooltip code, and kept BOTH badges — the fork's hardware-type badge pill (badgesGroup, TODO exo-explore#14) AND upstream's telemetry badge (linksGroup, TODO exo-explore#4/exo-explore#5) offset vertically so they don't overlap. What the port does ------------------ Backend (src/exo/): - New types: LinkTelemetry (per-link RTT latency, bandwidth capacity, real-time usage) and NodeLinkTelemetry (collection of outbound link telemetry). - State: adds node_link_telemetry mapping to State, with apply_node_timed_out cleanup and apply_node_gathered_info handler for NodeLinkTelemetryData. - net_profile.py: adds InterfaceBandwidthTracker (psutil net_io_counters delta -> MB/s per iface) and LinkTelemetryRegistry (EMA-smoothed latency, interface-type-based capacity estimation, full telemetry snapshot generation). - check_reachability now measures RTT via time.perf_counter around the HTTP probe and records it in the registry. - worker/main.py: _poll_connection_updates broadcasts a NodeLinkTelemetryData snapshot every 10s alongside the existing topology edge updates. Dashboard (dashboard/src/): - app.svelte.ts: TopologyEdge gains latencyMs/bandwidthCapacityMbps/ usedBandwidthMbps fields, populated from granularState.nodeLinkTelemetry lookup per source->sink pair. - TopologyGraph.svelte: link tooltips now include latency RTT, bandwidth usage, and link capacity when available. A second badge pill (translucent white outline, offset above the hardware-type badge) shows real-time "MB/s · ms" at each link midpoint when telemetry data exists. Verification ------------ - npm run build: exit 0. - npm run check: 15 errors on the changed tree AND 15 on the base worktree (ecef3ad); the error-location sets are byte-identical (comm diff empty). No new diagnostics. - pytest src/exo/utils/info_gatherer: 6 passed on base, 11 passed on the changed tree (the 5 new upstream tests: EMA smoothing, capacity estimation, generate_node_telemetry, apply_node_link_telemetry_state, interface_bandwidth_tracker). - pytest src/exo/shared/tests: 5 passed on both (no regressions from apply.py/state.py changes). - pytest src/exo/worker/tests/unittests (excluding mlx/image/runner): 34 passed on both (no regressions). - pytest src/exo/api: 12 passed on both (no regressions from channels.py). Completeness: all upstream symbols (LinkTelemetry, NodeLinkTelemetry, NodeLinkTelemetryData, InterfaceBandwidthTracker, LinkTelemetryRegistry, link_telemetry_registry, node_link_telemetry) are present in the ported tree. All fork symbols from exo-explore#2470 (connection-badge, badgesGroup, connectionCategory, ip_to_interface_type, themeColors) are preserved.
cdvankammen
pushed a commit
to cdvankammen/exo
that referenced
this pull request
Oct 23, 2026
…ge in UI (exo#2471) Ports upstream exo-explore#2471 (9a2b47a) onto the fork chain head ecef3ad (the exo-explore#2470 connection-types-viz port). Resolves TODO.md items 4 and 5. Resolved 5 conflict blocks across 4 files. The fork's exo-explore#2470 port added hardware connection-type badges (TB5/Ethernet/WiFi/RDMA) to the same topology graph regions upstream's exo-explore#2471 extends with telemetry badges, so every conflict was a "keep both" merge rather than a take-one-side. Conflict resolution summary --------------------------- - channels.py: fork already had the anyio MemoryObjectStreamState try/except fallback from an earlier port. Upstream's hunk was a no-op whitespace change. Kept fork side entirely. - worker/main.py: merged imports — kept fork's full GatheredInfo import list (MacThunderboltIdentifiers, MiscData, etc.) AND added upstream's NodeLinkTelemetryData + link_telemetry_registry. Also applied upstream's minor cleanups (timezone.utc -> UTC, set() -> set literal). - app.svelte.ts: kept fork's full connection-type classification block (RDMA check, IP heuristic fallback, tb5/tb/wifi/ethernet/virtual routing) AND added upstream's telemetry lookup (nodeLinkTelemetry match per edge). Both are independent additions to the same edge transform. - TopologyGraph.svelte (5 blocks): merged ConnectionInfo type fields (connectionType/connectionCategory + latencyMs/bandwidthCapacityMbps/ usedBandwidthMbps), merged entry.connections.push fields, assigned the themed link to `baseLine` so upstream's tooltip append works, kept upstream's telemetry extraction + tooltip code, and kept BOTH badges — the fork's hardware-type badge pill (badgesGroup, TODO exo-explore#14) AND upstream's telemetry badge (linksGroup, TODO exo-explore#4/exo-explore#5) offset vertically so they don't overlap. Adaptation note: upstream's tooltip helper `getNodeDisplayName()` does not exist in the fork's TopologyGraph.svelte. The fork's equivalent helper is `getNodeLabel(nodeId)` (line 39, resolves data.nodes[id].friendly_name with an 8-char id fallback). The telemetry tooltip call site was rewritten to use getNodeLabel — without this the port introduced a new svelte-check error. What the port does ------------------ Backend (src/exo/): - New types: LinkTelemetry (per-link RTT latency, bandwidth capacity, real-time usage) and NodeLinkTelemetry (collection of outbound link telemetry). - State: adds node_link_telemetry mapping to State, with apply_node_timed_out cleanup and apply_node_gathered_info handler for NodeLinkTelemetryData. - net_profile.py: adds InterfaceBandwidthTracker (psutil net_io_counters delta -> MB/s per iface) and LinkTelemetryRegistry (EMA-smoothed latency, interface-type-based capacity estimation, full telemetry snapshot generation). - check_reachability now measures RTT via time.perf_counter around the HTTP probe and records it in the registry. - worker/main.py: _poll_connection_updates broadcasts a NodeLinkTelemetryData snapshot every 10s alongside the existing topology edge updates. Dashboard (dashboard/src/): - app.svelte.ts: TopologyEdge gains latencyMs/bandwidthCapacityMbps/ usedBandwidthMbps fields, populated from granularState.nodeLinkTelemetry lookup per source->sink pair. - TopologyGraph.svelte: link tooltips now include latency RTT, bandwidth usage, and link capacity when available. A second badge pill (translucent white outline, offset above the hardware-type badge) shows real-time "MB/s · ms" at each link midpoint when telemetry data exists. Verification (measured on this tree vs a side-by-side ecef3ad worktree) ----------------------------------------------------------------------- - npm run build: exit 0. - npm run check: 15 errors / 6 warnings on the changed tree; 15 errors / 6 warnings on the base worktree. Error-message sets diffed identical after the getNodeLabel fix (the first pass had 17 vs 15 — the 2 extra were the getNodeDisplayName errors, now gone). - pytest src/exo/utils/info_gatherer: 8 passed (base) -> 13 passed (changed). The 5 new tests are the upstream telemetry tests: registry EMA smoothing, capacity estimation, generate_node_telemetry, apply_node_link_telemetry_state, interface_bandwidth_tracker. - pytest src/exo/shared/tests: 228 passed, 1 skipped on BOTH trees. - pytest src/exo/worker/tests/unittests (excluding mlx/image/runner): 50 passed on BOTH trees. - pytest src/exo/api: 82 passed on BOTH trees. Completeness: all upstream symbols (LinkTelemetry, NodeLinkTelemetry, NodeLinkTelemetryData, InterfaceBandwidthTracker, LinkTelemetryRegistry, link_telemetry_registry, node_link_telemetry) are present in the ported tree. All fork symbols from exo-explore#2470 (connection-badge, badgesGroup, connectionCategory, ip_to_interface_type, themeColors) are preserved.
cdvankammen
pushed a commit
to cdvankammen/exo
that referenced
this pull request
Oct 23, 2026
…ge in UI (exo#2471) Ports upstream exo-explore#2471 (9a2b47a) onto the fork chain head ecef3ad (the exo-explore#2470 connection-types-viz port). Resolves TODO.md items 4 and 5. Resolved 5 conflict blocks across 4 files. The fork's exo-explore#2470 port added hardware connection-type badges (TB5/Ethernet/WiFi/RDMA) to the same topology graph regions upstream's exo-explore#2471 extends with telemetry badges, so every conflict was a "keep both" merge rather than a take-one-side. Conflict resolution summary --------------------------- - channels.py: fork already had the anyio MemoryObjectStreamState try/except fallback from an earlier port. Upstream's hunk was a no-op whitespace change. Kept fork side entirely. - worker/main.py: merged imports — kept fork's full GatheredInfo import list (MacThunderboltIdentifiers, MiscData, etc.) AND added upstream's NodeLinkTelemetryData + link_telemetry_registry. Also applied upstream's minor cleanups (timezone.utc -> UTC, set() -> set literal). - app.svelte.ts: kept fork's full connection-type classification block (RDMA check, IP heuristic fallback, tb5/tb/wifi/ethernet/virtual routing) AND added upstream's telemetry lookup (nodeLinkTelemetry match per edge). Both are independent additions to the same edge transform. - TopologyGraph.svelte (5 blocks): merged ConnectionInfo type fields (connectionType/connectionCategory + latencyMs/bandwidthCapacityMbps/ usedBandwidthMbps), merged entry.connections.push fields, assigned the themed link to `baseLine` so upstream's tooltip append works, kept upstream's telemetry extraction + tooltip code, and kept BOTH badges — the fork's hardware-type badge pill (badgesGroup, TODO exo-explore#14) AND upstream's telemetry badge (linksGroup, TODO exo-explore#4/exo-explore#5) offset vertically so they don't overlap. Adaptation note: upstream's tooltip helper `getNodeDisplayName()` does not exist in the fork's TopologyGraph.svelte. The fork's equivalent helper is `getNodeLabel(nodeId)` (line 39, resolves data.nodes[id].friendly_name with an 8-char id fallback). The telemetry tooltip call site was rewritten to use getNodeLabel — without this the port introduced a new svelte-check error. What the port does ------------------ Backend (src/exo/): - New types: LinkTelemetry (per-link RTT latency, bandwidth capacity, real-time usage) and NodeLinkTelemetry (collection of outbound link telemetry). - State: adds node_link_telemetry mapping to State, with apply_node_timed_out cleanup and apply_node_gathered_info handler for NodeLinkTelemetryData. - net_profile.py: adds InterfaceBandwidthTracker (psutil net_io_counters delta -> MB/s per iface) and LinkTelemetryRegistry (EMA-smoothed latency, interface-type-based capacity estimation, full telemetry snapshot generation). - check_reachability now measures RTT via time.perf_counter around the HTTP probe and records it in the registry. - worker/main.py: _poll_connection_updates broadcasts a NodeLinkTelemetryData snapshot every 10s alongside the existing topology edge updates. Dashboard (dashboard/src/): - app.svelte.ts: TopologyEdge gains latencyMs/bandwidthCapacityMbps/ usedBandwidthMbps fields, populated from granularState.nodeLinkTelemetry lookup per source->sink pair. - TopologyGraph.svelte: link tooltips now include latency RTT, bandwidth usage, and link capacity when available. A second badge pill (translucent white outline, offset above the hardware-type badge) shows real-time "MB/s · ms" at each link midpoint when telemetry data exists. Verification (measured on this tree vs a side-by-side ecef3ad worktree) ----------------------------------------------------------------------- - npm run build: exit 0. - npm run check: 15 errors / 6 warnings on the changed tree; 15 errors / 6 warnings on the base worktree. Error-message sets diffed identical after the getNodeLabel fix (the first pass had 17 vs 15 — the 2 extra were the getNodeDisplayName errors, now gone). - pytest src/exo/utils/info_gatherer: 8 passed (base) -> 13 passed (changed). The 5 new tests are the upstream telemetry tests: registry EMA smoothing, capacity estimation, generate_node_telemetry, apply_node_link_telemetry_state, interface_bandwidth_tracker. - pytest src/exo/shared/tests: 228 passed, 1 skipped on BOTH trees. - pytest src/exo/worker/tests/unittests (excluding mlx/image/runner): 50 passed on BOTH trees. - pytest src/exo/api: 82 passed on BOTH trees. Completeness: all upstream symbols (LinkTelemetry, NodeLinkTelemetry, NodeLinkTelemetryData, InterfaceBandwidthTracker, LinkTelemetryRegistry, link_telemetry_registry, node_link_telemetry) are present in the ported tree. All fork symbols from exo-explore#2470 (connection-badge, badgesGroup, connectionCategory, ip_to_interface_type, themeColors) are preserved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR implements items 4 and 5 in
TODO.md:Profiled network latency (RTT) and link bandwidth capacity.Real-time link bandwidth usage (throughput in MB/s per link).Background & Motivation
In distributed LLM inference clusters (e.g. Pipeline Parallelism and Tensor Parallelism / Ring / Jaccl), communication overhead and network bottlenecking directly constrain token throughput. Operators previously had no visibility into:
This PR establishes an end-to-end telemetry pipeline from low-level network sampling and EWMA latency profiling up to the dashboard UI.
Key Changes
Profiling Data Models & State (
src/exo/shared/types/profiling.py,src/exo/shared/types/state.py):LinkTelemetryandNodeLinkTelemetrydata structures capturinglatency_ms,bandwidth_capacity_mbps,used_bandwidth_mbps, and interface metrics.node_link_telemetry: Mapping[NodeId, NodeLinkTelemetry]toState.NodeLinkTelemetryDatagatherer event and updatedapply_node_gathered_info/apply_node_timed_outstate reducers insrc/exo/shared/apply.py.Differential Bandwidth Tracking & EWMA Latency Profiler (
src/exo/utils/info_gatherer/net_profile.py):InterfaceBandwidthTracker: Samples monotonic delta counters viapsutil.net_io_counters(pernic=True)to calculate real-time transfer rates in MB/s per network interface.LinkTelemetryRegistry: Applies Exponential Weighted Moving Average (EWMA,time.perf_counter()RTT measurement intocheck_reachability._ping_nodesinsrc/exo/worker/main.py.Frontend Dashboard Visualization (
dashboard/src/lib/stores/app.svelte.ts,dashboard/src/lib/components/TopologyGraph.svelte):TopologyEdgeandGranularNodeStatewith latency, capacity, and throughput properties.14.2 MB/s · 0.8 ms) inTopologyGraph.svelte.<title>tooltips displaying latency RTT, active throughput, and physical link capacity.Compatibility & Unit Tests (
src/exo/utils/channels.py,src/exo/utils/info_gatherer/tests/test_net_telemetry.py):_MemoryObjectStreamStateinchannels.py.Verification
.venv/Scripts/pytest.exe src/exo/utils/info_gatherer/tests/test_net_telemetry.py -v # 5 passed in 1.05s.venv/Scripts/pytest.exe src/exo/shared/tests/test_apply/ -v # 25 passed in 0.34s.venv/Scripts/ruff.exe check src/exo/utils/info_gatherer/net_profile.py src/exo/shared/apply.py src/exo/shared/types/profiling.py src/exo/shared/types/state.py src/exo/worker/main.py # All checks passed!Related Issues
TODO.md(Profiled network latency (RTT) and link bandwidth capacity)TODO.md(Real-time link bandwidth usage (throughput in MB/s per link))