[presentyourlove]

Description

This PR resolves item 13 in TODO.md: Memory pressure instead of memory used.

Background & Motivation

On macOS and Apple Silicon unified memory architectures, the Mach virtual memory subsystem actively retains file-backed caches, purgeable pages, and inactive allocations rather than leaving RAM empty ("Free RAM is wasted RAM"). Consequently:

1. macmon and standard tools report "memory used" hovering around 85%–95%, even when 30–60 GB of file cache can be immediately and painlessly reclaimed by the OS for Metal/MLX allocations.
2. In src/exo/utils/info_gatherer/macmon.py, naive ram_available = ram_total - ram_usage falsely pegged available memory to a few gigabytes.
3. In src/exo/master/placement_utils.py, the placement engine erroneously rejected model shards larger than the tiny reported headroom (ValueError: Node has insufficient memory), or severely under-allocated layers to Apple Silicon devices in mixed clusters.
4. Conversely, when true memory exhaustion occurs and macOS starts thrashing swap space, a simple usage percentage fails to differentiate between harmless file cache and critical memory stall.

Key Changes

1. Memory Pressure Models (src/exo/shared/types/profiling.py):

   • Introduced MemoryPressureLevel(str, Enum) with NORMAL (macOS level 1, comfortable headroom), WARNING (macOS level 2, compression/swap starting), CRITICAL (macOS level 4, thrashing/OOM risk), and UNKNOWN.
   • Added pressure_level and pressure_pct to MemoryUsage.

2. Core Memory Pressure Prober (src/exo/utils/info_gatherer/memory_pressure.py):

   • Darwin / macOS: Directly reads kern.memorystatus_vm_pressure_level via libc sysctlbyname (sub-microsecond execution without subprocess overhead) with CLI fallback.
   • Parses Mach VM statistics via vm_stat to distinguish unpaged wired memory from purgeable/inactive file cache.
   • Computes realistic effective available memory under NORMAL pressure: $\text{ram\_total} - \text{wired\_bytes} - \text{safety\_margin}$ (capped at Apple Silicon Metal unified memory allocation threshold of ~85% total RAM).
   • Throttles available memory to 0 under CRITICAL pressure to prevent OOM kills.
   • Linux & Windows Fallback: Reads Linux PSI (/proc/pressure/memory) or falls back to commit-charge/psutil metrics.

3. Collector Integration (src/exo/utils/info_gatherer/macmon.py, src/exo/utils/info_gatherer/info_gatherer.py):

   • Replaced naive subtraction in MacmonMetrics.from_raw with probe_memory_pressure.
   • Wired pressure telemetry into _monitor_memory_usage for non-macmon platforms.

4. Adaptive Layer Placement (src/exo/master/placement_utils.py, src/exo/master/placement.py):

   • In filter_cycles_by_memory, automatically filters out cycles containing nodes under CRITICAL memory pressure.
   • In _allocate_and_validate_layers, derates layer allocation weights for WARNING nodes (0.6x) to naturally steer workloads toward healthier NORMAL nodes.
   • Enriched error messages in _not_enough_memory and allocation checks with explicit memory pressure levels.

5. Dashboard Visualization (dashboard/src/lib/stores/app.svelte.ts, dashboard/src/lib/components/TopologyGraph.svelte):

   • Preserved pressureLevel and pressurePct across frontend state pipeline.
   • Surfaced memory pressure status (NORMAL, WARNING, CRITICAL) in node tooltips on the topology graph.

6. Unit Tests (src/exo/utils/info_gatherer/tests/test_memory_pressure.py, src/exo/master/tests/test_memory_pressure_placement.py):

   • Added 7 comprehensive unit tests verifying vm_stat parsing, sysctl probing under Normal/Critical levels, macmon serialization, critical node cycle filtering, and warning node layer derating.

Verification

• Unit Tests:

  .venv/Scripts/pytest.exe src/exo/utils/info_gatherer/tests/test_memory_pressure.py src/exo/master/tests/test_memory_pressure_placement.py src/exo/master/tests/test_placement.py -v
  # 40 passed in 0.85s

• Code Linting:

  .venv/Scripts/ruff.exe check src/exo/shared/types/profiling.py src/exo/utils/info_gatherer/memory_pressure.py src/exo/utils/info_gatherer/macmon.py src/exo/master/placement_utils.py src/exo/master/placement.py src/exo/utils/info_gatherer/tests/test_memory_pressure.py src/exo/master/tests/test_memory_pressure_placement.py
  # All checks passed!

Related Issues

• Resolves: Item 13 in TODO.md (Memory pressure instead of memory used.)