Backend container registration and runtime state model in SAGE

[kms12425-ctrl]

Background

SAGE must model heterogeneous inference backends and co-resident models as schedulable resources.

Scope

• Abstract vLLM-HUST instances as backend containers.
• Model per-device and per-model runtime state for shared GPU/NPU usage.
• Implement health checks and metrics normalization across heterogeneous nodes.

Deliverables

• Backend registration module.
• Runtime state snapshot API.
• Observability dashboard or equivalent state view.

Acceptance Criteria

• Maintain: backend_id, node_id, accelerator_type, model_family, precision, parallelism, resident_models, health, schedulable, runtime_metrics.
• Expose per-device and per-model snapshots suitable for multi-model scheduling.
• Normalize GPU/NPU state so it can participate in one scheduling pipeline.

Dependencies

• Epic: Epic: SLO-aware LLM inference service system on SAGE #1506

Notes

Foundational for global scheduling and memory-aware admission.