Experiment harness, baselines, and reproducibility artifacts

[kms12425-ctrl]

Background

Scientific evaluation requires strong baselines, reproducible runs, and cost-aware reporting.

Scope

• Implement baseline comparison framework.
• Automate experiment execution and result collection.
• Track memory utilization and cost efficiency in addition to latency metrics.

Deliverables

• Experiment scripts.
• Data summary templates.
• Result archive and reproducibility instructions.

Acceptance Criteria

• Baselines include: FIFO, load-aware, static single-model exclusive allocation, no-spillover, no-memory-aware, full-policy.
• Reports include VRAM utilization, cost per request/token, P95 latency, and SLO violation rate.
• Reproducible results available for paper and final report.

Dependencies

• Epic: Epic: SLO-aware LLM inference service system on SAGE #1506

Notes

Ensures credibility of the research results and acceptance package.