Fix prompt path resolution when CWD differs from project root

[simonrosenberg]

Summary

• Fixes prompt template path resolution in all 6 run_infer.py files. The old code used p.relative_to(Path.cwd()) which raises ValueError when the process is launched from a directory other than the project root (e.g. when running as an installed package via NeMo Evaluator).
• Extracts duplicated logic into a shared add_prompt_path_argument() utility in benchmarks/utils/args_parser.py, replacing ~14 identical lines in each file with a single function call.

Extracted from #455 (NeMo Evaluator Integration).

Files changed

File	Change
benchmarks/utils/args_parser.py	New add_prompt_path_argument(parser, caller_file) function
benchmarks/swebench/run_infer.py	Use shared utility
benchmarks/swtbench/run_infer.py	Use shared utility
benchmarks/swebenchmultimodal/run_infer.py	Use shared utility
benchmarks/swefficiency/run_infer.py	Use shared utility
benchmarks/multiswebench/run_infer.py	Use shared utility
benchmarks/commit0/run_infer.py	Use shared utility

Before / After

# Before (broken when CWD != project root):
prompt_dir = (Path(__file__).parent / "prompts").resolve()
choices = [str(p.relative_to(Path.cwd())) for p in prompt_dir.glob("*.j2")]
default_prompt_path = prompt_dir / "default.j2"
assert default_prompt_path.exists(), ...
parser.add_argument("--prompt-path", ..., choices=choices, default=str(default_prompt_path))

# After (single line per benchmark):
parser = get_parser()
add_prompt_path_argument(parser, __file__)

Validation

All benchmarks pass on the nemo-evaluator branch (which includes this change):

• commit0
• gaia
• swebench
• swebenchmultimodal
• swtbench

Test plan

• Verify --prompt-path argument still works when running from the project root
• Verify --prompt-path argument works when running from a different directory (the bug this fixes)
• Verify --help still lists available prompt templates

🤖 Generated with Claude Code

[all-hands-bot]

🟡 Acceptable - Solves Real Problem, but with Trade-offs

Good: Fixes actual bug (ValueError when CWD ≠ project root) and eliminates code duplication.

Issues:

1. UX regression: Help shows ugly absolute paths instead of clean relative ones
2. Breaking change risk: Existing users passing relative paths will fail validation
3. Missing tests: Bug fix without regression tests

See inline comments for specifics.

[all-hands-bot]

✅ Clean refactor that fixes a real bug

Eliminates 84 lines of duplication while fixing ValueError when CWD ≠ project root. Path resolution logic is pragmatic, tests cover real scenarios (not mocks), and backwards compatibility is preserved.

Previous review concerns (UX regression, breaking changes) were properly addressed. Ready to ship.