feat(spartqa): add SpartQA spatial-reasoning benchmark

[pachmu]

SpartQA — spatial-reasoning benchmark

Adds a native gym port of SpartQA
— a spatial-reasoning QA benchmark where the model is shown a scene-description
query and must return the matching answer phrase.

What it does

Every task is single-turn generation — no agent fork, no judge model.
verify() is pure, rule-based scoring (stdlib re / string only).

The model is prompted to end its response with Final answer: <phrase>.
verify() extracts that phrase, normalizes it, and compares against every
accepted phrase in all_targets:

Outcome	reward	exact
Extracted phrase == any accepted phrase (normalized)	1.0	true
Extracted phrase contains any accepted phrase (normalized)	1.0	false
No match	0.0	false
Empty / unparsed output	0.0	false

compute_metrics's mean_reward equals corpus-level accuracy (the headline
metric). exact_match_rate and parse_rate are reported alongside for
downstream inspection.

Scoring details

_extract_answer handles:

• <think>…</think> reasoning-block stripping (reasoning models)
• Final answer: … / Selected answer: … / Answer: … patterns with
  fallback to last non-empty line when no explicit marker is present
• Markdown artifacts (**, `, *, bullet prefixes)

Field placement and all_targets survival

The primary accepted answer phrase lives at target (a top-level scalar,
forwarded intact by the nemo-evaluator gym://...protocol=native driver). The
full accepted set (all_targets) is a list and dropped by the driver — it
is mirrored into verifier_metadata (forwarded intact), so verify() always
sees every phrase via the fallback chain:

body.all_targets → body.verifier_metadata["all_targets"] → [body.target]

Contents

• resources_servers/spartqa/app.py — SpartqaResourcesServer (/verify +
  compute_metrics / get_key_metrics) with answer extraction, normalization,
  and multi-target scoring.
• prepare_spartqa.py — joins mteb/SpartQA (HF datasets) into
  data/spartqa_test.jsonl. Committed data/example.jsonl is the 5-row smoke
  set.
• configs/spartqa.yaml — server + simple_agent wiring for local
  eval/training.
• configs/spartqa_serve.yaml — serve-only scorer for nemo-evaluator via the
  gym:// adapter (rule-based verify needs no model on the gym side).
• requirements.txt — no extra runtime deps; -e nemo-gym[dev] editable
  install only.
• tests/test_app.py — 25 tests over _extract_answer, _strip_reasoning,
  _normalize, verify() (exact / contains / empty / multi-target /
  metadata-fallback), compute_metrics, get_key_metrics, _response_text,
  and acceptance parity against every example.jsonl row.

Testing

gym env test --resources-server spartqa
ruff check resources_servers/spartqa/app.py resources_servers/spartqa/prepare_spartqa.py resources_servers/spartqa/tests/

Notes

• No API keys required — all scoring is rule-based.
• Reasoning models — verify() strips leading <think>…</think> blocks
  before extracting the answer, so thinking-model outputs score correctly.
• verified: false — not yet baselined. Reward-profiling across an
  open-instruct / open-thinking / closed-source model suite is a follow-up
  before flipping to verified: true.
• Heavy dataset-prep deps (datasets) are opt-in and not in requirements.txt
  — install only when running prepare_spartqa.py.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.