README's --agent openhands command is broken with current openhands-ai (1.x)

[zhaoyb1990]

Summary

The README's quick-start command for the OpenHands scaffold:

harbor run --path data/opt-3.0.0-roadmap --agent openhands --model ...

fails today because the upstream openhands-ai package (1.x series, the current PyPI latest) no longer exposes the openhands.core.main module that harbor's installed/openhands adapter uses for verification and agent invocation.

Reproduction

Fresh environment, default args from README:

uv tool install harbor          # 0.6.1 today
git clone https://github.com/UniPat-AI/RoadmapBench.git
cd RoadmapBench

export OPENAI_API_KEY=...
export OPENAI_BASE_URL=...

harbor run \
    --path test-assets/rat-0.21.0-roadmap \
    --agent openhands \
    --model "openai/your-model" \
    --n-attempts 1 --n-concurrent 1 --yes \
    --jobs-dir results/

Result (after ~3-5 minutes of cold install):

NonZeroAgentExitCodeError: Command failed (exit 1)
stdout: ... Installed 352 packages in 139ms ...
stderr: /opt/openhands-venv/bin/python: No module named openhands.core.main

Root cause

harbor/agents/installed/openhands.py hardcodes two operations that assume openhands-ai ships an openhands.core.main runpy entry point:

# verification (install step)
"/opt/openhands-venv/bin/python -m openhands.core.main --version"

# runtime (run step)
"/opt/openhands-venv/bin/python -m openhands.core.main"

This was true in openhands-ai 0.x but the package was restructured around 1.x — the SDK and tooling moved into a separate openhands-sdk package, and the monolithic CLI entry was removed. openhands-ai 1.7.0 (today's PyPI latest) contains openhands.agent_server, openhands.sdk, openhands.tools, openhands.server — but no openhands.core.

Confirmed:

• harbor v0.6.1 (RoadmapBench README's expected default install): broken
• harbor v0.6.5 (released same day as RoadmapBench): broken
• harbor main (today): broken — adapter has not been updated since 2026-03-24

Pinning a known-historical version doesn't trivially help: openhands-ai 0.9.x is yanked / has broken transitive deps (e.g. e2b<0.18.0 no longer resolvable on PyPI); 0.10.0+ requires Python ≥ 3.12 but is on the post-restructure side of the boundary.

Impact

OpenHands rows in Table 2 (Resolved %, Completion Score, Avg. Turns, Output Tok.) cannot be reproduced as documented. We verified across several models that the documented command fails identically.

(The --agent terminus-2 scaffold path is unaffected and works correctly.)

Requests

To restore reproducibility, could the maintainers please share:

1. The exact openhands-ai version used to produce the Table 2 OpenHands baselines.
2. The harbor version / commit hash used.
3. The Python version inside the task containers.
4. Model snapshot identifiers (e.g. claude-opus-4-7-20260420 rather than Claude-Opus-4.7).

Environment

• harbor: 0.6.1 (also tested 0.6.5, 0.8.0, main)
• openhands-ai: 1.7.0 (latest)
• Python: 3.13 (uv-managed inside container)
• Base image: znpt/roadmapbench-rat-0.21.0-roadmap (DockerHub)
• OS: Linux x86_64
• Date: 2026-05-24

[Zn-Pt]

Apologies for the delayed response, and thanks for the detailed report.

You are right that the OpenHands scaffold is sensitive to the openhands-ai and Python versions. We have updated the README to pin the verified runtime configuration for OpenHands:

• harbor: 0.18.0
• openhands-ai: 1.6.0
• Python for the OpenHands runtime: 3.12

The key change is to pass these through Harbor agent kwargs:

harbor run \
  --path data/opt-3.0.0-roadmap \
  --agent openhands \
  --model "your-model-name" \
  --n-attempts 1 \
  --n-concurrent 10 \
  --yes \
  --jobs-dir results/openhands \
  --ae OPENAI_API_KEY=your-api-key \
  --ae OPENAI_BASE_URL=https://your-api-endpoint/v1 \
  --ak version=1.6.0 \
  --ak python_version=3.12

Without these pins, Harbor may install a newer openhands-ai package and/or use Python 3.13, which can reproduce the No module named openhands.core.main failure you observed. In our recent verification, openhands-ai==1.6.0 with Python 3.12 works with Harbor's OpenHands adapter.

Thanks again for catching this. The README now includes this compatibility note.