Skip to content
@responsible-robot-benchmark

Responsible Robot Benchmark

Benchmarking risk-aware task-level decision making for robotic manipulation with large multimodal models.

ResponsibleRobotBench

Benchmarking risk-aware task-level decision making for robotic manipulation with large multimodal models.

Project website · Open-source code · Paper and appendix

Benchmark design. We introduce ResponsibleRobotBench, a benchmark of 23 manipulation task configurations spanning conditionally safe tasks, hazard-free controls, and inherently unsafe goals. Task goals and safety conditions are specified and checked separately, with task variants varying hazard presence and grasping requirements.

Empirical analysis. We evaluate five LMMs through episode-level analyses linking hazard predictions, mitigation choices, action prerequisites, and outcomes. The results reveal failures after correct hazard recognition, divergence between terminal safety and safe completion, and differences between assistance availability and recovery after actual help requests.

Open-source implementation. We release the benchmark implementation, a modular LMM planning system supporting skill, pose, and robot-API code interfaces with 12 input configurations, and tools for collecting motion-planner demonstrations.

Lei Zhang, Ju Dong, Kaixin Bai, Minheng Ni, Yu Fu, Zoltán-Csaba Márton, Zhaopeng Chen, and Jianwei Zhang.

Pinned Loading

  1. responsible-robot-benchmark.github.io responsible-robot-benchmark.github.io Public

    ResponsibleRobotBench project website: paper, benchmark, simulation videos, and real-robot demonstrations.

    HTML

  2. ResponsibleRobotBench ResponsibleRobotBench Public

    ResponsibleRobotBench: 23 robotic manipulation tasks for risk-aware planning and execution with Skill, Pose, and Code interfaces.

    Python 1

Repositories

Showing 3 of 3 repositories

Top languages

Loading…

Most used topics

Loading…