Skip to content

Add UnicBench benchmark support for Bagel#285

Open
hongsexiaotanhua wants to merge 5 commits intoByteDance-Seed:mainfrom
hongsexiaotanhua:benchmark/add-unicbench-test
Open

Add UnicBench benchmark support for Bagel#285
hongsexiaotanhua wants to merge 5 commits intoByteDance-Seed:mainfrom
hongsexiaotanhua:benchmark/add-unicbench-test

Conversation

@hongsexiaotanhua
Copy link
Copy Markdown

Summary

This PR adds native support for UnicBench, a newly proposed benchmark for image editing that covers a broader range of editing categories with more fine-grained and accurate evaluation dimensions.

Changes

  • New evaluation module: eval/gen/unicbench/
    Implements core evaluation logic for UnicBench.
  • Image generation script: eval/gen/gen_images_mp_unicbench.py
    Generates edited images for UnicBench evaluation.
  • End-to-end evaluation script: scripts/eval/run_unicbench.sh
    Provides a complete, reproducible pipeline to run UnicBench.
  • Documentation update: EVAL.md
    Adds UnicBench data preparation instructions, evaluation usage, and benchmark results.

Motivation

UnicBench is designed to provide a more comprehensive and precise evaluation protocol for image editing models, enabling better analysis across diverse and complex editing scenarios. Integrating UnicBench into BAGEL allows users to directly evaluate models under this unified and extensible benchmark.

Notes

  • No existing evaluation logic is modified.
  • The added components follow the current evaluation and scripting conventions in the repository.

Please let me know if any adjustments or additional integrations are preferred.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant