Align ORPO with DPO: support iterable and dict eval datasets

[DaoyuanLi2816]

ORPOTrainer prepares its datasets in __init__ with .map(..., num_proc=...), which fails on two inputs that DPOTrainer already accepts:

1. IterableDataset (streaming). IterableDataset.map() does not accept num_proc, so training on a streaming dataset fails immediately:

TypeError: IterableDataset.map() got an unexpected keyword argument 'num_proc'

2. dict eval_dataset. Passing multiple eval datasets as a dict calls .map() on the dict itself:

AttributeError: 'dict' object has no attribute 'map'

Dataset preparation is factored into a _prepare_dataset helper (mirroring DPOTrainer): num_proc is only passed for map-style Datasets, and a dict eval_dataset is prepared per key. For iterable datasets the raw (string) columns are dropped during tokenization, because streaming batches are passed through accelerate's find_batch_size, which only handles tensors. Regular datasets keep those columns since generate_during_eval relies on them (and that path — which needs select / len — isn't available for iterable datasets anyway).

Verification

• Added test_train_with_iterable_dataset (streaming=True) and test_train_with_multiple_eval_dataset (dict eval_dataset). Both fail on main with the errors above and pass with this change.
• Full tests/experimental/test_orpo_trainer.py passes locally (RTX 4080); ruff check / ruff format --check clean.

---

Note

Low Risk
Changes are confined to dataset preparation and Accelerate config defaults in ORPOTrainer, with regression tests; core ORPO loss/training logic is unchanged.

Overview
ORPOTrainer dataset setup is refactored to match DPOTrainer: preparation lives in _prepare_dataset, which only passes num_proc for map-style Datasets so streaming IterableDataset training no longer breaks on .map(..., num_proc=...).

For iterable training, dispatch_batches is forced to False (with a warning if it was True), and tokenization drops raw string columns so Accelerate’s batch handling only sees tensors. Eval now accepts a dict of datasets; each entry is prepared separately instead of calling .map on the dict.

Tests cover streaming train and multi-key eval (eval_data1_loss / eval_data2_loss in logs).

^{Reviewed by Cursor Bugbot for commit 13bdeaf. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit fead16e. Configure here.}