Skip to content

Create create_data_blend utility #1876

Description

@danielkorzekwa

Detailed description of the requested feature

Create an examples/dataset/create_data_blend.py utility to enable preparing data blends for distillation like described here:

Input yaml file with data blend configuration:

YAML::

    tokenizer: /models/Qwen3-8B
    output_dir: /datasets/qwen3-blend
    target_tokens: 1000000
    sources:
      - hf_dataset: nvidia/Nemotron-Pretraining-SFT-v1
        config: Nemotron-SFT-General
        split: train
        content_field: text
        weight: 60
      - hf_dataset: nvidia/Nemotron-SFT-Competitive-Programming-v2
        files:
          - data/competitive_programming_python_00.jsonl
        content_field: messages
        weight: 40

Metadata

Metadata

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions