Skip to content

Conversation

@appleparan
Copy link

To enable kbit quantization, gradient_checkpointing must be passed into TrainingArguments

  1. I remove library version restriction.
!pip install -q accelerate peft bitsandbytes transformers trl[quantization]
  1. Also, I pass gradient_checkpointing into TrainingArguments. (trl>=0.7.2)

# Set training parameters
training_arguments = TrainingArguments(
    output_dir=output_dir,
    num_train_epochs=num_train_epochs,
    per_device_train_batch_size=per_device_train_batch_size,
    gradient_accumulation_steps=gradient_accumulation_steps,
    gradient_checkpointing=gradient_checkpointing,
    optim=optim,
    save_steps=save_steps,
    logging_steps=logging_steps,
    learning_rate=learning_rate,
    weight_decay=weight_decay,
    fp16=fp16,
    bf16=bf16,
    max_grad_norm=max_grad_norm,
    max_steps=max_steps,
    warmup_ratio=warmup_ratio,
    group_by_length=group_by_length,
    lr_scheduler_type=lr_scheduler_type,
    report_to="tensorboard"
)

* To enable kbit quantization, gradient_checkpointing must be passed into TrainingArguments
@appleparan appleparan changed the title Update library of Fine-tune Llama 2 Update Fine-tune Llama 2 libraries Jan 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant