Unsloth Integration
Unsloth provides optimized training for LoRA fine-tuning, significantly reducing training time and memory usage.
Requirements
| Requirement | Details |
|---|
| Installation | pip install unsloth |
| Supported Trainers | default, sft only |
| Supported Models | llama, mistral, gemma, qwen2 |
| Platform | Linux recommended |
Unsloth only works with SFT training (--trainer sft or --trainer default). DPO, ORPO, PPO, and other trainers are not supported.
Supported Model Architectures
Unsloth is optimized for specific model families:
| Architecture | Example Models |
|---|
llama | Llama 2, Llama 3, Llama 3.1, Llama 3.2 |
mistral | Mistral 7B, Mistral Nemo |
gemma | Gemma, Gemma 2 |
qwen2 | Qwen 2, Qwen 2.5 |
Quick Start
aitraining llm --train \
--model meta-llama/Llama-3.2-1B \
--data-path ./data \
--project-name fast-model \
--trainer sft \
--unsloth \
--peft \
--lora-r 16
Parameters
| Parameter | CLI Flag | Default | Description |
|---|
unsloth | --unsloth | False | Enable Unsloth for faster training |
use_sharegpt_mapping | --use-sharegpt-mapping | False | Use Unsloth’s ShareGPT mapping instead of converting |
Python API
from autotrain.trainers.clm.params import LLMTrainingParams
from autotrain.project import AutoTrainProject
params = LLMTrainingParams(
model="meta-llama/Llama-3.2-1B",
data_path="./data",
project_name="fast-model",
trainer="sft",
unsloth=True,
peft=True,
lora_r=16,
lora_alpha=32,
epochs=3,
batch_size=4,
)
project = AutoTrainProject(params=params, backend="local", process=True)
project.create()
With Quantization
Unsloth works with int4 and int8 quantization for reduced memory usage:
aitraining llm --train \
--model meta-llama/Llama-3.2-3B \
--data-path ./data \
--project-name quantized-model \
--trainer sft \
--unsloth \
--peft \
--quantization int4 \
--lora-r 16
How It Works
When Unsloth is enabled and requirements are met:
- Uses
FastLanguageModel from Unsloth library for optimized model loading
- Applies optimized gradient checkpointing (
use_gradient_checkpointing="unsloth")
- Automatically configures LoRA target modules:
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Integrates with PEFT for efficient adapter training
Fallback Behavior
If Unsloth cannot be used, training continues with standard transformers/PEFT:
- Unsloth not installed: Warning logged, continues without Unsloth
- Unsupported model type: Warning logged, continues without Unsloth
- Unsupported trainer: Unsloth not applied (only SFT supported)
WARNING: Unsloth not available, continuing without it...
ShareGPT Mapping
Use --use-sharegpt-mapping to preserve ShareGPT format instead of converting:
aitraining llm --train \
--model meta-llama/Llama-3.2-1B \
--data-path ./sharegpt_data.json \
--project-name model \
--trainer sft \
--unsloth \
--use-sharegpt-mapping \
--peft
Next Steps
