跳转到主要内容

​
配置模板

复制这些模板并根据您的用例进行自定义。

​
LLM 训练模板

​
快速 SFT（小模型）

最适合测试和小数据集： 
task: llm-sft
backend: local
base_model: google/gemma-3-270m
project_name: quick-sft

data:
  path: ./data.jsonl
  train_split: train
  valid_split: null
  chat_template: tokenizer
  column_mapping:
    text_column: text

log: wandb

params:
  epochs: 3
  batch_size: 4
  lr: 3e-5

​
生产 SFT（7B 模型）

用于使用 LoRA 进行全量微调： 
task: llm-sft
backend: local
base_model: meta-llama/Llama-3.2-8B
project_name: production-sft

data:
  path: ./conversations.jsonl
  train_split: train
  valid_split: validation
  chat_template: tokenizer
  column_mapping:
    text_column: text

log: wandb

hub:
  push_to_hub: false

params:
  epochs: 3
  batch_size: 2
  gradient_accumulation: 8
  lr: 3e-5
  warmup_ratio: 0.1
  mixed_precision: bf16

  peft: true
  lora_r: 32
  lora_alpha: 64
  lora_dropout: 0.05

  use_flash_attention_2: true
  packing: true

  save_strategy: steps
  save_steps: 500
  save_total_limit: 1

​
DPO 偏好训练

用于偏好对齐： 
task: llm-dpo
backend: local
base_model: meta-llama/Llama-3.2-1B
project_name: dpo-aligned

data:
  path: ./preferences.jsonl
  train_split: train
  valid_split: null
  chat_template: tokenizer
  column_mapping:
    prompt_text_column: prompt
    text_column: chosen
    rejected_text_column: rejected

log: wandb

params:
  dpo_beta: 0.1
  max_prompt_length: 128
  max_completion_length: null

  epochs: 1
  batch_size: 2
  gradient_accumulation: 4
  lr: 5e-6

  peft: true
  lora_r: 16
  lora_alpha: 32

​
ORPO 训练

结合 SFT + 偏好优化： 
task: llm-orpo
backend: local
base_model: google/gemma-2-2b
project_name: gemma-orpo

data:
  path: ./preferences.jsonl
  train_split: train
  valid_split: null
  chat_template: tokenizer
  column_mapping:
    prompt_text_column: prompt
    text_column: chosen
    rejected_text_column: rejected

log: wandb

params:
  dpo_beta: 0.1
  max_prompt_length: 128
  max_completion_length: null

  epochs: 3
  batch_size: 2
  lr: 5e-5

  peft: true
  lora_r: 16

​
知识蒸馏

用于压缩更大的模型： 
task: llm-sft
backend: local
base_model: google/gemma-3-270m
project_name: distilled-gemma

data:
  path: ./prompts.jsonl
  train_split: train
  valid_split: null
  chat_template: tokenizer
  column_mapping:
    text_column: text

log: wandb

params:
  use_distillation: true
  teacher_model: google/gemma-2-2b
  distill_temperature: 3.0
  distill_alpha: 0.7

  epochs: 5
  batch_size: 8
  lr: 1e-4

​
分类模板

​
文本分类

task: text-classification
backend: local
base_model: bert-base-uncased
project_name: sentiment

data:
  path: ./reviews.csv
  train_split: train
  valid_split: null
  column_mapping:
    text_column: text
    target_column: target

log: wandb

params:
  epochs: 5
  batch_size: 16
  lr: 5e-5

​
多类分类

task: text-classification
backend: local
base_model: microsoft/deberta-v3-base
project_name: categorizer

data:
  path: ./categories.csv
  train_split: train
  valid_split: validation
  column_mapping:
    text_column: content
    target_column: target

log: wandb

params:
  epochs: 10
  batch_size: 8
  lr: 1e-5
  warmup_ratio: 0.1

​
标记分类（NER）

task: token-classification
backend: local
base_model: bert-base-cased
project_name: entity-extractor

data:
  path: ./ner_data.json
  train_split: train
  valid_split: null
  column_mapping:
    tokens_column: tokens
    tags_column: tags

log: wandb

params:
  epochs: 5
  batch_size: 16
  lr: 5e-5

​
视觉模板

​
图像分类

task: image-classification
backend: local
base_model: google/vit-base-patch16-224
project_name: image-classifier

data:
  path: ./images/
  train_split: train
  valid_split: null
  column_mapping:
    image_column: image
    target_column: target

log: wandb

params:
  epochs: 10
  batch_size: 32
  lr: 5e-5

​
目标检测

task: object-detection
backend: local
base_model: facebook/detr-resnet-50
project_name: detector

data:
  path: ./coco_format/
  train_split: train
  valid_split: null
  column_mapping:
    image_column: image
    objects_column: objects

log: wandb

params:
  epochs: 20
  batch_size: 8
  lr: 1e-4

​
视觉-语言模型（VQA）

task: vlm:vqa
backend: local
base_model: google/paligemma-3b-pt-224
project_name: vlm-vqa

data:
  path: ./vqa_data.jsonl
  train_split: train
  valid_split: null
  column_mapping:
    image_column: image
    text_column: text
    prompt_text_column: prompt

log: wandb

params:
  epochs: 3
  batch_size: 2
  lr: 5e-5
  gradient_accumulation: 4

  peft: true
  lora_r: 16
  lora_alpha: 32

​
高级模板

​
超参数扫描

task: llm-sft
backend: local
base_model: google/gemma-3-270m
project_name: sweep-experiment

data:
  path: ./data.jsonl
  train_split: train
  valid_split: validation
  chat_template: tokenizer
  column_mapping:
    text_column: text

log: wandb

params:
  use_sweep: true
  sweep_backend: optuna
  sweep_n_trials: 20
  sweep_metric: eval_loss
  sweep_direction: minimize

  # Base parameters (sweep will vary these)
  epochs: 3
  batch_size: 4
  lr: 3e-5

  peft: true
  lora_r: 16

​
句子转换器

task: sentence-transformers:pair_score
backend: local
base_model: sentence-transformers/all-MiniLM-L6-v2
project_name: embeddings

data:
  path: ./pairs.csv
  train_split: train
  valid_split: null
  column_mapping:
    sentence1_column: sentence1
    sentence2_column: sentence2
    target_column: score

log: wandb

params:
  epochs: 3
  batch_size: 8
  lr: 3e-5

​
用法

将任何模板保存为 config.yaml 并运行： 
aitraining --config config.yaml

​
下一步

YAML 配置

配置文件结构详情

LLM 训练

参数参考