层1: 修复远程 sanity OOM——per_device batch 8→2、累积 2→8(全局 64 不变)

根因:150k 大词表下显存大头是 (B,T,V) logits 链(fp32 ~20G@B=8)与逐层激活,
均正比于 B 而与 0.6B 参数量无关。docs/02 §2.6 旧显存估算勘误入档;
train_sft.sh 加 expandable_segments 防碎片。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-18 09:21:38 -04:00
parent 68216ace22
commit a0faec0df7
5 changed files with 14 additions and 6 deletions
+2 -2
View File
@@ -35,8 +35,8 @@ FULL = SFTConfig(
max_prompt_length=1024,
enable_thinking=False,
learning_rate=2e-5,
per_device_train_batch_size=8,
gradient_accumulation_steps=2, # 全局 batch = 8 × 4 卡 × 2 = 64
per_device_train_batch_size=2, # B=8 曾爆 80G:大头是 (B,T,V) logits 链与激活,见 SFTConfig 注释
gradient_accumulation_steps=8, # 全局 batch = 2 × 4 卡 × 8 = 64
num_train_epochs=1,
max_steps=-1,
lr_scheduler_type="linear",