重构: load_sft_dataset 改吃散装参数(磨平接口回看记录的毛刺)

深模块修正:本函数只用 5 个字段,却索要整个 SFTConfig——层 1 无痛,但诊断脚本
被迫伪造 output_dir(4 处 /tmp/diag、outputs/_unused),层 2 更因 DistillConfig
无 teacher_completions_path 而无法复用。改收 dataset_path/split/subset_size/seed/
teacher_completions_path 五个散装参数(接口终于比实现轻)。

- data.py: 签名改散装参数;移除 TYPE_CHECKING 的 SFTConfig 依赖
- train_sft / diag_loss_probe / diag_collator: 仍持 SFTConfig(喂 collator),改调用点
- diag_generate / generate_teacher_completions: 只为 load 而造 config,直接丢弃、
  去掉伪造 output_dir,改传字面量
- 为 U5 层 2 训练脚本能直接 load_sft_dataset(distill_cfg 的字段) 铺路

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-19 05:21:16 -04:00
parent 0ca60ea93f
commit 404abc22bf
6 changed files with 50 additions and 34 deletions
+3 -8
View File
@@ -7,19 +7,14 @@
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from ars_opd.configs import SFTConfig
from ars_opd.data import load_sft_dataset
MODEL_DIR = "/data/zym/outputs/sft_qwen3-0.6b_dapo1k" # 正式 1 epoch 的产物
cfg = SFTConfig(
dataset_path="data/dapo-math-17k-unique.parquet",
output_dir="/tmp/diag",
subset_size=1000,
seed=42,
# 不挂 teacher 解答:只取题目做推理输入
# 不挂 teacher 解答(teacher_completions_path 缺省):只取题目做推理输入
ds = load_sft_dataset(
"data/dapo-math-17k-unique.parquet", subset_size=1000, seed=42
)
ds = load_sft_dataset(cfg)
tok = AutoTokenizer.from_pretrained(MODEL_DIR)
model = AutoModelForCausalLM.from_pretrained(MODEL_DIR, dtype=torch.float32)