重构: load_sft_dataset 改吃散装参数(磨平接口回看记录的毛刺)
深模块修正:本函数只用 5 个字段,却索要整个 SFTConfig——层 1 无痛,但诊断脚本 被迫伪造 output_dir(4 处 /tmp/diag、outputs/_unused),层 2 更因 DistillConfig 无 teacher_completions_path 而无法复用。改收 dataset_path/split/subset_size/seed/ teacher_completions_path 五个散装参数(接口终于比实现轻)。 - data.py: 签名改散装参数;移除 TYPE_CHECKING 的 SFTConfig 依赖 - train_sft / diag_loss_probe / diag_collator: 仍持 SFTConfig(喂 collator),改调用点 - diag_generate / generate_teacher_completions: 只为 load 而造 config,直接丢弃、 去掉伪造 output_dir,改传字面量 - 为 U5 层 2 训练脚本能直接 load_sft_dataset(distill_cfg 的字段) 铺路 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -46,7 +46,13 @@ def main() -> None:
|
||||
seed=42,
|
||||
teacher_completions_path=CACHE,
|
||||
)
|
||||
ds = load_sft_dataset(cfg)
|
||||
ds = load_sft_dataset(
|
||||
cfg.dataset_path,
|
||||
cfg.dataset_split,
|
||||
cfg.subset_size,
|
||||
cfg.seed,
|
||||
cfg.teacher_completions_path,
|
||||
)
|
||||
msgs = ds[0]["messages"]
|
||||
completion_text = msgs[-1]["content"]
|
||||
|
||||
|
||||
Reference in New Issue
Block a user