feat(batching): unit 粒度切分——pair 整锁 + 单元级分桶 + 非 AR 独立 rng
build_batches 改以 QuestionUnit 为原子调度单元:孪生对 2 题整锁进同一 batch、 按单元级正确性(双向 AND)落 correct/error 桶,不再因 P 对 Q 错被劈或被 FFD 拆箱。 - 非 AR(single)用 random.Random(seed) 复现旧逐题算法确切 draw 序列,AR(pair) 用 _rng_ns(seed,"AR") SHA-256 派生独立流;二者 draw 流互不干扰,故 AR 折叠不改变 非 AR 抽样/洗牌序列——纯非 AR 输入 build_batches 结果与引入 QuestionUnit 前逐字节一致。 - FFD 容量按 unit.size(pair 占 2),round-robin 遇碎片新开 bin 兜底而非报错。 - _select_mixed_by_task_type 分流各跑一次后合并,大类洗牌按 kind 拆分各用对应 rng。 新增黄金测试 test_batching_pair_lock.py 覆盖三条铁律(同 batch / 单元分桶 / 非 AR byte-identical + draw 流独立);既有 batching 测试全绿。
This commit is contained in:
@@ -53,7 +53,7 @@ class _FakeInferenceResult:
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _FakeQuestion:
|
||||
"""GeneratedQuestion 替身。"""
|
||||
"""GeneratedQuestion 替身(含 pair 契约字段,供 build_units 聚合)。"""
|
||||
|
||||
question_id: str
|
||||
video_id: str = "v1"
|
||||
@@ -63,6 +63,15 @@ class _FakeQuestion:
|
||||
answer: str = "A"
|
||||
source_nodes: tuple = ()
|
||||
difficulty: str = "medium"
|
||||
pair_id: str | None = None
|
||||
question_role: str = "single"
|
||||
unit_id: str = ""
|
||||
flip_axis: str | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""缺省 unit_id 回填为 pair_id 或 question_id,对齐真实 GeneratedQuestion。"""
|
||||
if not self.unit_id:
|
||||
object.__setattr__(self, "unit_id", self.pair_id or self.question_id)
|
||||
|
||||
|
||||
@dataclass
|
||||
|
||||
Reference in New Issue
Block a user