 iomgaa
|
2429dad393
|
feat(batching): unit 粒度切分——pair 整锁 + 单元级分桶 + 非 AR 独立 rng
build_batches 改以 QuestionUnit 为原子调度单元：孪生对 2 题整锁进同一 batch、
按单元级正确性（双向 AND）落 correct/error 桶，不再因 P 对 Q 错被劈或被 FFD 拆箱。
- 非 AR（single）用 random.Random(seed) 复现旧逐题算法确切 draw 序列，AR（pair）
用 _rng_ns(seed,"AR") SHA-256 派生独立流；二者 draw 流互不干扰，故 AR 折叠不改变
非 AR 抽样/洗牌序列——纯非 AR 输入 build_batches 结果与引入 QuestionUnit 前逐字节一致。
- FFD 容量按 unit.size（pair 占 2），round-robin 遇碎片新开 bin 兜底而非报错。
- _select_mixed_by_task_type 分流各跑一次后合并，大类洗牌按 kind 拆分各用对应 rng。
新增黄金测试 test_batching_pair_lock.py 覆盖三条铁律（同 batch / 单元分桶 /
非 AR byte-identical + draw 流独立）；既有 batching 测试全绿。
|
2026-07-15 06:46:28 -04:00 |
|
|
iomgaa
|
c412698cff
|
style(test): 修 loader unit-sampling 测试的 ruff 告警
TC003 将 Path 导入移入 TYPE_CHECKING 块（仅注解使用），
C408 将 base = dict(...) 改为字典字面量。行为不变。
|
2026-07-15 06:30:14 -04:00 |
|
|
iomgaa
|
d6a3107e4e
|
feat(question_gen): loader 按 unit 分层采样 + load_benchmark 读回 pair 字段
stratified_sample 先 build_units 聚合，以 QuestionUnit 为采样原子做
分层/去重/补足/rng.sample，返回前 flatten_units 展开为逐题列表；
size/correct_ratio/min_per_class 均按 unit 计数，单元正确性走成员 AND，
孪生对两题永不被劈开。纯 single 输入下 build_units 1:1 折叠、顺序不变，
rng 消耗与旧逐题实现字节级一致（新增回归测试守护）。
_backfill_per_class candidates 改按 unit 枚举去重；build_units/flatten_units
函数内延迟导入以规避 question_gen<->harness 循环依赖（沿用 adversarial_filter）。
load_benchmark 反序列化补 pair_id/question_role/flip_axis/unit_id 四字段，
用 .get 兼容旧 JSON（缺失退化为 single，unit_id 由 __post_init__ 回填）。
pools._sample_excluding 随之改为透传 flatten_units(candidates) 给已单元化的
stratified_sample（不再用 lone pair-original 代表），行为对 single-only 保持等价。
|
2026-07-15 06:27:41 -04:00 |
|
|
iomgaa
|
ddb9a44f75
|
feat(pools): 三池切分以 unit 为原子，孪生对同池不被拆散
build_pools/_sample_excluding 与 PerCategoryPoolStrategy._split_one_category/
build_incremental 两条切分路径均改为以 QuestionUnit 为采样原子：progressive
exclusion 互斥集合与 train/val 分层划分都按 unit_id 计数（pair 计 1 个 unit），
命中单元整体展开，AR 孪生对两题永不落入不同池/split。
复用 app.harness.question_units 的 build_units/flatten_units，不重写分组逻辑。
single-only 输入下 unit 与 question 一一对应、rng 消耗量不变，采样与划分结果
与逐题口径完全一致；抽出 _unit_correct/_assert_correctness_complete 两个 helper
将 _split_one_category 复杂度压回基线以下。
新增 tests/unit/test_pools_pair_atomic.py 覆盖两条路径的 pair 原子性回归。
|
2026-07-15 06:09:14 -04:00 |
|
|
iomgaa
|
5ef5f2b8b7
|
refactor: extract _assemble_pair to lower build_units complexity to B
将 pair 分组的数量/角色校验与组装下沉到 _assemble_pair 私有 helper，
build_units 主体只做分组+派发，radon 复杂度从 C(11) 降到 A(4)。
fail-fast 语义、公共函数签名、稳定顺序均不变。
|
2026-07-15 05:54:08 -04:00 |
|
|
iomgaa
|
bef46636fe
|
feat: add question_units helper as pair contract entry point
build_units/flatten_units/validate_units/unit_correctness——pair 契约唯一入口。
build_units 对孤儿/超员/角色缺失重复 fail-fast raise ValueError（防 next 静默
StopIteration）；unit_correctness 走 per_q[qid] KeyError 防静默兜底。
|
2026-07-15 05:52:30 -04:00 |
|
|
iomgaa
|
7ef9b99217
|
feat: add QuestionUnit entity and GeneratedQuestion pair fields
新增 QuestionUnit 契约实体（kind/unit_id/task_type/questions/unit_hash + size/from_single/from_pair），
作为 AR pair 孪生对贯穿评测/训练 harness 的最小调度单元。
GeneratedQuestion 追加 unit_id/pair_id/question_role/flip_axis 四字段，
__post_init__ 回填 unit_id，默认值保证现有非 AR single 题构造点行为不变。
|
2026-07-15 05:45:34 -04:00 |
|
|
iomgaa
|
0fe1c96393
|
docs: add question-gen v3 construction-paradigm design and phase1 plan
Complete v3 planning: construction-first paradigm (frame-perception grounded fact extraction + 4-family independent judges + 6-layer verification + QuestionUnit contract), adversarial audit, paradigm-shift finding, real-data spike validation, logging schema, and phase1 contract implementation plan.
|
2026-07-15 05:41:10 -04:00 |
|
|
iomgaa
|
d9f7dee2df
|
docs: revise question-gen v2 design per Codex review (pair-unit contract, AR-only gate, phasing)
|
2026-07-15 01:21:28 -04:00 |
|
|
iomgaa
|
7949f16906
|
docs: add question-gen v2 grounded-contrastive design
|
2026-07-15 01:10:00 -04:00 |
|
|
iomgaa
|
fd96f906c2
|
docs: add question-gen v2 diagnosis and locked redesign direction
|
2026-07-15 00:50:29 -04:00 |
|
|
iomgaa
|
49b7be78ae
|
fix: derive OUTPUT from config output_dir so CONFIG override counts correctly
|
2026-07-14 21:12:48 -04:00 |
|
|
iomgaa
|
d84f9bf777
|
chore: harden generate_ar30.sh (drop no-op mock, add zero-gain early stop)
|
2026-07-14 21:08:00 -04:00 |
|
|
iomgaa
|
b1c1bf7aac
|
fix: isolate selector VLM exceptions at slot level (no batch crash)
|
2026-07-14 21:08:00 -04:00 |
|
|
iomgaa
|
441a0aa6c3
|
chore: format Phase B modules and register Phase B plan in wiki
|
2026-07-14 17:11:04 -04:00 |
|
|
iomgaa
|
f36eb66c18
|
fix: tolerate backfill under-delivery, fix predict None-fill, drop dead session_id
|
2026-07-14 17:07:55 -04:00 |
|
|
iomgaa
|
1d222d9f18
|
feat: wire real agent runner, backfill assembly and adversarial-filter CLI
|
2026-07-14 16:58:32 -04:00 |
|
|
iomgaa
|
73d0bb9190
|
feat: add adversarial round loop with backfill iteration and difficulty report
|
2026-07-14 16:40:05 -04:00 |
|
|
iomgaa
|
c1565a01c2
|
test: assert stage/verdict/pair_id and Q-refusal flip_skipped path
|
2026-07-14 16:33:09 -04:00 |
|
|
iomgaa
|
15aee0cfc1
|
feat: add flip gate reusing P prediction and mirror-Q agent run
|
2026-07-14 16:26:51 -04:00 |
|
|
iomgaa
|
8731e448fe
|
fix: harden mirror material rebuild (empty/invalid frames, dedup, observability)
|
2026-07-14 16:16:38 -04:00 |
|
|
iomgaa
|
4af99b700b
|
feat: add mirror question generation with canonical distinctness check
|
2026-07-14 16:08:13 -04:00 |
|
|
iomgaa
|
d4e9852864
|
feat: add cheater gate with resume-safe survivor recovery
|
2026-07-14 16:00:14 -04:00 |
|
|
iomgaa
|
24ed7ca322
|
fix: harden canonical_answer_text and align decision-core signatures to spec
|
2026-07-14 15:55:36 -04:00 |
|
|
iomgaa
|
c109f2257a
|
feat: add pure decision core for adversarial filter (hash/fingerprint/canonical/flip)
|
2026-07-14 15:51:28 -04:00 |
|
|
iomgaa
|
334fbbc94d
|
feat: add AdversarialFilterConfig for Phase B post-hoc filter layer
|
2026-07-14 15:47:21 -04:00 |
|
|
iomgaa
|
ac115d96fb
|
test: cover backfill params end-to-end (mutation isolation + seq_offset)
|
2026-07-14 15:44:43 -04:00 |
|
|
iomgaa
|
8b9e8aa19f
|
feat: add optional backfill params to run_pipeline_v2
|
2026-07-14 15:39:57 -04:00 |
|
|
iomgaa
|
d77cbc95eb
|
feat: add adversarial_verdicts table with resume and terminal-verdict query
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-07-14 15:35:11 -04:00 |
|
|
iomgaa
|
e41a2b0d08
|
test: guard non-AR flip isolation and AR flip-capable set
|
2026-07-14 15:30:18 -04:00 |
|
|
iomgaa
|
f12dd7e559
|
feat: add supports_flip/flip_axis to SubPattern for Phase B flip gate
|
2026-07-14 15:27:21 -04:00 |
|
|
iomgaa
|
841112c6af
|
docs: revise Phase B plan per Codex review (C1-C3,I1-I6,M1-M3)
|
2026-07-14 15:24:44 -04:00 |
|
|
iomgaa
|
abc9c097d2
|
docs: add Phase B adversarial question-gen implementation plan
|
2026-07-14 15:05:36 -04:00 |
|
|
iomgaa
|
96e314c3a0
|
fix: mark item rejected on selector_error for consistent bookkeeping
selector_error 分支（捕获 ValueError/FileNotFoundError）此前只返回 reason，
未 mark_item_rejected，导致 Phase 3 已 record 的 pending attempt 行永远停在
pending；而 hard-fail 分支会标记 rejected。两条失败路径落库风格现统一为
mark_item_rejected（异常路径无 outcome/observation，故不写 selector_scores）。
补单测 test_apply_grounded_selector_marks_rejected_on_error 守卫该路径。
|
2026-07-14 14:49:41 -04:00 |
|
|
iomgaa
|
46ac848176
|
fix: converge non-numeric selector scores to ValueError
|
2026-07-14 14:47:21 -04:00 |
|
|
iomgaa
|
eaab865d84
|
docs: register Phase A plan in research wiki
|
2026-07-14 14:41:53 -04:00 |
|
|
iomgaa
|
e64c26e578
|
style: format distractor_selector
|
2026-07-14 14:41:21 -04:00 |
|
|
iomgaa
|
76f719018c
|
feat: loosen multi_true gate to qualifier-scoped correctness
|
2026-07-14 14:33:22 -04:00 |
|
|
iomgaa
|
b1f15ddb3a
|
fix: keep cross_segment rule single-dimension, guard no-absent-events
|
2026-07-14 14:31:30 -04:00 |
|
|
iomgaa
|
58278c6de4
|
feat: enforce single-dimension counterfactual in AR distractor rules
|
2026-07-14 14:28:32 -04:00 |
|
|
iomgaa
|
ea38e4717b
|
refactor: extract grounded selector application into helper
将 Phase 3.5 的 grounded selector 逻辑抽成模块级 _apply_grounded_selector，
内部完成策略门控 / 异常捕获 / observation 落库 / hard-fail 拒绝，调用点仅剩
单一失败分支（selector_reason → 重出）。行为不变，测试全绿。
顺带移除 resample_video_interval 死子表达式（硬编码 1，(attempt-1)%1==0 恒真），
使 _process_one_slot 圈复杂度回落至 Task 6 前的 D(25)。_apply_grounded_selector
自身为 A(4)，均满足质量门。
|
2026-07-14 14:22:30 -04:00 |
|
|
iomgaa
|
b13eab0659
|
feat: wire grounded selector into AR slot processing
将 Task 5 的 grounded selector 织入 AR 出题路径（Phase 3.5，位于
record_item 与 postprocess 之间），仅在 strategy.uses_grounded_selector
为真时进入。observation 始终落库（含 hard-fail），硬失败走重出。
PipelineConfig 新增 candidate_pool_size/selector_delta_low/
selector_delta_high 三参，YAML 与 CLI seed override 同步。
|
2026-07-14 14:15:50 -04:00 |
|
|
iomgaa
|
d0194f5840
|
fix: degrade distractor pool gracefully on malformed VLM response
|
2026-07-14 14:11:13 -04:00 |
|
|
iomgaa
|
8a54055d02
|
feat: add grounded distractor selector with visual scoring
|
2026-07-14 14:06:07 -04:00 |
|
|
iomgaa
|
207e834f30
|
feat: add selector_scores observation column to question_gen_items
|
2026-07-14 13:59:29 -04:00 |
|
|
iomgaa
|
3f984acc18
|
feat: add uses_grounded_selector strategy switch (AR only)
|
2026-07-14 13:53:17 -04:00 |
|
|
iomgaa
|
2608a3841f
|
style: reformat sub_pattern round-trip tests
|
2026-07-14 13:51:51 -04:00 |
|
|
iomgaa
|
e68e4b7d57
|
fix: restore sub_pattern when loading benchmark JSON
|
2026-07-14 13:51:05 -04:00 |
|
|
iomgaa
|
111c88488f
|
style: format sub_pattern test file
|
2026-07-14 13:46:25 -04:00 |
|
|
iomgaa
|
ae0a718f67
|
feat: thread and persist sub_pattern into accepted questions
|
2026-07-14 13:45:31 -04:00 |
|