14 Commits

Author SHA1 Message Date
iomgaa f36eb66c18 fix: tolerate backfill under-delivery, fix predict None-fill, drop dead session_id 2026-07-14 17:07:55 -04:00
iomgaa 1d222d9f18 feat: wire real agent runner, backfill assembly and adversarial-filter CLI 2026-07-14 16:58:32 -04:00
iomgaa b13eab0659 feat: wire grounded selector into AR slot processing
将 Task 5 的 grounded selector 织入 AR 出题路径(Phase 3.5,位于
record_item 与 postprocess 之间),仅在 strategy.uses_grounded_selector
为真时进入。observation 始终落库(含 hard-fail),硬失败走重出。
PipelineConfig 新增 candidate_pool_size/selector_delta_low/
selector_delta_high 三参,YAML 与 CLI seed override 同步。
2026-07-14 14:15:50 -04:00
iomgaa ae0a718f67 feat: thread and persist sub_pattern into accepted questions 2026-07-14 13:45:31 -04:00
iomgaa 36c712defa feat(pipeline): replace QuestionFamilySpec with TaskTypeStrategy
- SlotAssignment: remove family field, strategy looked up at process time
- PipelineConfig: remove family_ratios field
- _assign_slots: remove family_ratios and rng params (pure deterministic)
- _process_one_slot: use get_strategy() for sampling, generation, gates
- Add sub_pattern support (level/constraint override, instruction injection)
- Add strategy.extra_gates() check after standard gates
- load_pipeline_config: stop reading family_ratios from YAML
- Update tools/generate_questions.py seed override and dry-run log
- Update all integration tests to match new API

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-14 06:05:23 -04:00
iomgaa eecb86e27a feat(question_gen): add generate-v2 CLI subcommand and experiment script
- Add generate-v2 subparser with --config, --store-dir, --db-path,
  --seed, and --dry-run arguments to tools/generate_questions.py
- Implement _run_generate_v2 async handler: config loading, video
  discovery, DI client construction, TreeIndex loading, pipeline
  invocation, and result persistence
- Add scripts/generate_questions_v2.sh following build_trees.sh
  conventions (source .env, conda run python path, MODE=mock support)
- Update app/question_gen/__init__.py to export full v2 public API:
  run_pipeline_v2, PipelineConfig, PipelineResult, QuestionFamilySpec,
  ALL_FAMILIES, CandidateQuestion, generate_one_v2, GateReport, run_gates
- Add QuestionGenStore.load_progress() for pipeline resumption
- Add integration tests for CLI help and dry-run behavior
- Update test_question_gen_api to match expanded __all__

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-12 00:04:42 -04:00
iomgaa da70eb6e23 refactor(question_gen): slim calibrate to compare two existing runs 2026-07-11 07:45:42 -04:00
iomgaa 735471df3c fix(question_gen): Information Synopsis 去重阈值放宽至 0.90
该题型天然高相似(benchmark 均值 0.66,中位数 0.67),
通用阈值 0.75/0.85 导致几乎所有新题被误判重复。
按题型覆盖为 0.90,只拦截真正的复制品。

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-10 08:55:47 -04:00
iomgaa 25c8d5ec42 fix(question_gen): 每次重试换视频避免同视频反复去重失败
Information Synopsis 163 道 benchmark 题,同一视频反复出题
必然相似。改为每次重试随机选不同视频。

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-10 08:15:03 -04:00
iomgaa 8d11513e54 fix: 禁止日志缓存,确保所有日志立刻输出
- scripts/*.sh: PYTHONUNBUFFERED=1
- tools/generate_questions.py: loguru file sink enqueue=False
- main.py: loguru file sink enqueue=False

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-09 23:15:02 -04:00
iomgaa c191be16e2 fix(question_gen): L3 frame_path 拼接视频目录前缀
tree.json 中 frame_path 是相对路径(frames/xxx.jpg),
generate_questions 需要拼上 store/videos/{video_id}/ 前缀供 VLM 读取。

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-09 23:10:18 -04:00
iomgaa f57ee45dc0 fix: Codex 全量审查修正
Critical:
- C1: assemble_mode 'plain' → 'ids'(合法枚举值)
- C2: question_id 加入 task_type slug 避免跨题型冲突

Important/Minor:
- generate_one 移除未用的 embed_fn/similarity_threshold 参数
- config.py 注释 11→12 同步
- 测试 question_id 断言更新

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-09 07:40:27 -04:00
iomgaa 93c9be8bfa feat(tools): generate_questions.py calibrate 子命令
- Fisher exact test + effect size 组合判定(PASS/WARN/FAIL)
- 按 video_id 分组推理,避免跨视频树错用
- baseline 支持从 DB 读取或自动跑推理
- 对比表输出 + 退出码控制

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-09 05:47:52 -04:00
iomgaa 11f3c90200 feat(tools): generate_questions.py generate 子命令
- VLM 出题 + embedding 去重 + 断点续跑 + 并发控制
- 单线程汇总点保证去重原子性
- 18 个单元测试覆盖 progress/exemplar/pool rebuild/JSON append

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-09 05:42:50 -04:00