iomgaa/Video-Tree-TRM5
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
272 Commits 2 Branches 0 Tags
334fbbc94d4354cabdbcb54f3ce64bafaec80c25
Commit Graph

272 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
iomgaa 334fbbc94d feat: add AdversarialFilterConfig for Phase B post-hoc filter layer 2026-07-14 15:47:21 -04:00
iomgaa ac115d96fb test: cover backfill params end-to-end (mutation isolation + seq_offset) 2026-07-14 15:44:43 -04:00
iomgaa 8b9e8aa19f feat: add optional backfill params to run_pipeline_v2 2026-07-14 15:39:57 -04:00
iomgaa d77cbc95eb feat: add adversarial_verdicts table with resume and terminal-verdict query 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 15:35:11 -04:00
iomgaa e41a2b0d08 test: guard non-AR flip isolation and AR flip-capable set 2026-07-14 15:30:18 -04:00
iomgaa f12dd7e559 feat: add supports_flip/flip_axis to SubPattern for Phase B flip gate 2026-07-14 15:27:21 -04:00
iomgaa 841112c6af docs: revise Phase B plan per Codex review (C1-C3,I1-I6,M1-M3) 2026-07-14 15:24:44 -04:00
iomgaa abc9c097d2 docs: add Phase B adversarial question-gen implementation plan 2026-07-14 15:05:36 -04:00
iomgaa 96e314c3a0 fix: mark item rejected on selector_error for consistent bookkeeping 
selector_error 分支（捕获 ValueError/FileNotFoundError）此前只返回 reason，
未 mark_item_rejected，导致 Phase 3 已 record 的 pending attempt 行永远停在
pending；而 hard-fail 分支会标记 rejected。两条失败路径落库风格现统一为
mark_item_rejected（异常路径无 outcome/observation，故不写 selector_scores）。

补单测 test_apply_grounded_selector_marks_rejected_on_error 守卫该路径。
 2026-07-14 14:49:41 -04:00
iomgaa 46ac848176 fix: converge non-numeric selector scores to ValueError 2026-07-14 14:47:21 -04:00
iomgaa eaab865d84 docs: register Phase A plan in research wiki 2026-07-14 14:41:53 -04:00
iomgaa e64c26e578 style: format distractor_selector 2026-07-14 14:41:21 -04:00
iomgaa 76f719018c feat: loosen multi_true gate to qualifier-scoped correctness 2026-07-14 14:33:22 -04:00
iomgaa b1f15ddb3a fix: keep cross_segment rule single-dimension, guard no-absent-events 2026-07-14 14:31:30 -04:00
iomgaa 58278c6de4 feat: enforce single-dimension counterfactual in AR distractor rules 2026-07-14 14:28:32 -04:00
iomgaa ea38e4717b refactor: extract grounded selector application into helper 
将 Phase 3.5 的 grounded selector 逻辑抽成模块级 _apply_grounded_selector，
内部完成策略门控 / 异常捕获 / observation 落库 / hard-fail 拒绝，调用点仅剩
单一失败分支（selector_reason → 重出）。行为不变，测试全绿。

顺带移除 resample_video_interval 死子表达式（硬编码 1，(attempt-1)%1==0 恒真），
使 _process_one_slot 圈复杂度回落至 Task 6 前的 D(25)。_apply_grounded_selector
自身为 A(4)，均满足质量门。
 2026-07-14 14:22:30 -04:00
iomgaa b13eab0659 feat: wire grounded selector into AR slot processing 
将 Task 5 的 grounded selector 织入 AR 出题路径（Phase 3.5，位于
record_item 与 postprocess 之间），仅在 strategy.uses_grounded_selector
为真时进入。observation 始终落库（含 hard-fail），硬失败走重出。
PipelineConfig 新增 candidate_pool_size/selector_delta_low/
selector_delta_high 三参，YAML 与 CLI seed override 同步。
 2026-07-14 14:15:50 -04:00
iomgaa d0194f5840 fix: degrade distractor pool gracefully on malformed VLM response 2026-07-14 14:11:13 -04:00
iomgaa 8a54055d02 feat: add grounded distractor selector with visual scoring 2026-07-14 14:06:07 -04:00
iomgaa 207e834f30 feat: add selector_scores observation column to question_gen_items 2026-07-14 13:59:29 -04:00
iomgaa 3f984acc18 feat: add uses_grounded_selector strategy switch (AR only) 2026-07-14 13:53:17 -04:00
iomgaa 2608a3841f style: reformat sub_pattern round-trip tests 2026-07-14 13:51:51 -04:00
iomgaa e68e4b7d57 fix: restore sub_pattern when loading benchmark JSON 2026-07-14 13:51:05 -04:00
iomgaa 111c88488f style: format sub_pattern test file 2026-07-14 13:46:25 -04:00
iomgaa ae0a718f67 feat: thread and persist sub_pattern into accepted questions 2026-07-14 13:45:31 -04:00
iomgaa 25f2a845ff fix: run_gates must use current_tree after video resample 2026-07-14 13:40:46 -04:00
iomgaa 92714db370 docs: revise Phase A plan per Codex review (C1-C2,I1-I5,M1) 2026-07-14 13:38:35 -04:00
iomgaa 645585e784 docs: add Phase A grounded question-gen implementation plan 2026-07-14 13:28:43 -04:00
iomgaa ad0bfe3a51 docs: add grounded + adversarial question-gen design (Phase A/B) 2026-07-14 13:21:06 -04:00
iomgaa 84b52a0311 feat(pools): auto-supplement maintenance correct questions in PerCategoryPoolStrategy 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:41:46 -04:00
iomgaa 72befa2bd4 feat(pools): add batch_correct_ratio field to PoolConfig 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:34:28 -04:00
iomgaa 453cf62088 docs: add maintenance pool implementation plan 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:22:52 -04:00
iomgaa 73c5d1e380 docs: add high-correctness warning to maintenance pool design 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:13:57 -04:00
iomgaa 89cad891de docs: add maintenance pool auto-supplement design 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:11:26 -04:00
iomgaa d7d7ce5bdc feat(question_gen): register ActionRecognitionStrategy, replace temp VISUAL binding 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:51:50 -04:00
iomgaa d0a8019fe1 feat(question_gen): add ActionRecognitionStrategy with 6 SubPatterns 
Self-contained strategy targeting 6 Agent failure modes in Action
Recognition: premature_evidence_anchoring, temporal_reasoning_failure,
semantic_rigidity, fine_grained_visual_action,
cross_segment_entity_tracking, evidence_gap_confabulation.

- L2 default sampling (upgrade from L3) with 3 patterns overriding to L1
- Weighted random SubPattern selection (0.20/0.20/0.15/0.15/0.15/0.15)
- Each SubPattern includes instruction, examples, distractor rules
- Satisfies TaskTypeStrategy Protocol without extending BaseTaskTypeStrategy
- 32 unit tests covering all properties, definitions, and selection behavior

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:49:03 -04:00
iomgaa eaa58604b3 feat(question_gen): add Action Recognition specialized prompt template 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:45:09 -04:00
iomgaa b976c789d2 docs: add Plan B — ActionRecognitionStrategy implementation plan 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:34:40 -04:00
iomgaa cd0805aad6 chore: fix N806 lint - rename _RESAMPLE_VIDEO_INTERVAL to lowercase 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:11:01 -04:00
iomgaa 7c7ad67d81 fix(pipeline): move extra_gates after standard gate pass check 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:08:52 -04:00
iomgaa 36c712defa feat(pipeline): replace QuestionFamilySpec with TaskTypeStrategy 
- SlotAssignment: remove family field, strategy looked up at process time
- PipelineConfig: remove family_ratios field
- _assign_slots: remove family_ratios and rng params (pure deterministic)
- _process_one_slot: use get_strategy() for sampling, generation, gates
- Add sub_pattern support (level/constraint override, instruction injection)
- Add strategy.extra_gates() check after standard gates
- load_pipeline_config: stop reading family_ratios from YAML
- Update tools/generate_questions.py seed override and dry-run log
- Update all integration tests to match new API

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:05:23 -04:00
iomgaa b9616de21e fix(tests): update test_generator_v2 to use new generator signatures 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:55:08 -04:00
iomgaa afa77173e3 refactor(question_gen): adapt generator/gates/store signatures for strategy 
- generator_v2: _load_prompt_template takes template_name str instead of
  QuestionFamilySpec; _build_v2_prompt takes prompt_template + strategy_name
  + sub_pattern_instruction; generate_one_v2 takes discrete params
  (prompt_template, strategy_name, skill_target, sub_pattern_instruction)
- gates: _gate_leak_test and run_gates take leak_probe_template str
  instead of QuestionFamilySpec
- run_store: add sub_pattern column to DDL + idempotent migration;
  record_item accepts optional sub_pattern param
- Remove QuestionFamilySpec imports from generator_v2 and gates modules
- Update test call sites accordingly

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:50:47 -04:00
iomgaa c49d0ff12f fix(sampler): validate level param rejects invalid values 
Add ValueError guard at the top of sample_material_v2 for level not in
{1, 2, 3}, preventing silent fallthrough to L1 sampling. Add unit test
for the new validation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:43:28 -04:00
iomgaa e2325b6535 refactor(sampler): replace family_spec param with level+constraint 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:38:14 -04:00
iomgaa b6b6a48503 feat(question_gen): add TaskTypeStrategy Protocol and BaseTaskTypeStrategy 
- TaskTypeStrategy Protocol: pipeline 的唯一接口，定义 task_type、
  sampling_level、sampling_constraint、prompt_template 等属性
- SubPattern frozen dataclass: 出题子模式，靶向特定失败机制
- BaseTaskTypeStrategy: 封装现有 QuestionFamilySpec 行为的默认策略，
  所有属性委托给绑定的 family
- _TASK_TYPE_TO_FAMILY: 消歧绑定表，12 个题型确定性绑定到 1 个 family
- register_strategy/get_strategy: 注册表 API，未注册题型自动创建
  BaseTaskTypeStrategy
- 13 个单元测试全部通过

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:24:29 -04:00
iomgaa 832838350a docs: add TaskTypeStrategy design + Plan A framework plan 2026-07-14 05:15:28 -04:00
iomgaa 9ee37a8534 fix(loader): use video_id from JSON data instead of filename 
load_benchmark used the JSON filename stem as video_id, which broke
v2-360 questions (all 180 questions got video_id='accepted_questions').
Now uses qa['video_id'] when present, falls back to filename for
Video-MME format compatibility.
 2026-07-14 01:02:55 -04:00
iomgaa dec7346da3 feat(harness): add Action Recognition training experiment 
- PerCategoryPoolStrategy: filter test pool by task_types
- RunConfig: add run_holdout_eval toggle (default true)
- load_config: fix YAML task_types list-to-tuple conversion
- Runner: conditionally skip _holdout_four_way when disabled
- CLI: add --no-run-holdout-eval flag
- New config/train_action_recognition.yaml (3 epochs, per_category)
- New scripts/train_action_recognition.sh (baseline + seed + train)
 2026-07-14 00:58:54 -04:00
iomgaa 37d4519905 chore: lint and format per-category pool strategy implementation 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:58:10 -04:00