iomgaa/Video-Tree-TRM5
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
261 Commits 2 Branches 0 Tags
e64c26e578db0a27a7f97280d6c96477a023081f
Commit Graph

261 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
iomgaa e64c26e578 style: format distractor_selector 2026-07-14 14:41:21 -04:00
iomgaa 76f719018c feat: loosen multi_true gate to qualifier-scoped correctness 2026-07-14 14:33:22 -04:00
iomgaa b1f15ddb3a fix: keep cross_segment rule single-dimension, guard no-absent-events 2026-07-14 14:31:30 -04:00
iomgaa 58278c6de4 feat: enforce single-dimension counterfactual in AR distractor rules 2026-07-14 14:28:32 -04:00
iomgaa ea38e4717b refactor: extract grounded selector application into helper 
将 Phase 3.5 的 grounded selector 逻辑抽成模块级 _apply_grounded_selector，
内部完成策略门控 / 异常捕获 / observation 落库 / hard-fail 拒绝，调用点仅剩
单一失败分支（selector_reason → 重出）。行为不变，测试全绿。

顺带移除 resample_video_interval 死子表达式（硬编码 1，(attempt-1)%1==0 恒真），
使 _process_one_slot 圈复杂度回落至 Task 6 前的 D(25)。_apply_grounded_selector
自身为 A(4)，均满足质量门。
 2026-07-14 14:22:30 -04:00
iomgaa b13eab0659 feat: wire grounded selector into AR slot processing 
将 Task 5 的 grounded selector 织入 AR 出题路径（Phase 3.5，位于
record_item 与 postprocess 之间），仅在 strategy.uses_grounded_selector
为真时进入。observation 始终落库（含 hard-fail），硬失败走重出。
PipelineConfig 新增 candidate_pool_size/selector_delta_low/
selector_delta_high 三参，YAML 与 CLI seed override 同步。
 2026-07-14 14:15:50 -04:00
iomgaa d0194f5840 fix: degrade distractor pool gracefully on malformed VLM response 2026-07-14 14:11:13 -04:00
iomgaa 8a54055d02 feat: add grounded distractor selector with visual scoring 2026-07-14 14:06:07 -04:00
iomgaa 207e834f30 feat: add selector_scores observation column to question_gen_items 2026-07-14 13:59:29 -04:00
iomgaa 3f984acc18 feat: add uses_grounded_selector strategy switch (AR only) 2026-07-14 13:53:17 -04:00
iomgaa 2608a3841f style: reformat sub_pattern round-trip tests 2026-07-14 13:51:51 -04:00
iomgaa e68e4b7d57 fix: restore sub_pattern when loading benchmark JSON 2026-07-14 13:51:05 -04:00
iomgaa 111c88488f style: format sub_pattern test file 2026-07-14 13:46:25 -04:00
iomgaa ae0a718f67 feat: thread and persist sub_pattern into accepted questions 2026-07-14 13:45:31 -04:00
iomgaa 25f2a845ff fix: run_gates must use current_tree after video resample 2026-07-14 13:40:46 -04:00
iomgaa 92714db370 docs: revise Phase A plan per Codex review (C1-C2,I1-I5,M1) 2026-07-14 13:38:35 -04:00
iomgaa 645585e784 docs: add Phase A grounded question-gen implementation plan 2026-07-14 13:28:43 -04:00
iomgaa ad0bfe3a51 docs: add grounded + adversarial question-gen design (Phase A/B) 2026-07-14 13:21:06 -04:00
iomgaa 84b52a0311 feat(pools): auto-supplement maintenance correct questions in PerCategoryPoolStrategy 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:41:46 -04:00
iomgaa 72befa2bd4 feat(pools): add batch_correct_ratio field to PoolConfig 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:34:28 -04:00
iomgaa 453cf62088 docs: add maintenance pool implementation plan 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:22:52 -04:00
iomgaa 73c5d1e380 docs: add high-correctness warning to maintenance pool design 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:13:57 -04:00
iomgaa 89cad891de docs: add maintenance pool auto-supplement design 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:11:26 -04:00
iomgaa d7d7ce5bdc feat(question_gen): register ActionRecognitionStrategy, replace temp VISUAL binding 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:51:50 -04:00
iomgaa d0a8019fe1 feat(question_gen): add ActionRecognitionStrategy with 6 SubPatterns 
Self-contained strategy targeting 6 Agent failure modes in Action
Recognition: premature_evidence_anchoring, temporal_reasoning_failure,
semantic_rigidity, fine_grained_visual_action,
cross_segment_entity_tracking, evidence_gap_confabulation.

- L2 default sampling (upgrade from L3) with 3 patterns overriding to L1
- Weighted random SubPattern selection (0.20/0.20/0.15/0.15/0.15/0.15)
- Each SubPattern includes instruction, examples, distractor rules
- Satisfies TaskTypeStrategy Protocol without extending BaseTaskTypeStrategy
- 32 unit tests covering all properties, definitions, and selection behavior

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:49:03 -04:00
iomgaa eaa58604b3 feat(question_gen): add Action Recognition specialized prompt template 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:45:09 -04:00
iomgaa b976c789d2 docs: add Plan B — ActionRecognitionStrategy implementation plan 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:34:40 -04:00
iomgaa cd0805aad6 chore: fix N806 lint - rename _RESAMPLE_VIDEO_INTERVAL to lowercase 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:11:01 -04:00
iomgaa 7c7ad67d81 fix(pipeline): move extra_gates after standard gate pass check 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:08:52 -04:00
iomgaa 36c712defa feat(pipeline): replace QuestionFamilySpec with TaskTypeStrategy 
- SlotAssignment: remove family field, strategy looked up at process time
- PipelineConfig: remove family_ratios field
- _assign_slots: remove family_ratios and rng params (pure deterministic)
- _process_one_slot: use get_strategy() for sampling, generation, gates
- Add sub_pattern support (level/constraint override, instruction injection)
- Add strategy.extra_gates() check after standard gates
- load_pipeline_config: stop reading family_ratios from YAML
- Update tools/generate_questions.py seed override and dry-run log
- Update all integration tests to match new API

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 06:05:23 -04:00
iomgaa b9616de21e fix(tests): update test_generator_v2 to use new generator signatures 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:55:08 -04:00
iomgaa afa77173e3 refactor(question_gen): adapt generator/gates/store signatures for strategy 
- generator_v2: _load_prompt_template takes template_name str instead of
  QuestionFamilySpec; _build_v2_prompt takes prompt_template + strategy_name
  + sub_pattern_instruction; generate_one_v2 takes discrete params
  (prompt_template, strategy_name, skill_target, sub_pattern_instruction)
- gates: _gate_leak_test and run_gates take leak_probe_template str
  instead of QuestionFamilySpec
- run_store: add sub_pattern column to DDL + idempotent migration;
  record_item accepts optional sub_pattern param
- Remove QuestionFamilySpec imports from generator_v2 and gates modules
- Update test call sites accordingly

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:50:47 -04:00
iomgaa c49d0ff12f fix(sampler): validate level param rejects invalid values 
Add ValueError guard at the top of sample_material_v2 for level not in
{1, 2, 3}, preventing silent fallthrough to L1 sampling. Add unit test
for the new validation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:43:28 -04:00
iomgaa e2325b6535 refactor(sampler): replace family_spec param with level+constraint 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:38:14 -04:00
iomgaa b6b6a48503 feat(question_gen): add TaskTypeStrategy Protocol and BaseTaskTypeStrategy 
- TaskTypeStrategy Protocol: pipeline 的唯一接口，定义 task_type、
  sampling_level、sampling_constraint、prompt_template 等属性
- SubPattern frozen dataclass: 出题子模式，靶向特定失败机制
- BaseTaskTypeStrategy: 封装现有 QuestionFamilySpec 行为的默认策略，
  所有属性委托给绑定的 family
- _TASK_TYPE_TO_FAMILY: 消歧绑定表，12 个题型确定性绑定到 1 个 family
- register_strategy/get_strategy: 注册表 API，未注册题型自动创建
  BaseTaskTypeStrategy
- 13 个单元测试全部通过

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 05:24:29 -04:00
iomgaa 832838350a docs: add TaskTypeStrategy design + Plan A framework plan 2026-07-14 05:15:28 -04:00
iomgaa 9ee37a8534 fix(loader): use video_id from JSON data instead of filename 
load_benchmark used the JSON filename stem as video_id, which broke
v2-360 questions (all 180 questions got video_id='accepted_questions').
Now uses qa['video_id'] when present, falls back to filename for
Video-MME format compatibility.
 2026-07-14 01:02:55 -04:00
iomgaa dec7346da3 feat(harness): add Action Recognition training experiment 
- PerCategoryPoolStrategy: filter test pool by task_types
- RunConfig: add run_holdout_eval toggle (default true)
- load_config: fix YAML task_types list-to-tuple conversion
- Runner: conditionally skip _holdout_four_way when disabled
- CLI: add --no-run-holdout-eval flag
- New config/train_action_recognition.yaml (3 epochs, per_category)
- New scripts/train_action_recognition.sh (baseline + seed + train)
 2026-07-14 00:58:54 -04:00
iomgaa 37d4519905 chore: lint and format per-category pool strategy implementation 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:58:10 -04:00
iomgaa 671db2f88c test(integration): add PerCategoryPoolStrategy end-to-end test 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:56:42 -04:00
iomgaa 5563d0d675 feat(cli): wire train mode with PoolStrategy selection and task_types in RunConfig 
- Add --pool-split-mode, --train-ratio, --test-questions CLI arguments
- Include task_types in cli_overrides (convert list to tuple for RunConfig)
- Wire train mode branch: select strategy based on pool_split_mode,
  resolve workspace paths, build pools, and call runner.train()
- Infer mode now reads task_types from config instead of raw args

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:56:18 -04:00
iomgaa c66a00c924 feat(harness): refactor build_or_load_pools to accept PoolStrategy + per_category freeze format 
- save_pools: extended with split_mode and config params; per_category
  mode writes categories metadata (seed, train_ratio, test_source) for
  incremental append and consistency validation
- load_pools: compatible with both old format (no split_mode) and new
  format; extra metadata fields ignored during load
- build_or_load_pools: signature changed to (config, strategy, db_path);
  baseline_run_id read from seed.json (not config.run_id); per_category
  mode does consistency check on reload and supports incremental category
  append via strategy.build_incremental
- Added _to_pool_config, _read_baseline_run_id,
  _validate_per_category_consistency helpers
- Tests: TestPerCategorySaveLoad with 5 test cases covering roundtrip,
  missing config error, global split_mode field, legacy format compat,
  multi-type categories

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:53:19 -04:00
iomgaa e5b07ac974 feat(harness): add task_types, pool_split_mode, train_ratio, test_questions to RunConfig 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:46:28 -04:00
iomgaa 73ae1f7143 fix(harness): change _runs INSERT OR IGNORE to ON CONFLICT DO UPDATE for incremental infer 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:45:18 -04:00
iomgaa 8b48005a17 config: add pool_split_mode, train_ratio, test_questions to default.yaml 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:43:43 -04:00
iomgaa 21c6a53aed feat(harness): add PerCategoryPoolStrategy with correctness-stratified 2:1 split 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:41:48 -04:00
iomgaa ec4cbbdd44 style(core): move Path import to TYPE_CHECKING block in types.py 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:36:00 -04:00
iomgaa cd5c9c01fb feat(app): add PoolStrategy Protocol to application ports 2026-07-12 22:34:47 -04:00
iomgaa 0d0f275134 feat(core): add PoolConfig dataclass for pool strategy configuration 2026-07-12 22:33:49 -04:00
iomgaa 4fb7a61f8b fix(question_gen): resolve pipeline integration issues from final review 
1. Apply postprocess shuffle result (pp.options, pp.answer) to final
   GeneratedQuestion output instead of using original candidate values.

2. Record dedup rejection in store via new mark_item_rejected() method,
   preventing items from staying as 'accepted' after dedup rejects them.

3. Add .flatten() to embed_fn outputs in _is_duplicate and embed_pool
   append to handle 2D (1,D) arrays from embedding implementations.

4. Validate exactly 4 options in _validate_parsed_fields (was >= 2),
   matching the A-D answer constraint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 00:14:12 -04:00