iomgaa/Video-Tree-TRM5
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
305 Commits 2 Branches 0 Tags
bd1f7a22a2db90353d29faa836039ca6b4d9632b
Commit Graph

45 Commits

Author SHA1 Message Date
iomgaa bd1f7a22a2 feat(harness): pools.json 序列化 pair 四字段防孤儿 single 
_q_to_dict 写出 pair_id/question_role/flip_axis/unit_id，_dict_to_q 用 .get
兼容旧 workspace 的 pools.json 读回并回填（unit_id 缺省交 __post_init__）。
pools.json 是训练主回路读回题目处，此前漏写会让孪生对解冻后退化成孤儿
single、配对指标失真。categories 块沿用 per-qid 记录，Task 3 的 unit 原子
切分已保证两 pair 成员同池同 key，序列化不破坏该原子性。
 2026-07-15 08:05:15 -04:00
iomgaa 273984674b feat(harness): 迁移 gate 信息量阶梯到 unit 粒度 + schema_version 
核心算法保真#5（信息阶梯）：gate_ladder.py 从逐题迁移到 unit 粒度，
只换键 question_id→unit_id，冷启动 2:1 错优先交错、gamma-EMA 公式、
Beta(1,1) 先验、反泄漏 _gate_ 过滤的公式/比例/顺序语义一字不改。

- LadderEntry 按 unit_id 键；AR pair 折叠为一个阶梯单元
- build_cold_entries 收单元列表，unit 错 = 任一成员错（双向 AND）折叠，
  2:1 交错 + probe 探针按 unit 抽，Beta 先验 p0 不变
- ladder_for 返回 unit_id 序、exclude 迁到 unit 口径（防半 pair 灌入
  触发下游 _ladder_units fail-fast）
- update_probs 先把逐题观测折叠成单元观测再按 unit_id 匹配更新，
  半观测单元跳过（防按 qid 匹配 pair 失效致 gamma-EMA 停摆）
- GatePools.save/load 加 schema_version=2；存量无版本/旧版本 json
  加载直接报错，拒绝静默混用 qid/unit 键
- BaselineCache 第四维键改名 unit_id（与 T7 validate 路径对齐）
- build_or_load_gate_pools 先折叠单元再排除 test（抽 helper 控复杂度 B）
- runner：_init_gate_pools 建 unit 索引；gate 验证 exclude/展开、
  _refresh_gate_ladder 折叠观测走 units_by_id

反泄漏 run_id 含 _gate_ 过滤不受影响（未改）。
测试：新增 test_gate_ladder_unit_migration.py（15 例覆盖 a-e），
既有 test_harness_gate_ladder.py 迁移到 unit API。全量 1363 passed。
 2026-07-15 07:56:15 -04:00
iomgaa 7e97081779 test(harness): 补 _ladder_units 直测 + 澄清 gate 观测表 unit_id 口径注释 
M1：quadrant_pair / gate_evidence 的 question_id 列注释与 write_* docstring
更正为承载 unit_id（single=question_id、pair=pair_id），提示逐题明细在
predictions 表溯源、按 pair_id join 真实 question 表会 join 不上。

M2：给 _ladder_units 补直接单测——纯非 AR 恒等（unit 序==原题序、
unit_id==question_id）、混格交错保持信息阶梯序（按单元最早出现下标重排、
pair 整锁）、且与 build_units 的 single-first 默认序显式区分（防阶梯序被污染）。
 2026-07-15 07:40:36 -04:00
iomgaa 4b6d1d8a50 feat(harness): correctness 三口径 + gate 块按 unit 跑 
进化引擎与 gate e-process 从 question_id 口径迁至 unit_id 口径，AR pair
双向 AND 折叠为单元、不被 P/Q 单题计分污染；逐题 predictions 仅作溯源。

- question_units: 新增 unit_correctness_view(units, per_q)->dict[unit_id,bool]
  作为逐题→单元折叠的唯一入口（复用 unit_correctness）。
- core/evolution/validate: pair_block/compute_accuracy 参数改 unit_ids、
  分母按单元数（键即 unit_id）。
- app/harness/validate（gate 实际执行路径）：阶梯题序聚合为单元并保持信息
  阶梯序（_ladder_units），gate 块按单元切分（AR pair 整锁不跨块拆）；
  baseline_cache 键含 unit_id、存单元级对错；候选逐题读回后折叠成单元视图；
  n_used/W/L/四象限/准确率均按单元计；证据行按 unit 口径，candidate_correctness
  独立保留逐题对错供 runner 二轨合并。
- runner: probation 结算按 unit 折叠计 W/L（_probation_unit_flips）；quadrant
  四象限 id 承载 unit_id。

核心算法保真 #5（信息阶梯 e-process）：本次仅迁移 correctness 口径，不改冷启动
2:1 / gamma-EMA / 反泄漏算法本身（gate_ladder 迁移见 Task 8）。
 2026-07-15 07:31:03 -04:00
iomgaa 8a69a54078 refactor: fail-loud unit grading and stricter orphan pair drop 
Address review: replace bare KeyError with a contextual ValueError invariant
check in _grade_unit (fail-loud, no catch/skip/fallback); tighten
_drop_orphan_pairs to require exactly one original + one mirror with no extra
illegal-role records (total==2); clarify InferenceResult docstring to unit-grained
semantics. Add tests for missing-prediction descriptive error and extra-illegal-role
pair drop.
 2026-07-15 07:05:37 -04:00
iomgaa 730caa7e9a feat: aggregate inference by question unit with pair AND 
Reuse build_units/unit_correctness (pair contract single entry) in the
inference aggregation step: single questions count as one unit, AR pairs
collapse original+mirror into one unit scored by bidirectional AND. total/
correct/per_task_type are unit-grained; orphan pairs (missing one side) are
warned and dropped, not counted. Per-question predictions still land row by
row (traceability unchanged).
 2026-07-15 06:57:48 -04:00
iomgaa 2429dad393 feat(batching): unit 粒度切分——pair 整锁 + 单元级分桶 + 非 AR 独立 rng 
build_batches 改以 QuestionUnit 为原子调度单元：孪生对 2 题整锁进同一 batch、
按单元级正确性（双向 AND）落 correct/error 桶，不再因 P 对 Q 错被劈或被 FFD 拆箱。

- 非 AR（single）用 random.Random(seed) 复现旧逐题算法确切 draw 序列，AR（pair）
  用 _rng_ns(seed,"AR") SHA-256 派生独立流；二者 draw 流互不干扰，故 AR 折叠不改变
  非 AR 抽样/洗牌序列——纯非 AR 输入 build_batches 结果与引入 QuestionUnit 前逐字节一致。
- FFD 容量按 unit.size（pair 占 2），round-robin 遇碎片新开 bin 兜底而非报错。
- _select_mixed_by_task_type 分流各跑一次后合并，大类洗牌按 kind 拆分各用对应 rng。

新增黄金测试 test_batching_pair_lock.py 覆盖三条铁律（同 batch / 单元分桶 /
非 AR byte-identical + draw 流独立）；既有 batching 测试全绿。
 2026-07-15 06:46:28 -04:00
iomgaa d6a3107e4e feat(question_gen): loader 按 unit 分层采样 + load_benchmark 读回 pair 字段 
stratified_sample 先 build_units 聚合，以 QuestionUnit 为采样原子做
分层/去重/补足/rng.sample，返回前 flatten_units 展开为逐题列表；
size/correct_ratio/min_per_class 均按 unit 计数，单元正确性走成员 AND，
孪生对两题永不被劈开。纯 single 输入下 build_units 1:1 折叠、顺序不变，
rng 消耗与旧逐题实现字节级一致（新增回归测试守护）。

_backfill_per_class candidates 改按 unit 枚举去重；build_units/flatten_units
函数内延迟导入以规避 question_gen<->harness 循环依赖（沿用 adversarial_filter）。

load_benchmark 反序列化补 pair_id/question_role/flip_axis/unit_id 四字段，
用 .get 兼容旧 JSON（缺失退化为 single，unit_id 由 __post_init__ 回填）。

pools._sample_excluding 随之改为透传 flatten_units(candidates) 给已单元化的
stratified_sample（不再用 lone pair-original 代表），行为对 single-only 保持等价。
 2026-07-15 06:27:41 -04:00
iomgaa ddb9a44f75 feat(pools): 三池切分以 unit 为原子，孪生对同池不被拆散 
build_pools/_sample_excluding 与 PerCategoryPoolStrategy._split_one_category/
build_incremental 两条切分路径均改为以 QuestionUnit 为采样原子：progressive
exclusion 互斥集合与 train/val 分层划分都按 unit_id 计数（pair 计 1 个 unit），
命中单元整体展开，AR 孪生对两题永不落入不同池/split。

复用 app.harness.question_units 的 build_units/flatten_units，不重写分组逻辑。
single-only 输入下 unit 与 question 一一对应、rng 消耗量不变，采样与划分结果
与逐题口径完全一致；抽出 _unit_correct/_assert_correctness_complete 两个 helper
将 _split_one_category 复杂度压回基线以下。

新增 tests/unit/test_pools_pair_atomic.py 覆盖两条路径的 pair 原子性回归。
 2026-07-15 06:09:14 -04:00
iomgaa 5ef5f2b8b7 refactor: extract _assemble_pair to lower build_units complexity to B 
将 pair 分组的数量/角色校验与组装下沉到 _assemble_pair 私有 helper，
build_units 主体只做分组+派发，radon 复杂度从 C(11) 降到 A(4)。
fail-fast 语义、公共函数签名、稳定顺序均不变。
 2026-07-15 05:54:08 -04:00
iomgaa bef46636fe feat: add question_units helper as pair contract entry point 
build_units/flatten_units/validate_units/unit_correctness——pair 契约唯一入口。
build_units 对孤儿/超员/角色缺失重复 fail-fast raise ValueError（防 next 静默
StopIteration）；unit_correctness 走 per_q[qid] KeyError 防静默兜底。
 2026-07-15 05:52:30 -04:00
iomgaa 84b52a0311 feat(pools): auto-supplement maintenance correct questions in PerCategoryPoolStrategy 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:41:46 -04:00
iomgaa 72befa2bd4 feat(pools): add batch_correct_ratio field to PoolConfig 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-14 10:34:28 -04:00
iomgaa dec7346da3 feat(harness): add Action Recognition training experiment 
- PerCategoryPoolStrategy: filter test pool by task_types
- RunConfig: add run_holdout_eval toggle (default true)
- load_config: fix YAML task_types list-to-tuple conversion
- Runner: conditionally skip _holdout_four_way when disabled
- CLI: add --no-run-holdout-eval flag
- New config/train_action_recognition.yaml (3 epochs, per_category)
- New scripts/train_action_recognition.sh (baseline + seed + train)
 2026-07-14 00:58:54 -04:00
iomgaa 37d4519905 chore: lint and format per-category pool strategy implementation 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:58:10 -04:00
iomgaa c66a00c924 feat(harness): refactor build_or_load_pools to accept PoolStrategy + per_category freeze format 
- save_pools: extended with split_mode and config params; per_category
  mode writes categories metadata (seed, train_ratio, test_source) for
  incremental append and consistency validation
- load_pools: compatible with both old format (no split_mode) and new
  format; extra metadata fields ignored during load
- build_or_load_pools: signature changed to (config, strategy, db_path);
  baseline_run_id read from seed.json (not config.run_id); per_category
  mode does consistency check on reload and supports incremental category
  append via strategy.build_incremental
- Added _to_pool_config, _read_baseline_run_id,
  _validate_per_category_consistency helpers
- Tests: TestPerCategorySaveLoad with 5 test cases covering roundtrip,
  missing config error, global split_mode field, legacy format compat,
  multi-type categories

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:53:19 -04:00
iomgaa e5b07ac974 feat(harness): add task_types, pool_split_mode, train_ratio, test_questions to RunConfig 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:46:28 -04:00
iomgaa 73ae1f7143 fix(harness): change _runs INSERT OR IGNORE to ON CONFLICT DO UPDATE for incremental infer 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:45:18 -04:00
iomgaa 21c6a53aed feat(harness): add PerCategoryPoolStrategy with correctness-stratified 2:1 split 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-12 22:41:48 -04:00
iomgaa 811ffa648b feat(types): extend GeneratedQuestion with skill_target & difficulty_steps 
- Add skill_target (str | None) and difficulty_steps (int | None) fields
  to GeneratedQuestion dataclass with field(default=None)
- Update loader.py to pass new fields from JSON (backward-compatible)
- Update pools.py _q_to_dict/_dict_to_q for serialization compat
- Add question_gen_v2 config section to default.yaml
- Add comprehensive test coverage (7 tests)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-11 23:05:38 -04:00
iomgaa 307c64c388 fix(harness): infer mode honors CLI --questions over manifest 2026-07-11 07:45:33 -04:00
iomgaa a7ca6d15ed feat(harness): InferenceDepsRouter per-video 路由器 
按 video_id 懒加载 InferenceDeps 并缓存，路由 dispatch/prompt_builder:
- create_dispatch: 按 session_id 路由到对应视频的工具调度
- create_prompt_builder: 自动注册 qid→vid 映射并路由 prompt 构建
- 三元组 (video_id, skills_dir, prompts_dir) 缓存键

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-09 12:20:32 -04:00
iomgaa f21bf345a6 feat(runner): 注入 tool_dispatch_factory/prompt_builder_factory + fail-fast 
Runner.__init__ 新增 2 个可选参数:
- tool_dispatch_factory: 工具调度工厂
- prompt_builder_factory: prompt 构建工厂

infer/eval/train 模式缺少工厂时 fail-fast 抛 ValueError。
_make_tool_dispatch_fn/_make_prompt_builder 优先使用注入工厂。

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-09 12:20:25 -04:00
iomgaa f57ee45dc0 fix: Codex 全量审查修正 
Critical:
- C1: assemble_mode 'plain' → 'ids'（合法枚举值）
- C2: question_id 加入 task_type slug 避免跨题型冲突

Important/Minor:
- generate_one 移除未用的 embed_fn/similarity_threshold 参数
- config.py 注释 11→12 同步
- 测试 question_id 断言更新

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-09 07:40:27 -04:00
iomgaa 6e46d184b8 feat(harness): add factory.py — InferenceDeps dataclass + build_inference_deps 
组装一次推理所需的全套依赖的工厂函数：
- TreeIndex 加载（FileNotFoundError if missing）
- TreeEnvironment 构建
- SkillRegistry 按需发现
- SearchToolDispatcher 装配
- PromptManager + prompt_builder 闭包

测试覆盖：正常路径、缺失树文件、skills 注入、frozen 不可变性。

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-09 05:36:07 -04:00
iomgaa eb15ab315e fix(config): _VIDEO_MME_TASK_TYPE_COUNT 11→12，Video-MME 实际有 12 种题型 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-09 05:08:05 -04:00
iomgaa 9a42bbcf85 style(harness): ruff format batching.py + log.py 2026-07-07 20:34:48 -04:00
iomgaa a668fc173c feat(harness): __init__.py public API + lint fixes 2026-07-07 13:45:48 -04:00
iomgaa d6e74f2734 feat(harness): runner.py — train loop orchestrator (#13 algorithm fidelity) 
Three-level nesting (epoch -> step -> per-skill), slow update 10-step
sequence, checkpoint/resume, early stop, probation accept/reject/rollback.

Key TRM4->TRM5 changes:
- sync -> async (all inference/diagnosis/evolve/validate awaited)
- LLMClient.from_env -> injected LLMProvider (DI via constructor)
- Direct DB/file access -> module functions (workspace/store/log)
- _TrainState as train() local, explicit param passing to helpers

Module-level pure functions extracted for testability:
resume_plan, _guard_infra_failures, _apply_batch_correctness,
_compute_total_steps, _should_early_stop, _format_applied_edits,
_fallback_summary, _write_skip_report, _outcome_to_quadrant_pairs,
_build_comparison_pairs, _batch_from_ids, _snapshot_current_skills.

Tests: 34 unit tests covering 13a-13e sub-tasks.
Radon: all functions Grade B or better.
 2026-07-07 13:43:20 -04:00
iomgaa 6baddcc17d feat(harness): validate.py — async 块序贯验证编排 + Probation 统一定义 2026-07-07 13:20:43 -04:00
iomgaa 8c9f7d2d37 feat(harness): checkpoint.py — TrainState 序列化 + 原子写 + 指纹校验 2026-07-07 13:13:37 -04:00
iomgaa 6b56fc8f98 feat(harness): inference.py — async run_inference + DI 2026-07-07 13:04:26 -04:00
iomgaa 49e272719e feat(harness): momentum.py — async 慢更新动量生成 2026-07-07 13:01:04 -04:00
iomgaa 767ac55787 feat(harness): observation.py — 五张观测表 + step/epoch 报告 2026-07-07 12:55:12 -04:00
iomgaa d7d8633dae feat(harness): gate_ladder.py — 信息阶梯 + BaselineCache (#6 算法保真) 2026-07-07 12:55:12 -04:00
iomgaa 48b423ef35 feat(harness): pools.py — 三池切分（test→validation→diagnosis） 2026-07-07 12:48:54 -04:00
iomgaa ad8fc176d0 feat(harness): batching.py — FFD + round-robin mini-batch (#10 算法保真) 2026-07-07 12:47:33 -04:00
iomgaa 461abf0329 feat(harness): workspace.py — Workspace lifecycle + VersionedSkillStore/PromptStore 
- ResolvedPaths frozen dataclass: store_dir, videos_dir, questions_dir,
  skills_dir, prompts_dir, workspace_dir, db_path, analyses_dir, runs_dir
- init_workspace: create ws + copy seed weights from store
- init_workspace_from_seed: create from seed with fail-fast questions check
- load_manifest / resolve_paths: manifest I/O + path resolution
  (skills/prompts resolve to workspace, videos/questions to store)
- update_manifest: key whitelist validation
- record_run: idempotent history append + per-video wiki dirs
- read_best / update_best: best pointer independent of current
- list_video_ids: videos with tree.json
- archive_workspace: move to .archive/<name>-<ts>
- VersionedSkillStore: implements core/evolution/protocols.py::SkillStore
- VersionedPromptStore: implements core/evolution/protocols.py::PromptStore
- 21 tests all passing (incl. Protocol compliance checks)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-07 12:41:59 -04:00
iomgaa 6858da80c7 feat(harness): store.py — Store 版本操作 + Seed 管理 
从 TRM4 core/workspace.py 拆出 Store + Seed 相关函数：
- _parse_version / list_versions / next_version / advance_version
- _write_meta / init_store
- init_seed / list_seeds / read_seed
- extract_run_db（保留原始 CREATE 语句重建主键约束）
- promote_to_seed（强校验版本一致 + 非 NULL + finally 清理）

26 个测试全部通过，radon 复杂度 A (2.83)。

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-07 12:36:36 -04:00
iomgaa fd76720dcf feat(harness): HarnessLog SQLite wrapper + RunLogImpl readonly port 
- HarnessLog: TRM4 direct port with WAL mode, threading.Lock, INSERT OR
  IGNORE idempotent _runs, context manager (completed/failed), create_table
  with auto run_id+timestamp, insert/insert_many/execute/query/log_event
- RunLogImpl: implements core/evolution/protocols.py::RunLog Protocol with
  independent sqlite3.connect for read-only SELECT (no _runs pollution),
  asyncio.to_thread wrapping for async interface
- _read_table: shared readonly helper with optional question_ids filtering,
  graceful empty-list return for missing tables
- Tests: 17 cases covering thread safety, idempotent inserts, context
  manager status, WAL mode, protocol compliance, readonly isolation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-07 12:29:29 -04:00
iomgaa b47f634e61 fix(harness): Codex functional review 修复 — 命名/集成测试/delta_min/promote 消息 
- _apply_env_overrides → _apply_env_var_overrides，docstring 明确从 os.environ 读取
- 新增 TestLoadConfigRealYaml：用真实 config/default.yaml 验证嵌套 harness 解析
- 新增 test_delta_min_negative_rejected：覆盖 gate_delta_min >= 0 校验
- 恢复 promote 模式独立错误消息（从合并分支分离回 TRM4 原始提示）
- 77 个单元测试全部通过，radon 全部 Grade B 或更好

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-07 12:23:27 -04:00
iomgaa 535bd320b0 fix(harness): 补充 train 模式 run_id 校验 + 拆分函数保持 radon Grade B 
- _validate_mode_deps: 恢复 train 非 resume/fresh 时必须提供 run_id 校验
- 提取 _validate_train_run_id 用 early return 展平条件，避免 radon Grade C
- 合并 promote run_id 检查到 diagnose/evolve/promote 统一检查
- 新增 4 个测试：train+run_id / train+resume / train+fresh / train+baseline
- radon cc -n C 无输出（全部 Grade B 或更好）
- 74 个单元测试全部通过

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-07 12:14:28 -04:00
iomgaa 5e57cf19b6 refactor(harness): 拆分校验函数降低 radon 圈复杂度至 Grade B 
- _validate: 拆出 _validate_mode（mode 依赖校验）+ _validate_basic（标量/枚举校验）
- _validate_gate: 拆为 _validate_gate_thresholds（e 值/净胜/方向）+ _validate_gate_ladder（阶梯/块序贯）
- load_config: 提取 _apply_env_overrides 函数
- radon cc -n C 无输出（全部 Grade B 或更好）
- 70 个单元测试全部通过

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-07 12:01:46 -04:00
iomgaa ca02b5a91d feat(harness): RunConfig frozen dataclass + 四层校验 + YAML/CLI/.env 三层加载 
- RunConfig: 46 字段 frozen dataclass，从 TRM4 core/harness/config.py 迁移
- 四层校验链：_validate → _validate_edit_budget + _validate_minibatch + _validate_gate
- 新增 .env 覆盖层：工程配置（workspace_dir, store_dir）可通过 HARNESS_* 环境变量注入
- 合并优先级：CLI > .env > YAML（CLAUDE.md §4.5）
- load_config 支持嵌套 harness 段和扁平 YAML 两种格式
- run_id 改为默认空字符串（CLI-only 字段，YAML 不提供）
- resume/fresh 互斥校验不在 config 层（移至 runner.py）
- 70 个单元测试全部通过

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-07 11:57:35 -04:00
iomgaa e60823de1a build: add README, pyproject.toml, package skeletons and smoke test 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-06 11:37:59 -04:00