docs: add TaskTypeStrategy design + Plan A framework plan
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,751 @@
|
||||
# TaskTypeStrategy 框架实现计划 (Plan A)
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** 将出题管线从 QuestionFamilySpec 替换为 TaskTypeStrategy 接口,12 个题型(含 AR 临时 Base)用 BaseTaskTypeStrategy 封装现有 family 行为,pipeline/sampler/generator/gates/store 签名适配。
|
||||
|
||||
**Architecture:** 新建 `strategy.py`(Protocol + SubPattern + Base + 注册表),修改 pipeline/sampler/generator/gates/store 5 个文件的函数签名,从 `family_spec` 参数改为 strategy 提供的离散参数。BaseTaskTypeStrategy 内部委托给绑定的 QuestionFamilySpec,确保 12 个题型行为不变。AR 临时绑定 VISUAL family(Plan B 替换为特化策略)。
|
||||
|
||||
**Codex 审查修复:**
|
||||
- C1: sub_pattern 空值判护 — `sub_pattern.x if sub_pattern else strategy.x`
|
||||
- C2: AR 在 Plan A 临时绑定 VISUAL family,Plan B 替换为 ActionRecognitionStrategy
|
||||
- C3: sampler 异常消息同步改为使用 task_type
|
||||
- C4: run_store 加幂等 ALTER TABLE 迁移
|
||||
- I1: pipeline 显式调用 extra_gates 并合并结果
|
||||
- I2: 移除 build_prompt_context,改为离散参数(prompt_template/strategy_name/skill_target/sub_pattern_instruction)
|
||||
|
||||
**Tech Stack:** Python 3.11, pytest, Protocol (typing)
|
||||
|
||||
**关联设计:** `research-wiki/designs/2026-07-14-task-type-strategy-design.md`
|
||||
|
||||
**范围:** 仅 Plan A(框架 + Base)。ActionRecognitionStrategy 特化实现在 Plan B。
|
||||
|
||||
---
|
||||
|
||||
### Task 1: strategy.py — Protocol + SubPattern + BaseTaskTypeStrategy + 注册表
|
||||
|
||||
**Files:**
|
||||
- Create: `app/question_gen/strategy.py`
|
||||
- Test: `tests/unit/test_strategy.py`
|
||||
|
||||
- [ ] **Step 1: 写失败测试 — Protocol 和 Base 行为**
|
||||
|
||||
```python
|
||||
# tests/unit/test_strategy.py
|
||||
"""TaskTypeStrategy Protocol 与 BaseTaskTypeStrategy 单元测试。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import random
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from app.question_gen.families import (
|
||||
ENUMERATION_FAMILY,
|
||||
REASONING_FAMILY,
|
||||
RETRIEVAL_FAMILY,
|
||||
SPATIAL_FAMILY,
|
||||
VISUAL_FAMILY,
|
||||
)
|
||||
from app.question_gen.strategy import (
|
||||
BaseTaskTypeStrategy,
|
||||
SubPattern,
|
||||
get_strategy,
|
||||
register_strategy,
|
||||
)
|
||||
|
||||
|
||||
class TestBaseTaskTypeStrategy:
|
||||
"""BaseTaskTypeStrategy 封装 family 行为。"""
|
||||
|
||||
def test_task_type_and_strategy_name(self):
|
||||
"""task_type 和 strategy_name 正确返回。"""
|
||||
s = BaseTaskTypeStrategy(task_type="Object Recognition", family=RETRIEVAL_FAMILY)
|
||||
assert s.task_type == "Object Recognition"
|
||||
assert s.strategy_name == "RETRIEVAL"
|
||||
|
||||
def test_sampling_from_family(self):
|
||||
"""采样参数从绑定的 family 读取。"""
|
||||
s = BaseTaskTypeStrategy(task_type="Temporal Reasoning", family=ENUMERATION_FAMILY)
|
||||
assert s.sampling_level == 1
|
||||
assert s.sampling_constraint.min_l3_nodes == 5
|
||||
|
||||
def test_skill_target_from_family(self):
|
||||
"""skill_target 从 family 读取。"""
|
||||
s = BaseTaskTypeStrategy(task_type="Action Reasoning", family=REASONING_FAMILY)
|
||||
assert s.skill_target == "M2"
|
||||
|
||||
def test_leak_probe_template(self):
|
||||
"""leak_probe_template 从 family.leak_profile 读取。"""
|
||||
s = BaseTaskTypeStrategy(task_type="Spatial Reasoning", family=SPATIAL_FAMILY)
|
||||
assert s.leak_probe_template == "gate_leak_spatial.md"
|
||||
|
||||
def test_prompt_template_from_family(self):
|
||||
"""prompt_template 从 family 读取。"""
|
||||
s = BaseTaskTypeStrategy(task_type="OCR Problems", family=VISUAL_FAMILY)
|
||||
assert s.prompt_template == "visual.md"
|
||||
|
||||
def test_select_sub_pattern_returns_none(self):
|
||||
"""BaseTaskTypeStrategy 无子模式。"""
|
||||
s = BaseTaskTypeStrategy(task_type="Object Recognition", family=RETRIEVAL_FAMILY)
|
||||
rng = random.Random(42)
|
||||
assert s.select_sub_pattern(rng) is None
|
||||
|
||||
def test_extra_gates_returns_empty(self):
|
||||
"""BaseTaskTypeStrategy 无额外 gate。"""
|
||||
s = BaseTaskTypeStrategy(task_type="Object Recognition", family=RETRIEVAL_FAMILY)
|
||||
assert s.extra_gates(None) == []
|
||||
|
||||
|
||||
class TestSamplingLevelMapping:
|
||||
"""BaseTaskTypeStrategy 的 sampling_level 从 _TASK_TYPE_TO_LEVEL 读取。"""
|
||||
|
||||
def test_l3_types(self):
|
||||
"""L3 题型。"""
|
||||
for tt in ("Object Recognition",):
|
||||
s = BaseTaskTypeStrategy(task_type=tt, family=RETRIEVAL_FAMILY)
|
||||
assert s.sampling_level == 3, f"{tt} should be L3"
|
||||
|
||||
def test_l2_types(self):
|
||||
"""L2 题型。"""
|
||||
s = BaseTaskTypeStrategy(task_type="Action Reasoning", family=REASONING_FAMILY)
|
||||
assert s.sampling_level == 2
|
||||
|
||||
def test_l1_types(self):
|
||||
"""L1 题型。"""
|
||||
s = BaseTaskTypeStrategy(task_type="Temporal Reasoning", family=ENUMERATION_FAMILY)
|
||||
assert s.sampling_level == 1
|
||||
|
||||
|
||||
class TestStrategyRegistry:
|
||||
"""注册表查找。"""
|
||||
|
||||
def test_get_unregistered_returns_base(self):
|
||||
"""未注册题型返回 BaseTaskTypeStrategy。"""
|
||||
s = get_strategy("Spatial Reasoning")
|
||||
assert isinstance(s, BaseTaskTypeStrategy)
|
||||
assert s.task_type == "Spatial Reasoning"
|
||||
|
||||
def test_register_and_get(self):
|
||||
"""注册后 get 返回注册的策略。"""
|
||||
custom = BaseTaskTypeStrategy(task_type="Object Recognition", family=RETRIEVAL_FAMILY)
|
||||
register_strategy(custom)
|
||||
assert get_strategy("Object Recognition") is custom
|
||||
|
||||
|
||||
class TestSubPattern:
|
||||
"""SubPattern 数据类。"""
|
||||
|
||||
def test_frozen(self):
|
||||
"""SubPattern 不可变。"""
|
||||
sp = SubPattern(
|
||||
name="test",
|
||||
weight=0.5,
|
||||
sampling_level_override=None,
|
||||
constraint_override=None,
|
||||
instruction="test instruction",
|
||||
positive_examples=[],
|
||||
negative_examples=[],
|
||||
distractor_rules="",
|
||||
)
|
||||
with pytest.raises(AttributeError):
|
||||
sp.name = "changed"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: 运行测试验证失败**
|
||||
|
||||
```bash
|
||||
conda activate Video-Tree-TRM & pytest tests/unit/test_strategy.py -v
|
||||
```
|
||||
|
||||
预期:ImportError — `app.question_gen.strategy` 不存在。
|
||||
|
||||
- [ ] **Step 3: 实现 strategy.py**
|
||||
|
||||
```python
|
||||
# app/question_gen/strategy.py
|
||||
"""题型出题策略 — Clean Architecture 的 Strategy 层。
|
||||
|
||||
将出题管线从 5 个粗粒度 QuestionFamilySpec 替换为 12 个题型级别的
|
||||
TaskTypeStrategy。BaseTaskTypeStrategy 封装现有 family 行为,
|
||||
特化策略(如 ActionRecognitionStrategy)自包含。
|
||||
|
||||
典型用法::
|
||||
|
||||
strategy = get_strategy("Action Reasoning")
|
||||
level = strategy.sampling_level
|
||||
constraint = strategy.sampling_constraint
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any, Protocol, runtime_checkable
|
||||
|
||||
from app.question_gen.families import (
|
||||
ENUMERATION_FAMILY,
|
||||
REASONING_FAMILY,
|
||||
RETRIEVAL_FAMILY,
|
||||
SPATIAL_FAMILY,
|
||||
VISUAL_FAMILY,
|
||||
QuestionFamilySpec,
|
||||
SamplingConstraint,
|
||||
)
|
||||
from app.question_gen.sampler_v2 import _TASK_TYPE_TO_LEVEL
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import random
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SubPattern
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SubPattern:
|
||||
"""出题子模式 — 靶向特定失败机制。
|
||||
|
||||
属性:
|
||||
name: 子模式标识名。
|
||||
weight: 选择权重(按比例分配)。
|
||||
sampling_level_override: 覆盖策略默认采样层级(None = 不覆盖)。
|
||||
constraint_override: 覆盖策略默认采样约束(None = 不覆盖)。
|
||||
instruction: 核心出题指令(1-3 句话)。
|
||||
positive_examples: VME 原题 few-shot 示范。
|
||||
negative_examples: 反面示例。
|
||||
distractor_rules: 干扰项构造规则。
|
||||
"""
|
||||
|
||||
name: str
|
||||
weight: float
|
||||
sampling_level_override: int | None
|
||||
constraint_override: SamplingConstraint | None
|
||||
instruction: str
|
||||
positive_examples: list[dict] = field(default_factory=list)
|
||||
negative_examples: list[dict] = field(default_factory=list)
|
||||
distractor_rules: str = ""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Protocol
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class TaskTypeStrategy(Protocol):
|
||||
"""题型出题策略 — pipeline 的唯一接口。"""
|
||||
|
||||
@property
|
||||
def task_type(self) -> str: ...
|
||||
|
||||
@property
|
||||
def sampling_level(self) -> int: ...
|
||||
|
||||
@property
|
||||
def sampling_constraint(self) -> SamplingConstraint: ...
|
||||
|
||||
@property
|
||||
def prompt_template(self) -> str: ...
|
||||
|
||||
@property
|
||||
def strategy_name(self) -> str: ...
|
||||
|
||||
@property
|
||||
def skill_target(self) -> str: ...
|
||||
|
||||
@property
|
||||
def leak_probe_template(self) -> str: ...
|
||||
|
||||
def select_sub_pattern(self, rng: random.Random) -> SubPattern | None: ...
|
||||
|
||||
def build_prompt_context(
|
||||
self, material: Any, sub_pattern: SubPattern | None
|
||||
) -> dict: ...
|
||||
|
||||
def extra_gates(self, candidate: Any) -> list: ...
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# BaseTaskTypeStrategy
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# 消歧绑定表:多归属题型确定性绑定到一个 family
|
||||
_TASK_TYPE_TO_FAMILY: dict[str, QuestionFamilySpec] = {
|
||||
"Action Recognition": VISUAL_FAMILY, # Plan A 临时绑定;Plan B 替换为特化策略
|
||||
"Object Recognition": RETRIEVAL_FAMILY,
|
||||
"Object Reasoning": REASONING_FAMILY,
|
||||
"Action Reasoning": REASONING_FAMILY,
|
||||
"Attribute Perception": VISUAL_FAMILY,
|
||||
"OCR Problems": VISUAL_FAMILY,
|
||||
"Counting Problem": ENUMERATION_FAMILY,
|
||||
"Information Synopsis": REASONING_FAMILY,
|
||||
"Temporal Reasoning": ENUMERATION_FAMILY,
|
||||
"Temporal Perception": ENUMERATION_FAMILY,
|
||||
"Spatial Reasoning": SPATIAL_FAMILY,
|
||||
"Spatial Perception": SPATIAL_FAMILY,
|
||||
}
|
||||
|
||||
|
||||
class BaseTaskTypeStrategy:
|
||||
"""封装现有 QuestionFamilySpec 行为的默认策略。
|
||||
|
||||
所有属性委托给绑定的 family,确保未特化题型的行为不变。
|
||||
|
||||
参数:
|
||||
task_type: 题型名。
|
||||
family: 绑定的 QuestionFamilySpec。
|
||||
"""
|
||||
|
||||
def __init__(self, task_type: str, family: QuestionFamilySpec) -> None:
|
||||
self._task_type = task_type
|
||||
self._family = family
|
||||
|
||||
@property
|
||||
def task_type(self) -> str:
|
||||
return self._task_type
|
||||
|
||||
@property
|
||||
def sampling_level(self) -> int:
|
||||
return _TASK_TYPE_TO_LEVEL[self._task_type]
|
||||
|
||||
@property
|
||||
def sampling_constraint(self) -> SamplingConstraint:
|
||||
return self._family.sampling
|
||||
|
||||
@property
|
||||
def prompt_template(self) -> str:
|
||||
return self._family.prompt_template
|
||||
|
||||
@property
|
||||
def strategy_name(self) -> str:
|
||||
return self._family.name
|
||||
|
||||
@property
|
||||
def skill_target(self) -> str:
|
||||
return self._family.skill_target
|
||||
|
||||
@property
|
||||
def leak_probe_template(self) -> str:
|
||||
return self._family.leak_profile.probe_template
|
||||
|
||||
def select_sub_pattern(self, rng: random.Random) -> SubPattern | None:
|
||||
"""BaseTaskTypeStrategy 无子模式。"""
|
||||
return None
|
||||
|
||||
def build_prompt_context(
|
||||
self, material: Any, sub_pattern: SubPattern | None
|
||||
) -> dict:
|
||||
"""返回基础 prompt 上下文(family_name + prompt_template)。"""
|
||||
return {
|
||||
"family_name": self._family.name,
|
||||
"prompt_template": self._family.prompt_template,
|
||||
}
|
||||
|
||||
def extra_gates(self, candidate: Any) -> list:
|
||||
"""BaseTaskTypeStrategy 无额外 gate。"""
|
||||
return []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 注册表
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_STRATEGY_REGISTRY: dict[str, TaskTypeStrategy] = {}
|
||||
|
||||
|
||||
def register_strategy(strategy: TaskTypeStrategy) -> None:
|
||||
"""注册一个题型策略。同一 task_type 重复注册会覆盖。
|
||||
|
||||
参数:
|
||||
strategy: 实现 TaskTypeStrategy 接口的策略实例。
|
||||
"""
|
||||
_STRATEGY_REGISTRY[strategy.task_type] = strategy
|
||||
|
||||
|
||||
def get_strategy(task_type: str) -> TaskTypeStrategy:
|
||||
"""获取题型策略。未注册的自动创建 BaseTaskTypeStrategy。
|
||||
|
||||
参数:
|
||||
task_type: 题型名。
|
||||
|
||||
返回:
|
||||
TaskTypeStrategy 实例。
|
||||
|
||||
异常:
|
||||
KeyError: task_type 不在消歧绑定表和注册表中。
|
||||
"""
|
||||
if task_type in _STRATEGY_REGISTRY:
|
||||
return _STRATEGY_REGISTRY[task_type]
|
||||
return _build_default_strategy(task_type)
|
||||
|
||||
|
||||
def _build_default_strategy(task_type: str) -> BaseTaskTypeStrategy:
|
||||
"""根据消歧绑定表自动构造 BaseTaskTypeStrategy。
|
||||
|
||||
参数:
|
||||
task_type: 题型名。
|
||||
|
||||
返回:
|
||||
BaseTaskTypeStrategy 实例。
|
||||
|
||||
异常:
|
||||
KeyError: task_type 不在消歧绑定表中。
|
||||
"""
|
||||
family = _TASK_TYPE_TO_FAMILY[task_type]
|
||||
return BaseTaskTypeStrategy(task_type=task_type, family=family)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: 运行测试验证通过**
|
||||
|
||||
```bash
|
||||
conda activate Video-Tree-TRM & pytest tests/unit/test_strategy.py -v
|
||||
```
|
||||
|
||||
预期:全部 PASS
|
||||
|
||||
- [ ] **Step 5: 提交**
|
||||
|
||||
```bash
|
||||
git add app/question_gen/strategy.py tests/unit/test_strategy.py
|
||||
git commit -m "feat(question_gen): add TaskTypeStrategy Protocol and BaseTaskTypeStrategy"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: sampler_v2.py 签名适配
|
||||
|
||||
**Files:**
|
||||
- Modify: `app/question_gen/sampler_v2.py:495-530`
|
||||
- Test: `tests/unit/test_sampler_v2.py`(现有测试应继续通过)
|
||||
|
||||
- [ ] **Step 1: 修改 sample_material_v2 签名**
|
||||
|
||||
将 `family_spec: QuestionFamilySpec` 参数替换为 `level: int` + `constraint: SamplingConstraint`:
|
||||
|
||||
在 `app/question_gen/sampler_v2.py` 中,修改 `sample_material_v2` 函数签名和内部使用:
|
||||
|
||||
```python
|
||||
def sample_material_v2(
|
||||
tree: TreeIndex,
|
||||
task_type: str,
|
||||
used_node_ids: set[str],
|
||||
rng: random.Random,
|
||||
*,
|
||||
level: int,
|
||||
constraint: SamplingConstraint,
|
||||
max_attempts: int = 10,
|
||||
) -> MaterialContext:
|
||||
```
|
||||
|
||||
函数体内将 `level = _TASK_TYPE_TO_LEVEL[task_type]` 和 `constraint = family_spec.sampling` 两行删除(改为直接使用参数)。
|
||||
|
||||
- [ ] **Step 2: 修改 pipeline_v2.py 中的调用点**
|
||||
|
||||
在 `_process_one_slot` 中(约 371 行),将:
|
||||
|
||||
```python
|
||||
material = sample_material_v2(
|
||||
tree=current_tree,
|
||||
family_spec=slot.family,
|
||||
task_type=slot.task_type,
|
||||
used_node_ids=used_node_ids,
|
||||
rng=rng,
|
||||
)
|
||||
```
|
||||
|
||||
改为:
|
||||
|
||||
```python
|
||||
material = sample_material_v2(
|
||||
tree=current_tree,
|
||||
task_type=slot.task_type,
|
||||
used_node_ids=used_node_ids,
|
||||
rng=rng,
|
||||
level=strategy.sampling_level,
|
||||
constraint=strategy.sampling_constraint,
|
||||
)
|
||||
```
|
||||
|
||||
(`strategy` 变量在 Task 4 中引入,此处先改签名。)
|
||||
|
||||
- [ ] **Step 3: 运行 sampler 测试**
|
||||
|
||||
```bash
|
||||
conda activate Video-Tree-TRM & pytest tests/unit/test_sampler_v2.py -v
|
||||
```
|
||||
|
||||
预期:需要更新测试中的调用签名。修复后全部 PASS。
|
||||
|
||||
- [ ] **Step 4: 提交**
|
||||
|
||||
```bash
|
||||
git add app/question_gen/sampler_v2.py tests/unit/test_sampler_v2.py
|
||||
git commit -m "refactor(sampler): replace family_spec param with level+constraint"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: generator_v2.py + gates.py + run_store.py 签名适配
|
||||
|
||||
**Files:**
|
||||
- Modify: `app/question_gen/generator_v2.py:88-104, 112-194, 351-420`
|
||||
- Modify: `app/question_gen/gates.py:399-431, 439-494`
|
||||
- Modify: `app/question_gen/run_store.py:110-128, 242-290`
|
||||
|
||||
- [ ] **Step 1: generator_v2.py — _load_prompt_template 改为接收文件名**
|
||||
|
||||
```python
|
||||
def _load_prompt_template(template_name: str) -> str:
|
||||
path = _PROMPTS_DIR / template_name
|
||||
if not path.exists():
|
||||
msg = f"Prompt 模板文件不存在: {path}"
|
||||
raise FileNotFoundError(msg)
|
||||
return path.read_text(encoding="utf-8")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: generator_v2.py — _build_v2_prompt 改为接收离散参数**
|
||||
|
||||
```python
|
||||
def _build_v2_prompt(
|
||||
prompt_template: str,
|
||||
strategy_name: str,
|
||||
material: MaterialContext,
|
||||
task_type: str,
|
||||
seq: int,
|
||||
*,
|
||||
reject_reason: str | None = None,
|
||||
sub_pattern_instruction: str | None = None,
|
||||
) -> tuple[list[dict[str, str]], list[str]]:
|
||||
```
|
||||
|
||||
函数体内:
|
||||
- `template_content = _load_prompt_template(family_spec)` → `template_content = _load_prompt_template(prompt_template)`
|
||||
- `family_spec.name` → `strategy_name`
|
||||
- 在 `reject_reason` 注入之后、输出格式之前,如果 `sub_pattern_instruction` 非 None,追加:
|
||||
|
||||
```python
|
||||
if sub_pattern_instruction is not None:
|
||||
user_parts.append(f"\n## Special Focus:\n{sub_pattern_instruction}")
|
||||
```
|
||||
|
||||
- [ ] **Step 3: generator_v2.py — generate_one_v2 签名适配**
|
||||
|
||||
```python
|
||||
async def generate_one_v2(
|
||||
vlm: VLMProvider,
|
||||
tree: TreeIndex,
|
||||
material: MaterialContext,
|
||||
task_type: str,
|
||||
seq: int,
|
||||
*,
|
||||
video_id: str,
|
||||
prompt_template: str,
|
||||
strategy_name: str,
|
||||
skill_target: str,
|
||||
reject_reason: str | None = None,
|
||||
sub_pattern_instruction: str | None = None,
|
||||
session_id: str,
|
||||
) -> CandidateQuestion:
|
||||
```
|
||||
|
||||
函数体内:
|
||||
- `_build_v2_prompt(family_spec=family_spec, ...)` → `_build_v2_prompt(prompt_template=prompt_template, strategy_name=strategy_name, ..., sub_pattern_instruction=sub_pattern_instruction)`
|
||||
- `family_spec.name` → `strategy_name`
|
||||
- `family_spec.skill_target` → `skill_target`
|
||||
|
||||
- [ ] **Step 4: gates.py — _gate_leak_test 和 run_gates 签名适配**
|
||||
|
||||
`_gate_leak_test`:
|
||||
|
||||
```python
|
||||
async def _gate_leak_test(
|
||||
candidate: CandidateQuestion,
|
||||
leak_probe_template: str,
|
||||
llm: LLMProvider,
|
||||
*,
|
||||
session_id: str,
|
||||
) -> GateResult:
|
||||
```
|
||||
|
||||
函数体内:`probe_template_name = family_spec.leak_profile.probe_template` → `probe_template_name = leak_probe_template`
|
||||
|
||||
`run_gates`:
|
||||
|
||||
```python
|
||||
async def run_gates(
|
||||
candidate: CandidateQuestion,
|
||||
tree: TreeIndex,
|
||||
llm: LLMProvider,
|
||||
leak_probe_template: str,
|
||||
postprocess: PostprocessResult,
|
||||
*,
|
||||
vlm: VLMProvider | None = None,
|
||||
session_id: str,
|
||||
) -> GateReport:
|
||||
```
|
||||
|
||||
函数体内:`_gate_leak_test(candidate, family_spec, llm, ...)` → `_gate_leak_test(candidate, leak_probe_template, llm, ...)`
|
||||
|
||||
- [ ] **Step 5: run_store.py — DDL 加 sub_pattern 列 + record_item 加参数**
|
||||
|
||||
DDL `_DDL_ITEMS` 在 `difficulty_steps` 行后加:
|
||||
|
||||
```sql
|
||||
sub_pattern TEXT,
|
||||
```
|
||||
|
||||
`record_item` 新增参数 `sub_pattern: str | None = None`,INSERT 语句加入该列。
|
||||
|
||||
同时在 `_init_schema` 方法中加幂等迁移(兼容已有数据库):
|
||||
|
||||
```python
|
||||
# 幂等迁移:为已有表加 sub_pattern 列
|
||||
cols = {r[1] for r in self._conn.execute("PRAGMA table_info(question_gen_items)")}
|
||||
if "sub_pattern" not in cols:
|
||||
self._conn.execute("ALTER TABLE question_gen_items ADD COLUMN sub_pattern TEXT")
|
||||
```
|
||||
|
||||
- [ ] **Step 6: 运行全部出题管线测试**
|
||||
|
||||
```bash
|
||||
conda activate Video-Tree-TRM & pytest tests/unit/test_generate_questions.py tests/unit/test_gates.py tests/unit/test_run_store.py tests/unit/test_sampler_v2.py tests/unit/test_families.py tests/integration/test_pipeline_v2.py tests/integration/test_cli_generate_v2.py -v --tb=short
|
||||
```
|
||||
|
||||
预期:需更新测试中的调用签名。修复后全部 PASS。
|
||||
|
||||
- [ ] **Step 7: 提交**
|
||||
|
||||
```bash
|
||||
git add app/question_gen/generator_v2.py app/question_gen/gates.py app/question_gen/run_store.py
|
||||
git commit -m "refactor(question_gen): adapt generator/gates/store signatures for strategy"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: pipeline_v2.py 集成 — 用 strategy 替换 family
|
||||
|
||||
**Files:**
|
||||
- Modify: `app/question_gen/pipeline_v2.py:37, 57-73, 170-211, 300-530`
|
||||
|
||||
- [ ] **Step 1: SlotAssignment 移除 family 字段,改为 strategy_name**
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class SlotAssignment:
|
||||
slot_id: str
|
||||
video_id: str
|
||||
task_type: str
|
||||
seq: int
|
||||
```
|
||||
|
||||
- [ ] **Step 2: _assign_slots 移除 family 选择逻辑**
|
||||
|
||||
```python
|
||||
def _assign_slots(
|
||||
video_ids: list[str],
|
||||
task_types: list[str],
|
||||
per_type: int,
|
||||
) -> list[SlotAssignment]:
|
||||
slots: list[SlotAssignment] = []
|
||||
global_seq = 0
|
||||
for task_type in task_types:
|
||||
for i in range(per_type):
|
||||
video_id = video_ids[i % len(video_ids)]
|
||||
global_seq += 1
|
||||
slot_id = f"{task_type}_{global_seq:04d}"
|
||||
slots.append(SlotAssignment(
|
||||
slot_id=slot_id,
|
||||
video_id=video_id,
|
||||
task_type=task_type,
|
||||
seq=global_seq,
|
||||
))
|
||||
return slots
|
||||
```
|
||||
|
||||
不再调用 `get_family_for_slot`,不再需要 `family_ratios` 和 `rng` 参数。
|
||||
|
||||
- [ ] **Step 3: _process_one_slot 通过 strategy 获取参数**
|
||||
|
||||
在函数开头加:
|
||||
|
||||
```python
|
||||
from app.question_gen.strategy import get_strategy
|
||||
strategy = get_strategy(slot.task_type)
|
||||
sub_pattern = strategy.select_sub_pattern(rng)
|
||||
```
|
||||
|
||||
替换所有 `slot.family` 引用:
|
||||
- `slot.family.name` → `strategy.strategy_name`
|
||||
- `slot.family.skill_target` → `strategy.skill_target`
|
||||
- `sample_material_v2(...)` → 采样参数带空值判护:
|
||||
```python
|
||||
level = sub_pattern.sampling_level_override if sub_pattern and sub_pattern.sampling_level_override is not None else strategy.sampling_level
|
||||
constraint = sub_pattern.constraint_override if sub_pattern and sub_pattern.constraint_override is not None else strategy.sampling_constraint
|
||||
sample_material_v2(..., level=level, constraint=constraint, ...)
|
||||
```
|
||||
- `generate_one_v2(...)` → `generate_one_v2(..., prompt_template=strategy.prompt_template, strategy_name=strategy.strategy_name, skill_target=strategy.skill_target, sub_pattern_instruction=sub_pattern.instruction if sub_pattern else None, ...)`
|
||||
- `run_gates(...)` → `run_gates(..., leak_probe_template=strategy.leak_probe_template, ...)`
|
||||
- `store.record_item(...)` → `store.record_item(..., family=strategy.strategy_name, skill_target=strategy.skill_target, sub_pattern=sub_pattern.name if sub_pattern else None, ...)`
|
||||
- gate 后追加 extra_gates 调用:
|
||||
```python
|
||||
extra_results = strategy.extra_gates(candidate)
|
||||
if any(r.verdict == GateVerdict.FAIL for r in extra_results):
|
||||
prev_reason = "; ".join(r.reason for r in extra_results if r.verdict == GateVerdict.FAIL)
|
||||
continue
|
||||
```
|
||||
|
||||
- [ ] **Step 4: run_pipeline_v2 移除 family_ratios 依赖**
|
||||
|
||||
`_assign_slots` 调用处移除 `config.family_ratios` 和 `rng` 参数。
|
||||
|
||||
- [ ] **Step 5: 运行集成测试**
|
||||
|
||||
```bash
|
||||
conda activate Video-Tree-TRM & pytest tests/integration/test_pipeline_v2.py tests/integration/test_cli_generate_v2.py -v --tb=short
|
||||
```
|
||||
|
||||
预期:需更新测试中的 SlotAssignment/family 引用。修复后全部 PASS。
|
||||
|
||||
- [ ] **Step 6: 提交**
|
||||
|
||||
```bash
|
||||
git add app/question_gen/pipeline_v2.py
|
||||
git commit -m "feat(pipeline): replace QuestionFamilySpec with TaskTypeStrategy"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: 全量回归测试 + lint
|
||||
|
||||
- [ ] **Step 1: Ruff**
|
||||
|
||||
```bash
|
||||
conda activate Video-Tree-TRM & ruff format app/question_gen/ tests/ && ruff check app/question_gen/ --fix
|
||||
```
|
||||
|
||||
- [ ] **Step 2: 全量测试**
|
||||
|
||||
```bash
|
||||
conda activate Video-Tree-TRM & pytest tests/unit/ tests/integration/ -v --tb=short
|
||||
```
|
||||
|
||||
预期:1160+ 全部 PASS
|
||||
|
||||
- [ ] **Step 3: 提交**
|
||||
|
||||
```bash
|
||||
git add -A && git commit -m "chore: lint and format strategy framework changes"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 核心算法保真校验
|
||||
|
||||
本计划不涉及核心算法迁移。修改仅限于出题管线的接口签名替换:
|
||||
- sampler/generator/gates/store 的函数签名从 `family_spec` 改为离散参数
|
||||
- pipeline 的 slot 分配从 family 随机选择改为 strategy 确定性查找
|
||||
- 所有核心出题逻辑(重出循环、后处理、四门 gate 内部、去重、heavy_check)不变
|
||||
|
||||
保真校验不适用。
|
||||
@@ -0,0 +1,9 @@
|
||||
---
|
||||
type: plan
|
||||
node_id: plan:action-recognition-training
|
||||
title: "Action Recognition 单题型首次训练实验计划"
|
||||
date: 2026-07-14
|
||||
---
|
||||
|
||||
# Action Recognition 单题型首次训练实验计划
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
---
|
||||
type: plan
|
||||
node_id: plan:per-category-pool-strategy
|
||||
title: "Per-Category Pool Strategy 实现计划"
|
||||
date: 2026-07-13
|
||||
---
|
||||
|
||||
# Per-Category Pool Strategy 实现计划
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
---
|
||||
type: plan
|
||||
node_id: plan:task-type-strategy-framework
|
||||
title: "TaskTypeStrategy 框架实现计划 (Plan A)"
|
||||
date: 2026-07-14
|
||||
---
|
||||
|
||||
# TaskTypeStrategy 框架实现计划 (Plan A)
|
||||
|
||||
Reference in New Issue
Block a user