775 lines
30 KiB
Markdown
775 lines
30 KiB
Markdown
# ActionRecognitionStrategy 特化实现计划 (Plan B)
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** 实现 ActionRecognitionStrategy 特化策略（6 个 SubPattern + AR 专属 prompt），替换 Plan A 中 AR 的临时 VISUAL_FAMILY 绑定，使 AR 出题靶向 22 道错题的 6 种失败子模式。
|
||
|
||
**Architecture:** 新建 `strategy_action_recognition.py`（ActionRecognitionStrategy 类 + 6 个 SubPattern 定义），新建 `action_recognition.md` prompt 模板，在 `strategy.py` 的模块初始化中注册。Pipeline 无需改动 —— Plan A 已接好全部 sub_pattern 接口（`instruction`、`sampling_level_override`、`constraint_override`）。
|
||
|
||
**Tech Stack:** Python 3.11, pytest, Protocol (typing)
|
||
|
||
**关联设计:** `research-wiki/designs/2026-07-14-task-type-strategy-design.md` §4
|
||
|
||
**范围:** 仅 Plan B（ActionRecognitionStrategy 特化）。不改 pipeline、sampler、generator、gates、store。
|
||
|
||
---
|
||
|
||
### Task 1: AR 专属 prompt 模板
|
||
|
||
**Files:**
|
||
- Create: `store/prompts/question_gen/action_recognition.md`
|
||
|
||
- [ ] **Step 1: 创建 AR 专属 prompt 模板**
|
||
|
||
```markdown
|
||
You are a question generator for video understanding benchmarks, specializing in **Action Recognition**.
|
||
|
||
Your task: Generate a multiple-choice question that tests whether the answerer can accurately recognize, distinguish, and reason about **actions and behaviors** observed across multiple segments of the video.
|
||
|
||
## Action Recognition Guidelines
|
||
|
||
- The question MUST require watching multiple segments or the full video — single-frame-answerable questions are failures.
|
||
- Focus on **dynamic actions**: what someone does, how they do it, the sequence of actions, or which action is absent.
|
||
- The correct answer must be grounded in observable behavior (body movements, interactions, operations), NOT in static visual attributes or text/OCR.
|
||
- Questions should target action details that require temporal tracking: order of events, manner of execution, repetition counts, or cross-segment entity behavior.
|
||
|
||
## Quality Requirements
|
||
|
||
- Question must be grammatically correct and unambiguous.
|
||
- All four options must be parallel in structure and length.
|
||
- The correct answer must not be identifiable from linguistic cues alone.
|
||
- Each option must begin with "A. ", "B. ", "C. ", or "D. ".
|
||
|
||
## Prohibited Patterns
|
||
|
||
- Do NOT generate questions answerable from a single frame or screenshot — if pausing the video at one moment gives the answer, the question is too easy.
|
||
- Do NOT generate OCR/text-reading questions disguised as action recognition — reading jersey numbers, scoreboards, or on-screen text is NOT action recognition.
|
||
- Do NOT fabricate actions not observable in the provided material.
|
||
- Do NOT construct options where the correct answer is obvious from common sense or world knowledge alone.
|
||
- Do NOT write questions where multiple options could reasonably be correct.
|
||
|
||
## Output
|
||
|
||
Respond with ONLY a valid JSON object. No additional text.
|
||
```
|
||
|
||
- [ ] **Step 2: 验证模板可加载**
|
||
|
||
```bash
|
||
conda run -n Video-Tree-TRM python -c "
|
||
from app.question_gen.generator_v2 import _load_prompt_template
|
||
content = _load_prompt_template('action_recognition.md')
|
||
assert 'Action Recognition' in content
|
||
print('OK: action_recognition.md loaded successfully')
|
||
"
|
||
```
|
||
|
||
预期：打印 OK 消息，无异常。
|
||
|
||
- [ ] **Step 3: 提交**
|
||
|
||
```bash
|
||
git add store/prompts/question_gen/action_recognition.md
|
||
git commit -m "feat(question_gen): add Action Recognition specialized prompt template"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 2: ActionRecognitionStrategy 类 + 6 个 SubPattern
|
||
|
||
**Files:**
|
||
- Create: `app/question_gen/strategy_action_recognition.py`
|
||
- Test: `tests/unit/test_strategy_action_recognition.py`
|
||
|
||
- [ ] **Step 1: 写失败测试**
|
||
|
||
```python
|
||
# tests/unit/test_strategy_action_recognition.py
|
||
"""ActionRecognitionStrategy 单元测试。"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import random
|
||
|
||
import pytest
|
||
|
||
from app.question_gen.strategy_action_recognition import (
|
||
AR_SUB_PATTERNS,
|
||
ActionRecognitionStrategy,
|
||
)
|
||
|
||
|
||
class TestActionRecognitionStrategy:
|
||
"""ActionRecognitionStrategy 属性和行为。"""
|
||
|
||
@pytest.fixture
|
||
def strategy(self) -> ActionRecognitionStrategy:
|
||
"""创建策略实例。"""
|
||
return ActionRecognitionStrategy()
|
||
|
||
def test_task_type(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""task_type 固定为 Action Recognition。"""
|
||
assert strategy.task_type == "Action Recognition"
|
||
|
||
def test_strategy_name(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""strategy_name 为 ACTION_RECOGNITION。"""
|
||
assert strategy.strategy_name == "ACTION_RECOGNITION"
|
||
|
||
def test_skill_target(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""skill_target 为 M1_AR。"""
|
||
assert strategy.skill_target == "M1_AR"
|
||
|
||
def test_sampling_level(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""默认采样层级为 L2。"""
|
||
assert strategy.sampling_level == 2
|
||
|
||
def test_sampling_constraint(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""采样约束加强：min_subtitles=3, min_l3_nodes=5, require_frames=True, cross_l2_span=True。"""
|
||
c = strategy.sampling_constraint
|
||
assert c.min_subtitles == 3
|
||
assert c.min_l3_nodes == 5
|
||
assert c.require_frames is True
|
||
assert c.cross_l2_span is True
|
||
|
||
def test_prompt_template(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""使用 AR 专属 prompt 模板。"""
|
||
assert strategy.prompt_template == "action_recognition.md"
|
||
|
||
def test_leak_probe_template(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""复用 RETRIEVAL 的泄漏检测模板。"""
|
||
assert strategy.leak_probe_template == "gate_leak_retrieval.md"
|
||
|
||
def test_extra_gates_empty(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""当前版本无额外 gate。"""
|
||
assert strategy.extra_gates(None) == []
|
||
|
||
|
||
class TestSubPatternSelection:
|
||
"""select_sub_pattern 行为。"""
|
||
|
||
@pytest.fixture
|
||
def strategy(self) -> ActionRecognitionStrategy:
|
||
return ActionRecognitionStrategy()
|
||
|
||
def test_returns_sub_pattern(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""select_sub_pattern 返回 SubPattern 而非 None。"""
|
||
rng = random.Random(42)
|
||
sp = strategy.select_sub_pattern(rng)
|
||
assert sp is not None
|
||
assert sp.name in [p.name for p in AR_SUB_PATTERNS]
|
||
|
||
def test_deterministic_with_same_seed(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""相同 seed 返回相同 SubPattern。"""
|
||
sp1 = strategy.select_sub_pattern(random.Random(42))
|
||
sp2 = strategy.select_sub_pattern(random.Random(42))
|
||
assert sp1.name == sp2.name
|
||
|
||
def test_distribution_covers_all_patterns(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""足够多次采样应覆盖全部 6 个子模式。"""
|
||
rng = random.Random(123)
|
||
names = {strategy.select_sub_pattern(rng).name for _ in range(200)}
|
||
assert len(names) == 6
|
||
|
||
def test_sub_pattern_has_instruction(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""每个 SubPattern 都有非空 instruction。"""
|
||
for sp in AR_SUB_PATTERNS:
|
||
assert sp.instruction.strip(), f"{sp.name} 的 instruction 为空"
|
||
|
||
def test_sub_pattern_has_distractor_rules(self, strategy: ActionRecognitionStrategy) -> None:
|
||
"""每个 SubPattern 都有非空 distractor_rules。"""
|
||
for sp in AR_SUB_PATTERNS:
|
||
assert sp.distractor_rules.strip(), f"{sp.name} 的 distractor_rules 为空"
|
||
|
||
|
||
class TestSubPatternOverrides:
|
||
"""SubPattern 的 sampling 覆盖。"""
|
||
|
||
def test_l1_patterns_override_level(self) -> None:
|
||
"""L1 子模式覆盖默认的 L2 采样层级。"""
|
||
l1_names = {"premature_evidence_anchoring", "temporal_reasoning_failure", "cross_segment_entity_tracking"}
|
||
for sp in AR_SUB_PATTERNS:
|
||
if sp.name in l1_names:
|
||
assert sp.sampling_level_override == 1, f"{sp.name} 应覆盖为 L1"
|
||
else:
|
||
assert sp.sampling_level_override is None, f"{sp.name} 不应覆盖采样层级"
|
||
|
||
def test_weights_sum_to_one(self) -> None:
|
||
"""权重总和为 1.0。"""
|
||
total = sum(sp.weight for sp in AR_SUB_PATTERNS)
|
||
assert abs(total - 1.0) < 1e-9
|
||
|
||
def test_individual_weights_match_design(self) -> None:
|
||
"""每个子模式权重与设计文档一致。"""
|
||
expected = {
|
||
"premature_evidence_anchoring": 0.20,
|
||
"temporal_reasoning_failure": 0.20,
|
||
"semantic_rigidity": 0.15,
|
||
"fine_grained_visual_action": 0.15,
|
||
"cross_segment_entity_tracking": 0.15,
|
||
"evidence_gap_confabulation": 0.15,
|
||
}
|
||
for sp in AR_SUB_PATTERNS:
|
||
assert abs(sp.weight - expected[sp.name]) < 1e-9, f"{sp.name} 权重不匹配"
|
||
|
||
def test_all_six_patterns_defined(self) -> None:
|
||
"""定义了 6 个子模式。"""
|
||
assert len(AR_SUB_PATTERNS) == 6
|
||
expected_names = {
|
||
"premature_evidence_anchoring",
|
||
"temporal_reasoning_failure",
|
||
"semantic_rigidity",
|
||
"fine_grained_visual_action",
|
||
"cross_segment_entity_tracking",
|
||
"evidence_gap_confabulation",
|
||
}
|
||
actual_names = {sp.name for sp in AR_SUB_PATTERNS}
|
||
assert actual_names == expected_names
|
||
```
|
||
|
||
- [ ] **Step 2: 运行测试验证失败**
|
||
|
||
```bash
|
||
conda run -n Video-Tree-TRM pytest tests/unit/test_strategy_action_recognition.py -v
|
||
```
|
||
|
||
预期：ImportError — `app.question_gen.strategy_action_recognition` 不存在。
|
||
|
||
- [ ] **Step 3: 实现 strategy_action_recognition.py**
|
||
|
||
```python
|
||
# app/question_gen/strategy_action_recognition.py
|
||
"""Action Recognition 特化出题策略 — 靶向 6 种失败子模式。
|
||
|
||
来源：22 道 VME AR 错题双分类器仲裁分析。
|
||
每个 SubPattern 定义独立的 instruction、采样覆盖、干扰项构造规则，
|
||
由 pipeline 在出题时注入到 prompt 的 Special Focus 区段。
|
||
|
||
设计文档: research-wiki/designs/2026-07-14-task-type-strategy-design.md §4
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
from typing import TYPE_CHECKING, Any
|
||
|
||
from app.question_gen.families import SamplingConstraint
|
||
from app.question_gen.strategy import SubPattern
|
||
|
||
if TYPE_CHECKING:
|
||
import random
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 默认采样约束（设计文档 §4 定义）
|
||
# ---------------------------------------------------------------------------
|
||
|
||
_AR_DEFAULT_CONSTRAINT = SamplingConstraint(
|
||
min_subtitles=3,
|
||
min_l3_nodes=5,
|
||
require_frames=True,
|
||
cross_l2_span=True,
|
||
)
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 6 个 SubPattern 定义
|
||
# ---------------------------------------------------------------------------
|
||
|
||
_SP_PREMATURE_EVIDENCE_ANCHORING = SubPattern(
|
||
name="premature_evidence_anchoring",
|
||
weight=0.20,
|
||
sampling_level_override=1,
|
||
constraint_override=None,
|
||
instruction=(
|
||
"Generate a question where the correct answer requires verifying evidence across "
|
||
"ALL options before committing — not just finding one matching piece of evidence. "
|
||
"The video must contain a plausible-looking but incorrect match that appears early "
|
||
"or prominently, while the true answer is confirmed only by cross-referencing "
|
||
"multiple segments. The question should punish an agent that stops searching after "
|
||
"the first evidence match."
|
||
),
|
||
positive_examples=[
|
||
{
|
||
"question": "Which of the following tasks did the heroine not complete while her baby was sleeping?",
|
||
"answer": "D. Doing laundry",
|
||
"why": "Requires checking all 4 options against all segments; stopping at first match misses the negation.",
|
||
},
|
||
{
|
||
"question": "Which acrobatic skill is absent from this video?",
|
||
"answer": "D. Somersault",
|
||
"why": "Must verify every option against every segment to confirm absence.",
|
||
},
|
||
],
|
||
negative_examples=[
|
||
{
|
||
"question": "What color jersey does the player wear?",
|
||
"why": "Single-frame answerable, no need to verify across segments.",
|
||
},
|
||
],
|
||
distractor_rules=(
|
||
"Place the most visually salient or early-appearing action as a distractor (not the answer). "
|
||
"Make one distractor partially correct (happens in a different segment or by a different person). "
|
||
"The correct answer should require exhaustive verification across segments."
|
||
),
|
||
)
|
||
|
||
_SP_TEMPORAL_REASONING_FAILURE = SubPattern(
|
||
name="temporal_reasoning_failure",
|
||
weight=0.20,
|
||
sampling_level_override=1,
|
||
constraint_override=None,
|
||
instruction=(
|
||
"Generate a question that requires precise temporal ordering or locating the Nth "
|
||
"occurrence of an event. The video must contain the same or similar action happening "
|
||
"multiple times, and the question must specify a temporal anchor (e.g., 'after X happens', "
|
||
"'the second time', 'at the beginning'). The correct answer depends on getting the "
|
||
"sequence order right."
|
||
),
|
||
positive_examples=[
|
||
{
|
||
"question": "In the video after feeding the ducks, what did the male protagonist do after riding his bike?",
|
||
"answer": "A. Went jogging in the park",
|
||
"why": "Requires precise temporal chain: feeding → biking → next action.",
|
||
},
|
||
{
|
||
"question": "What happened to the team on the counterattack after Sabonis' first steal?",
|
||
"answer": "D. They scored a three-pointer",
|
||
"why": "Must locate the FIRST steal (not second) and track what follows.",
|
||
},
|
||
],
|
||
negative_examples=[
|
||
{
|
||
"question": "What does the person do in the video?",
|
||
"why": "No temporal anchor, any observation suffices.",
|
||
},
|
||
],
|
||
distractor_rules=(
|
||
"Include actions that genuinely occur in the video but at a different time point. "
|
||
"One distractor should be what happens before the anchored moment. "
|
||
"Another should be what happens after the Nth+1 occurrence (off-by-one trap)."
|
||
),
|
||
)
|
||
|
||
_SP_SEMANTIC_RIGIDITY = SubPattern(
|
||
name="semantic_rigidity",
|
||
weight=0.15,
|
||
sampling_level_override=None,
|
||
constraint_override=None,
|
||
instruction=(
|
||
"Generate a question where the correct answer option uses a synonym, paraphrase, or "
|
||
"semantic equivalent of what is shown in the video — NOT the exact words from subtitles. "
|
||
"The agent must recognize that a rephrased description matches the observed action. "
|
||
"Include a distractor that uses near-verbatim subtitle wording but describes a "
|
||
"different or incorrect action."
|
||
),
|
||
positive_examples=[
|
||
{
|
||
"question": "What are the magic tricks about?",
|
||
"answer": "B. Sleight of hand with everyday objects",
|
||
"why": "Video shows card and coin manipulation; correct answer paraphrases rather than quoting subtitles.",
|
||
},
|
||
],
|
||
negative_examples=[
|
||
{
|
||
"question": "According to the narrator, what is the main topic?",
|
||
"why": "Invites verbatim subtitle matching, not semantic understanding.",
|
||
},
|
||
],
|
||
distractor_rules=(
|
||
"One distractor must reuse exact subtitle phrasing but apply it to the wrong action/context. "
|
||
"Another distractor should use a semantically related but distinct action verb "
|
||
"(e.g., 'cutting' vs 'slicing' vs 'chopping' when only one is correct). "
|
||
"The correct answer must be a valid semantic equivalent, not a stretch."
|
||
),
|
||
)
|
||
|
||
_SP_FINE_GRAINED_VISUAL_ACTION = SubPattern(
|
||
name="fine_grained_visual_action",
|
||
weight=0.15,
|
||
sampling_level_override=None,
|
||
constraint_override=None,
|
||
instruction=(
|
||
"Generate a question that distinguishes between visually similar actions — "
|
||
"the MANNER of how something is done, not just WHAT is done. "
|
||
"The video must show a specific technique, method, or style of performing an action, "
|
||
"and the question must test whether the agent can differentiate it from similar alternatives. "
|
||
"Frames are essential — the answer cannot come from subtitles alone."
|
||
),
|
||
positive_examples=[
|
||
{
|
||
"question": "How does the chef prepare the garlic in this recipe?",
|
||
"answer": "C. Crushes it with the flat side of a knife",
|
||
"why": "All options are valid garlic preparations; only visual observation distinguishes.",
|
||
},
|
||
{
|
||
"question": "What does the man with a laughing face do at the beginning of the video?",
|
||
"answer": "C. Clasps his hands together and bows",
|
||
"why": "Specific gesture detail requires frame-level observation.",
|
||
},
|
||
],
|
||
negative_examples=[
|
||
{
|
||
"question": "Does the person cook in the video?",
|
||
"why": "Binary yes/no, no manner distinction needed.",
|
||
},
|
||
],
|
||
distractor_rules=(
|
||
"All four options must describe the same general category of action "
|
||
"(e.g., all are ways of cutting, all are types of greetings). "
|
||
"Distractors must be visually plausible alternatives that could occur in the same context. "
|
||
"The distinction must be observable only from frames, not from subtitles."
|
||
),
|
||
)
|
||
|
||
_SP_CROSS_SEGMENT_ENTITY_TRACKING = SubPattern(
|
||
name="cross_segment_entity_tracking",
|
||
weight=0.15,
|
||
sampling_level_override=1,
|
||
constraint_override=None,
|
||
instruction=(
|
||
"Generate a question that requires tracking a specific entity (person, object, or group) "
|
||
"across multiple video segments and merging observations. The correct answer depends on "
|
||
"information from at least two separate segments — a single segment gives only a partial "
|
||
"or misleading picture. The entity must appear in different contexts or states across segments."
|
||
),
|
||
positive_examples=[
|
||
{
|
||
"question": "In the video, what happened in the car when the heroine came home from shopping?",
|
||
"answer": "D. The car wouldn't start and she had to call for help",
|
||
"why": "Must track heroine across shopping segment → car segment → resolution.",
|
||
},
|
||
],
|
||
negative_examples=[
|
||
{
|
||
"question": "What is the person wearing?",
|
||
"why": "Single-segment observation, no cross-segment tracking needed.",
|
||
},
|
||
],
|
||
distractor_rules=(
|
||
"One distractor should be correct for the entity in a DIFFERENT segment (right entity, wrong time). "
|
||
"Another distractor should be correct for a DIFFERENT entity in the same segment (right time, wrong entity). "
|
||
"The correct answer must require merging observations from multiple segments."
|
||
),
|
||
)
|
||
|
||
_SP_EVIDENCE_GAP_CONFABULATION = SubPattern(
|
||
name="evidence_gap_confabulation",
|
||
weight=0.15,
|
||
sampling_level_override=None,
|
||
constraint_override=None,
|
||
instruction=(
|
||
"Generate a question about an action where the video evidence is INCOMPLETE — "
|
||
"the full causal chain is not directly shown. The correct answer is the one that "
|
||
"stays faithful to what IS observable, while distractors fill in the gap with "
|
||
"plausible but unsupported causal narratives. The agent must resist inventing "
|
||
"explanations for unobserved transitions."
|
||
),
|
||
positive_examples=[
|
||
{
|
||
"question": "How were the Sawtooth ranges formed?",
|
||
"answer": "D. The video describes geological uplift but does not show the formation process",
|
||
"why": "Video describes result but not process; agent must not confabulate mechanism.",
|
||
},
|
||
],
|
||
negative_examples=[
|
||
{
|
||
"question": "Why did the person leave the room?",
|
||
"why": "If the reason is explicitly stated in dialogue, no evidence gap exists.",
|
||
},
|
||
],
|
||
distractor_rules=(
|
||
"Distractors must be plausible causal narratives that COULD explain the outcome but "
|
||
"are NOT supported by the video evidence. Each distractor should fill the evidence gap "
|
||
"with a different invented mechanism. The correct answer must be the one that "
|
||
"either: (a) states only what is directly observable, or (b) acknowledges the limitation."
|
||
),
|
||
)
|
||
|
||
# 导出 tuple（不可变，按名称排序供测试）
|
||
AR_SUB_PATTERNS: tuple[SubPattern, ...] = (
|
||
_SP_PREMATURE_EVIDENCE_ANCHORING,
|
||
_SP_TEMPORAL_REASONING_FAILURE,
|
||
_SP_SEMANTIC_RIGIDITY,
|
||
_SP_FINE_GRAINED_VISUAL_ACTION,
|
||
_SP_CROSS_SEGMENT_ENTITY_TRACKING,
|
||
_SP_EVIDENCE_GAP_CONFABULATION,
|
||
)
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Strategy 实现
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
class ActionRecognitionStrategy:
|
||
"""Action Recognition 特化出题策略。
|
||
|
||
自包含 —— 不依赖 QuestionFamilySpec，直接定义采样约束、prompt、gate 合约。
|
||
6 个 SubPattern 按权重随机选择，各自可覆盖默认采样层级。
|
||
|
||
设计文档: research-wiki/designs/2026-07-14-task-type-strategy-design.md §4
|
||
"""
|
||
|
||
@property
|
||
def task_type(self) -> str:
|
||
"""固定为 Action Recognition。"""
|
||
return "Action Recognition"
|
||
|
||
@property
|
||
def sampling_level(self) -> int:
|
||
"""默认 L2 事件级（从 L3 提升，有意变更）。"""
|
||
return 2
|
||
|
||
@property
|
||
def sampling_constraint(self) -> SamplingConstraint:
|
||
"""加强约束：要求字幕、帧、跨段。"""
|
||
return _AR_DEFAULT_CONSTRAINT
|
||
|
||
@property
|
||
def prompt_template(self) -> str:
|
||
"""AR 专属 prompt 模板。"""
|
||
return "action_recognition.md"
|
||
|
||
@property
|
||
def strategy_name(self) -> str:
|
||
"""store 的 family 字段。"""
|
||
return "ACTION_RECOGNITION"
|
||
|
||
@property
|
||
def skill_target(self) -> str:
|
||
"""M1_AR — 继承 RETRIEVAL 的 M1 + AR 后缀区分。"""
|
||
return "M1_AR"
|
||
|
||
@property
|
||
def leak_probe_template(self) -> str:
|
||
"""复用 RETRIEVAL 的泄漏检测模板。"""
|
||
return "gate_leak_retrieval.md"
|
||
|
||
def select_sub_pattern(self, rng: random.Random) -> SubPattern:
|
||
"""按权重随机选择一个子模式。
|
||
|
||
参数:
|
||
rng: 可控随机数生成器。
|
||
|
||
返回:
|
||
选中的 SubPattern 实例。
|
||
"""
|
||
names = [sp.name for sp in AR_SUB_PATTERNS]
|
||
weights = [sp.weight for sp in AR_SUB_PATTERNS]
|
||
chosen_name = rng.choices(names, weights=weights, k=1)[0]
|
||
return next(sp for sp in AR_SUB_PATTERNS if sp.name == chosen_name)
|
||
|
||
def build_prompt_context(self, material: Any, sub_pattern: SubPattern | None) -> dict:
|
||
"""返回 AR 的 prompt 上下文。
|
||
|
||
参数:
|
||
material: 采样素材上下文。
|
||
sub_pattern: 选中的子模式（AR 策略下始终非 None）。
|
||
|
||
返回:
|
||
上下文字典。
|
||
"""
|
||
return {
|
||
"family_name": "ACTION_RECOGNITION",
|
||
"prompt_template": "action_recognition.md",
|
||
"sub_pattern": sub_pattern.name if sub_pattern else None,
|
||
}
|
||
|
||
def extra_gates(self, candidate: Any) -> list:
|
||
"""当前版本无额外 gate，预留接口。
|
||
|
||
参数:
|
||
candidate: 候选题目。
|
||
|
||
返回:
|
||
空列表。
|
||
"""
|
||
return []
|
||
```
|
||
|
||
- [ ] **Step 4: 运行测试验证通过**
|
||
|
||
```bash
|
||
conda run -n Video-Tree-TRM pytest tests/unit/test_strategy_action_recognition.py -v
|
||
```
|
||
|
||
预期：全部 PASS
|
||
|
||
- [ ] **Step 5: 格式和 lint 检查**
|
||
|
||
```bash
|
||
conda run -n Video-Tree-TRM ruff format app/question_gen/strategy_action_recognition.py tests/unit/test_strategy_action_recognition.py
|
||
conda run -n Video-Tree-TRM ruff check app/question_gen/strategy_action_recognition.py tests/unit/test_strategy_action_recognition.py
|
||
conda run -n Video-Tree-TRM radon cc app/question_gen/strategy_action_recognition.py -n C -s
|
||
```
|
||
|
||
预期：零 error，无 C 级以上复杂度。
|
||
|
||
- [ ] **Step 6: 提交**
|
||
|
||
```bash
|
||
git add app/question_gen/strategy_action_recognition.py tests/unit/test_strategy_action_recognition.py
|
||
git commit -m "feat(question_gen): add ActionRecognitionStrategy with 6 SubPatterns"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: 注册 AR 策略 + 移除临时绑定
|
||
|
||
**Files:**
|
||
- Modify: `app/question_gen/strategy.py:107`
|
||
- Test: `tests/unit/test_strategy.py`（追加测试）
|
||
|
||
- [ ] **Step 1: 写测试 — AR 注册后 get_strategy 返回特化策略**
|
||
|
||
在 `tests/unit/test_strategy.py` 末尾追加：
|
||
|
||
```python
|
||
class TestActionRecognitionRegistration:
|
||
"""AR 策略注册后 get_strategy 返回特化实例。"""
|
||
|
||
def test_get_strategy_returns_ar_strategy(self):
|
||
"""get_strategy('Action Recognition') 返回 ActionRecognitionStrategy。"""
|
||
from app.question_gen.strategy_action_recognition import ActionRecognitionStrategy
|
||
|
||
s = get_strategy("Action Recognition")
|
||
assert isinstance(s, ActionRecognitionStrategy)
|
||
assert s.task_type == "Action Recognition"
|
||
assert s.strategy_name == "ACTION_RECOGNITION"
|
||
|
||
def test_ar_not_base_strategy(self):
|
||
"""get_strategy('Action Recognition') 不再返回 BaseTaskTypeStrategy。"""
|
||
s = get_strategy("Action Recognition")
|
||
assert not isinstance(s, BaseTaskTypeStrategy)
|
||
|
||
def test_other_types_still_base(self):
|
||
"""其他题型仍返回 BaseTaskTypeStrategy。"""
|
||
for tt in ("Object Recognition", "Temporal Reasoning", "Spatial Reasoning"):
|
||
s = get_strategy(tt)
|
||
assert isinstance(s, BaseTaskTypeStrategy), f"{tt} 应该是 BaseTaskTypeStrategy"
|
||
```
|
||
|
||
- [ ] **Step 2: 运行测试验证失败**
|
||
|
||
```bash
|
||
conda run -n Video-Tree-TRM pytest tests/unit/test_strategy.py::TestActionRecognitionRegistration -v
|
||
```
|
||
|
||
预期：`test_get_strategy_returns_ar_strategy` FAIL（返回 BaseTaskTypeStrategy）。
|
||
|
||
- [ ] **Step 3: 在 get_strategy 中添加延迟注册**
|
||
|
||
修改 `app/question_gen/strategy.py` 中的 `get_strategy` 函数，在首次调用时触发特化策略注册（避免循环导入）：
|
||
|
||
```python
|
||
_BUILTIN_REGISTERED = False
|
||
|
||
def get_strategy(task_type: str) -> TaskTypeStrategy:
|
||
"""获取题型策略。未注册的自动创建 BaseTaskTypeStrategy。
|
||
|
||
首次调用时延迟注册内建特化策略（避免循环导入）。
|
||
|
||
参数:
|
||
task_type: 题型名。
|
||
|
||
返回:
|
||
TaskTypeStrategy 实例。
|
||
|
||
异常:
|
||
KeyError: task_type 不在消歧绑定表和注册表中。
|
||
"""
|
||
global _BUILTIN_REGISTERED # noqa: PLW0603
|
||
if not _BUILTIN_REGISTERED:
|
||
_BUILTIN_REGISTERED = True
|
||
_register_builtin_strategies()
|
||
|
||
if task_type in _STRATEGY_REGISTRY:
|
||
return _STRATEGY_REGISTRY[task_type]
|
||
return _build_default_strategy(task_type)
|
||
|
||
|
||
def _register_builtin_strategies() -> None:
|
||
"""注册内建的特化策略。由 get_strategy 首次调用时延迟执行。"""
|
||
from app.question_gen.strategy_action_recognition import ActionRecognitionStrategy
|
||
register_strategy(ActionRecognitionStrategy())
|
||
```
|
||
|
||
同时更新 `_TASK_TYPE_TO_FAMILY` 中 AR 的注释：
|
||
|
||
```python
|
||
# 修改前：
|
||
"Action Recognition": VISUAL_FAMILY, # Plan A 临时绑定；Plan B 替换为特化策略
|
||
|
||
# 修改后：
|
||
"Action Recognition": VISUAL_FAMILY, # fallback — 注册表中已被 ActionRecognitionStrategy 替换
|
||
```
|
||
|
||
- [ ] **Step 4: 运行测试验证通过**
|
||
|
||
```bash
|
||
conda run -n Video-Tree-TRM pytest tests/unit/test_strategy.py tests/unit/test_strategy_action_recognition.py -v
|
||
```
|
||
|
||
预期：全部 PASS。
|
||
|
||
- [ ] **Step 5: 提交**
|
||
|
||
```bash
|
||
git add app/question_gen/strategy.py tests/unit/test_strategy.py
|
||
git commit -m "feat(question_gen): register ActionRecognitionStrategy, replace temp VISUAL binding"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: 全量回归测试 + lint
|
||
|
||
- [ ] **Step 1: lint**
|
||
|
||
```bash
|
||
conda run -n Video-Tree-TRM ruff format app/question_gen/ tests/ && conda run -n Video-Tree-TRM ruff check app/question_gen/ --fix
|
||
```
|
||
|
||
- [ ] **Step 2: 全量测试**
|
||
|
||
```bash
|
||
conda run -n Video-Tree-TRM pytest tests/unit/ tests/integration/ -v --tb=short
|
||
```
|
||
|
||
预期：1173+ 全部 PASS
|
||
|
||
- [ ] **Step 3: 提交（如有 lint 修复）**
|
||
|
||
```bash
|
||
git add -A && git commit -m "chore: lint and format Plan B changes"
|
||
```
|
||
|
||
---
|
||
|
||
## 行为保真检查清单
|
||
|
||
| # | 行为 | 状态 | 说明 |
|
||
|---|------|------|------|
|
||
| 1 | 11 个非 AR 题型行为不变 | 保留 | get_strategy 对非 AR 类型仍返回 BaseTaskTypeStrategy |
|
||
| 2 | AR 从 VISUAL_FAMILY(L3) 变为 AR 特化(L2) | **有意变更** | 设计文档 §4 明确标注 |
|
||
| 3 | AR 采样约束加强 | **有意变更** | 设计文档 §4: min_subtitles=3, cross_l2_span=True |
|
||
| 4 | AR prompt 从 visual.md 变为 action_recognition.md | **有意变更** | 专属 prompt 靶向动作识别 |
|
||
| 5 | AR strategy_name 从 "VISUAL" 变为 "ACTION_RECOGNITION" | **有意变更** | store 中新记录可区分 |
|
||
| 6 | AR skill_target 从 "M4" 变为 "M1_AR" | **有意变更** | 设计文档 §4 |
|
||
| 7 | select_sub_pattern 返回 SubPattern（非 None） | **有意变更** | AR 始终有 sub_pattern |
|
||
| 8 | sub_pattern.instruction 注入到 prompt | 保留 | Plan A 已接线（generator_v2.py:176-177） |
|
||
| 9 | sub_pattern.sampling_level_override 覆盖 level | 保留 | Plan A 已接线（pipeline_v2.py:359-360） |
|
||
| 10 | sub_pattern.name 写入 store | 保留 | Plan A 已接线（pipeline_v2.py:425） |
|
||
| 11 | pipeline 重出循环/后处理/四门 gate 不变 | 保留 | Plan B 不改 pipeline |
|
||
| 12 | 断点续跑 | 保留 | pipeline 的 progress 机制不变 |
|
||
|
||
## 核心算法保真校验
|
||
|
||
本计划不涉及核心算法迁移，保真校验不适用。
|
||
|
||
## 非功能性需求
|
||
|
||
| 维度 | 设计 |
|
||
|------|------|
|
||
| 持久化 | 不变 — on_accept 逐题回调。sub_pattern 名已在 Plan A 中接入 store |
|
||
| 幂等性 | 不变 — strategy 查找确定性，sub_pattern 选择由 rng 控制 |
|
||
| 断点续跑 | 不变 — progress 机制在 pipeline 层，strategy 无状态 |
|
||
| 原子性 | 不变 — 逐题落库 |
|