Video-Tree-TRM5/research-wiki/plans/2026-07-14-grounded-question-gen-phaseA-plan.md

Grounded Question-Gen Phase A Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use subagent-driven-development to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: 把 Action Recognition 干扰项从"VLM 主观写得像"下沉到机制层——候选池 + VLM 视觉打分 selector 按视觉可信度区间选出 grounded near-miss 干扰项，消灭 Easy-Options Bias（负空间干扰项→排除法秒杀）；同时修复 gate tree 错配 bug、落实 sub_pattern 持久化（Phase B 硬前提）。

Architecture: 路径隔离靠 uses_grounded_selector 策略属性——AR=True 走 selector 两步出题（先用现有 generate_one_v2 拿到"正解"，再用新模块 distractor_selector 生成候选池 + 视觉打分选 3 个干扰项重组四选项）；11 个非 AR 题型=False 走原路径，字节级行为不变。公共层只做纯 bug 修复、数据字段透传、门控 rubric 松绑。

Tech Stack: Python 3.11、asyncio、VLMProvider（chat_with_images）、sqlite3（幂等 ALTER TABLE）、json_repair、pytest。全部命令在 conda 环境 Video-Tree-TRM 内执行。

---

前置约定（所有任务通用）

• 环境：每条 Python/pytest/ruff 命令前缀 conda run -n Video-Tree-TRM。示例：conda run -n Video-Tree-TRM pytest tests/unit/test_x.py -v。
• 路径隔离铁律：除"公共纯 bug/纯数据/门控 rubric"外，任何行为变更只能发生在 AR 路径（uses_grounded_selector=True 分支）。每个任务末尾的回归步骤必须证明 11 个非 AR 题型行为不变。
• 提交：每个 Task 末尾 commit，走 commit skill 的消息规范（英文、imperative、<type>: <desc>，禁止任何 AI 署名）。
• 设计来源：research-wiki/designs/2026-07-14-grounded-question-gen-phaseA-design.md。

---

Task 1: 修复 gate tree 错配 bug（公共，纯 bug）

_process_one_slot retry 换视频后，current_tree 已切换到新视频，但第 6 门 run_gates 仍传旧 tree，导致门控用错树验证（"无 source material"假拒绝）。

Files:

• Modify: app/question_gen/pipeline_v2.py:471（run_gates(candidate=candidate, tree=tree, ...) → tree=current_tree）

• Test: tests/unit/test_pipeline_v2_tree_fix.py（新建）

• Step 1: 写失败测试

新建 tests/unit/test_pipeline_v2_tree_fix.py：断言源码中 run_gates 调用使用 current_tree 而非 tree（AST/正则守卫测试，锁死回归）。

"""守卫 gate tree 错配 bug：run_gates 必须用 current_tree（换视频后的当前树）。"""

import ast
from pathlib import Path

_PIPELINE = Path(__file__).resolve().parents[2] / "app" / "question_gen" / "pipeline_v2.py"


def _find_run_gates_tree_arg() -> str:
    """解析 pipeline_v2.py，返回 run_gates 调用中 tree= 关键字实参的变量名。"""
    tree_src = ast.parse(_PIPELINE.read_text(encoding="utf-8"))
    for node in ast.walk(tree_src):
        if isinstance(node, ast.Call):
            func = node.func
            name = getattr(func, "id", None) or getattr(func, "attr", None)
            if name == "run_gates":
                for kw in node.keywords:
                    if kw.arg == "tree":
                        assert isinstance(kw.value, ast.Name)
                        return kw.value.id
    raise AssertionError("未找到 run_gates 的 tree= 关键字实参")


def test_run_gates_uses_current_tree():
    assert _find_run_gates_tree_arg() == "current_tree"

• Step 2: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_pipeline_v2_tree_fix.py -v Expected: FAIL（当前实参为 tree）

• Step 3: 改代码

app/question_gen/pipeline_v2.py Phase 6 的 run_gates 调用（约 471 行），把 tree=tree 改为 tree=current_tree：

                report = await run_gates(
                    candidate=candidate,
                    tree=current_tree,
                    llm=llm,
                    leak_probe_template=strategy.leak_probe_template,
                    postprocess=pp,
                    vlm=vlm,
                    session_id=session_id,
                )

• Step 4: 跑测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_pipeline_v2_tree_fix.py -v Expected: PASS

• Step 5: 提交

git add app/question_gen/pipeline_v2.py tests/unit/test_pipeline_v2_tree_fix.py
git commit -m "fix: run_gates must use current_tree after video resample"

---

Task 2: sub_pattern 字段透传 + 持久化（公共，纯数据 / Phase B 硬前提）

sub_pattern 目前只在 _process_one_slot 选出并写入 store.record_item，未随 GeneratedQuestion 传出，accepted_questions.json 也不含该字段。Phase B 按题的 sub_pattern 查 supports_flip，缺则无法工作。

Files:

• Modify: core/types.py（GeneratedQuestion 加 sub_pattern: str | None）

• Modify: app/question_gen/pipeline_v2.py（_to_generated_question 加 sub_pattern 形参；_process_one_slot 传入 sub_pattern.name）

• Modify: tools/generate_questions.py（_on_accept 与 _append_to_json 写 sub_pattern）

• Test: tests/unit/test_generated_question_sub_pattern.py（新建）

• Step 1: 写失败测试

新建 tests/unit/test_generated_question_sub_pattern.py：

"""GeneratedQuestion.sub_pattern 字段 + _to_generated_question 透传。"""

from app.question_gen.generator_v2 import CandidateQuestion
from app.question_gen.pipeline_v2 import _to_generated_question
from core.types import GeneratedQuestion


def _candidate() -> CandidateQuestion:
    return CandidateQuestion(
        question_id="v1_Action Recognition_0001",
        video_id="v1",
        task_type="Action Recognition",
        skill_target="M1_AR",
        question="厨师最终采用了哪种烹饪方式？",
        options=("A. 蒸", "B. 炒", "C. 煮", "D. 炸"),
        answer="A",
        source_nodes=("n1", "n2"),
        difficulty="hard",
    )


def test_generated_question_has_sub_pattern_default_none():
    q = GeneratedQuestion(
        question_id="q1", video_id="v1", task_type="Action Recognition",
        question="?", options=("A. x",), answer="A",
        source_nodes=("n1",), difficulty="easy",
    )
    assert q.sub_pattern is None


def test_to_generated_question_threads_sub_pattern():
    q = _to_generated_question(
        _candidate(), family="ACTION_RECOGNITION",
        sub_pattern="premature_evidence_anchoring",
    )
    assert q.sub_pattern == "premature_evidence_anchoring"


def test_to_generated_question_sub_pattern_defaults_none():
    q = _to_generated_question(_candidate(), family="RETRIEVAL")
    assert q.sub_pattern is None

• Step 2: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_generated_question_sub_pattern.py -v Expected: FAIL（GeneratedQuestion 无 sub_pattern；_to_generated_question 无该形参）

• Step 3: 改 core/types.py

GeneratedQuestion 末尾新增字段（保持 frozen dataclass，带默认值以兼容既有构造点）：

    family: str | None = field(default=None)
    skill_target: str | None = field(default=None)
    difficulty_steps: int | None = field(default=None)
    sub_pattern: str | None = field(default=None)

同步在 docstring 属性列表补一行：sub_pattern: 出题子模式标识（AR 特化策略使用，None 表示无）。

• Step 4: 改 _to_generated_question

app/question_gen/pipeline_v2.py，函数签名加 keyword-only 形参并透传：

def _to_generated_question(
    candidate: CandidateQuestion,
    *,
    family: str,
    options: tuple[str, ...] | None = None,
    answer: str | None = None,
    sub_pattern: str | None = None,
) -> GeneratedQuestion:
    """... (在 docstring 参数区补 sub_pattern 说明) ..."""
    return GeneratedQuestion(
        question_id=candidate.question_id,
        video_id=candidate.video_id,
        task_type=candidate.task_type,
        question=candidate.question,
        options=options if options is not None else candidate.options,
        answer=answer if answer is not None else candidate.answer,
        source_nodes=candidate.source_nodes,
        difficulty=candidate.difficulty,
        family=family,
        skill_target=candidate.skill_target,
        difficulty_steps=None,
        sub_pattern=sub_pattern,
    )

• Step 5: 在 _process_one_slot 接受点传入 sub_pattern

app/question_gen/pipeline_v2.py Phase 8 接受构造处（约 529 行）：

            result = _to_generated_question(
                candidate,
                family=strategy.strategy_name,
                options=pp.options,
                answer=pp.answer,
                sub_pattern=sub_pattern.name if sub_pattern else None,
            )

• Step 6: 跑单元测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_generated_question_sub_pattern.py -v Expected: PASS

• Step 7: 写共享序列化测试

_append_to_json 与 _on_accept 是两条独立写路径（后者才写 accepted_questions.json），各自维护一份 entry dict——易漏改一处而测试不红。抽共享函数 _question_to_entry(q) -> dict 供两处复用，直接测它保证两条路径都含 sub_pattern：

def test_question_to_entry_includes_sub_pattern():
    from tools.generate_questions import _question_to_entry
    q = GeneratedQuestion(
        question_id="v1_Action Recognition_0001", video_id="v1",
        task_type="Action Recognition", question="?",
        options=("A. 蒸", "B. 炒", "C. 煮", "D. 炸"), answer="A",
        source_nodes=("n1",), difficulty="hard",
        family="ACTION_RECOGNITION", skill_target="M1_AR",
        sub_pattern="temporal_reasoning_failure",
    )
    entry = _question_to_entry(q)
    assert entry["sub_pattern"] == "temporal_reasoning_failure"
    assert entry["question_id"] == "v1_Action Recognition_0001"
    assert entry["options"] == ["A. 蒸", "B. 炒", "C. 煮", "D. 炸"]


def test_append_to_json_writes_sub_pattern(tmp_path):
    from tools.generate_questions import _append_to_json
    q = GeneratedQuestion(
        question_id="v1_Action Recognition_0001", video_id="v1",
        task_type="Action Recognition", question="?",
        options=("A. 蒸", "B. 炒", "C. 煮", "D. 炸"), answer="A",
        source_nodes=("n1",), difficulty="hard",
        family="ACTION_RECOGNITION", skill_target="M1_AR",
        sub_pattern="temporal_reasoning_failure",
    )
    _append_to_json(tmp_path, q)
    import json
    data = json.loads((tmp_path / "v1.json").read_text(encoding="utf-8"))
    assert data[0]["sub_pattern"] == "temporal_reasoning_failure"

• Step 8: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_generated_question_sub_pattern.py -k "entry or append" -v Expected: FAIL（_question_to_entry 不存在）

• Step 9: 抽共享 _question_to_entry + 两处复用

tools/generate_questions.py，在 _append_to_json 之前加共享函数：

def _question_to_entry(question: GeneratedQuestion) -> dict:
    """将题目序列化为 JSON entry（_append_to_json 与 _on_accept 共用）。"""
    return {
        "question_id": question.question_id,
        "video_id": question.video_id,
        "task_type": question.task_type,
        "question": question.question,
        "options": list(question.options),
        "answer": question.answer,
        "source_nodes": list(question.source_nodes),
        "difficulty": question.difficulty,
        "family": question.family,
        "skill_target": question.skill_target,
        "sub_pattern": question.sub_pattern,
    }

_append_to_json 内 entry = {...} 整体替换为 entry = _question_to_entry(question)。_on_accept 内 existing.append({...}) 整体替换为 existing.append(_question_to_entry(q))。

注：_append_to_json 原 entry 不含 video_id 键（按 video 分文件），改用共享函数后会多出 video_id 键——无害（下游按需取键），且与 accepted_questions.json 格式统一。若下游有严格 schema 校验，保留两函数但都调用 _question_to_entry 后 entry.pop("video_id", None)；实现时确认下游读取无强约束即可直接统一。

• Step 10: 跑测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_generated_question_sub_pattern.py -v Expected: PASS（3+1 全绿）

• Step 11: 提交

git add core/types.py app/question_gen/pipeline_v2.py tools/generate_questions.py tests/unit/test_generated_question_sub_pattern.py
git commit -m "feat: thread and persist sub_pattern into accepted questions"

---

Task 3: uses_grounded_selector 策略开关（路径隔离核心）

Files:

• Modify: app/question_gen/strategy.py（Protocol 加 property；BaseTaskTypeStrategy 默认 False）

• Modify: app/question_gen/strategy_action_recognition.py（ActionRecognitionStrategy 覆盖为 True）

• Test: tests/unit/test_strategy_grounded_flag.py（新建）

• Step 1: 写失败测试

"""uses_grounded_selector 分流：仅 AR=True，其余 11 类=False。"""

from app.question_gen.strategy import get_strategy

_NON_AR = [
    "Action Reasoning", "Attribute Perception", "Counting Problem",
    "Information Synopsis", "Object Recognition", "Object Reasoning",
    "OCR Problems", "Spatial Perception", "Spatial Reasoning",
    "Temporal Perception", "Temporal Reasoning",
]


def test_action_recognition_uses_grounded_selector():
    assert get_strategy("Action Recognition").uses_grounded_selector is True


def test_non_ar_do_not_use_grounded_selector():
    for tt in _NON_AR:
        assert get_strategy(tt).uses_grounded_selector is False, tt

• Step 2: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_strategy_grounded_flag.py -v Expected: FAIL（AttributeError: 'uses_grounded_selector'）

• Step 3: 改 Protocol + Base

app/question_gen/strategy.py，在 TaskTypeStrategy Protocol 里（leak_probe_template 之后）加：

    @property
    def uses_grounded_selector(self) -> bool: ...

BaseTaskTypeStrategy 里加（返回 False）：

    @property
    def uses_grounded_selector(self) -> bool:
        """默认不启用 grounded selector（11 类题型走原路径）。"""
        return False

• Step 4: 改 ActionRecognitionStrategy

app/question_gen/strategy_action_recognition.py，在类里加：

    @property
    def uses_grounded_selector(self) -> bool:
        """AR 启用候选池 + VLM 视觉打分 selector。"""
        return True

• Step 5: 跑测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_strategy_grounded_flag.py -v Expected: PASS

• Step 6: 提交

git add app/question_gen/strategy.py app/question_gen/strategy_action_recognition.py tests/unit/test_strategy_grounded_flag.py
git commit -m "feat: add uses_grounded_selector strategy switch (AR only)"

---

Task 4: selector_scores 观测列 + update_selector_scores（公共，纯数据）

Files:

• Modify: app/question_gen/run_store.py（_DDL_ITEMS 加列 + 幂等 ALTER TABLE + 新方法）

• Modify: research-wiki/schemas/question-gen-items.md（登记 selector_scores 列 + JSON 结构）

• Test: tests/unit/test_run_store_selector_scores.py（新建）

• Step 1: 写失败测试

"""selector_scores 列幂等迁移 + update_selector_scores 写入。"""

import json
from app.question_gen.run_store import QuestionGenStore


def _store(tmp_path):
    return QuestionGenStore(tmp_path / "q.db")


def test_selector_scores_column_exists(tmp_path):
    store = _store(tmp_path)
    cols = {r[1] for r in store._conn.execute("PRAGMA table_info(question_gen_items)")}
    assert "selector_scores" in cols
    store.close()


def test_update_selector_scores_writes_json(tmp_path):
    store = _store(tmp_path)
    store.record_run_start("run1", "sha", "{}")
    store.record_item(
        item_id="it1", run_id="run1", slot_id="s1", video_id="v1",
        family="ACTION_RECOGNITION", task_type="Action Recognition",
        skill_target="M1_AR", attempt=1, question_text="?",
        sub_pattern="temporal_reasoning_failure",
    )
    payload = {"correct_score": 0.8, "chosen": [0.7, 0.6, 0.55],
               "pool_size": 24, "anneal_rounds": 0, "hard_fail": False}
    store.update_selector_scores("it1", json.dumps(payload))
    row = store._conn.execute(
        "SELECT selector_scores FROM question_gen_items WHERE item_id='it1'"
    ).fetchone()
    assert json.loads(row[0])["correct_score"] == 0.8
    store.close()


def test_update_selector_scores_unknown_item_raises(tmp_path):
    store = _store(tmp_path)
    try:
        store.update_selector_scores("missing", "{}")
        raise AssertionError("应抛 ValueError")
    except ValueError:
        pass
    store.close()

• Step 2: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_run_store_selector_scores.py -v Expected: FAIL

• Step 3: 改 DDL + 幂等迁移

app/question_gen/run_store.py，_DDL_ITEMS 在 difficulty_steps INTEGER, 后加 selector_scores TEXT,（新建库直接带列）。_init_schema 里，仿照 sub_pattern 的幂等迁移追加：

        cols = {r[1] for r in self._conn.execute("PRAGMA table_info(question_gen_items)")}
        if "sub_pattern" not in cols:
            self._conn.execute("ALTER TABLE question_gen_items ADD COLUMN sub_pattern TEXT")
            self._conn.commit()
        if "selector_scores" not in cols:
            self._conn.execute("ALTER TABLE question_gen_items ADD COLUMN selector_scores TEXT")
            self._conn.commit()

• Step 4: 加 update_selector_scores 方法

在 update_difficulty 之后加：

    def update_selector_scores(self, item_id: str, selector_scores_json: str) -> None:
        """写入 grounded selector 打分观测（JSON 字符串）。

        Parameters
        ----------
        item_id : str
            题目唯一 ID。
        selector_scores_json : str
            观测 JSON：correct_score / chosen / pool_size / anneal_rounds / hard_fail。

        Raises
        ------
        ValueError
            item_id 不存在时抛出。
        """
        cursor = self._conn.execute(
            "UPDATE question_gen_items SET selector_scores=? WHERE item_id=?",
            (selector_scores_json, item_id),
        )
        self._conn.commit()
        if cursor.rowcount == 0:
            raise ValueError(f"item_id 不存在: {item_id}")

• Step 5: 跑测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_run_store_selector_scores.py -v Expected: PASS

• Step 6: 登记 schema 文档

research-wiki/schemas/question-gen-items.md 的列清单补一行 selector_scores TEXT，并说明其 JSON 结构：{correct_score: float, chosen: float[], pool_size: int, anneal_rounds: int, delta_high_final: float, hard_fail: bool}。若文档用表格，追加一行；保持与既有 sub_pattern 条目同风格。

• Step 7: 提交

git add app/question_gen/run_store.py research-wiki/schemas/question-gen-items.md tests/unit/test_run_store_selector_scores.py
git commit -m "feat: add selector_scores observation column to question_gen_items"

---

Task 5: distractor_selector.py — 候选池 + VLM 视觉打分 + 区间选择 + 退火

模块核心。纯逻辑（区间选择）单测；VLM 调用用 mock 集成测。新增两个版本化 prompt。

Files:

• Create: app/question_gen/distractor_selector.py
• Create: store/prompts/question_gen/ar_distractor_pool.md
• Create: store/prompts/question_gen/ar_distractor_score.md
• Test: tests/unit/test_distractor_selector.py（新建）

5.1 纯逻辑：区间选择

• Step 1: 写失败测试（区间选择）

"""distractor_selector 区间选择纯逻辑。"""

from app.question_gen.distractor_selector import _select_in_interval


def test_select_three_in_interval_by_highest_score():
    # correct=0.90, 区间 = [0.90-0.35, 0.90-0.05] = [0.55, 0.85]
    cands = ["a", "b", "c", "d", "e"]
    scores = [0.84, 0.70, 0.60, 0.50, 0.88]  # e=0.88 太接近(>0.85)剔除, d=0.50 太低剔除
    chosen = _select_in_interval(0.90, cands, scores, delta_low=0.05, delta_high=0.35)
    assert chosen == ["a", "b", "c"]  # 落区间的按分数降序取 3（最难）


def test_select_returns_none_when_fewer_than_three():
    cands = ["a", "b"]
    scores = [0.80, 0.70]
    assert _select_in_interval(0.90, cands, scores, 0.05, 0.35) is None


def test_select_excludes_out_of_band():
    cands = ["hi", "lo", "ok1", "ok2", "ok3"]
    scores = [0.89, 0.10, 0.80, 0.75, 0.70]  # hi>上界, lo<下界
    chosen = _select_in_interval(0.90, cands, scores, 0.05, 0.35)
    assert chosen == ["ok1", "ok2", "ok3"]

• Step 2: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_distractor_selector.py -v Expected: FAIL（模块不存在）

• Step 3: 建模块骨架 + 区间选择纯函数

新建 app/question_gen/distractor_selector.py：

"""Grounded 干扰项 selector — 候选池 + VLM 视觉打分 + 区间选择（仅 AR 路径）。

把干扰项从"VLM 主观写得像"下沉到机制层：VLM 生成 N 个候选干扰项，再对
候选 + 正解逐一打"视觉可信度"分，按 [正解分-δ_high, 正解分-δ_low] 区间
选 3 个 grounded near-miss，从机制上消灭 Easy-Options Bias。

设计: research-wiki/designs/2026-07-14-grounded-question-gen-phaseA-design.md §3
"""

from __future__ import annotations

import json
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING

from json_repair import repair_json
from loguru import logger

if TYPE_CHECKING:
    from app.question_gen.sampler_v2 import MaterialContext
    from core.protocols import VLMProvider

_PROMPTS_DIR = Path(__file__).resolve().parent.parent.parent / "store" / "prompts" / "question_gen"


@dataclass(frozen=True)
class SelectorConfig:
    """selector 科研参数。

    属性:
        candidate_pool_size: 首轮候选干扰项数 N。
        delta_low: 干扰项视觉分与正解的最小差（上界，太近=真歧义）。
        delta_high: 干扰项视觉分与正解的最大差（下界，太低=负空间）。
        max_delta_relax: δ_high 放宽次数上限（退火）。
        delta_relax_step: 每次放宽 δ_high 的增量。
    """

    candidate_pool_size: int
    delta_low: float
    delta_high: float
    max_delta_relax: int = 2
    delta_relax_step: float = 0.1


@dataclass(frozen=True)
class SelectorOutcome:
    """selector 产出。observation 始终存在（含 hard-fail），供 run_store 落库。

    属性:
        observation: 打分观测 dict（correct_score/chosen/pool_size/anneal_rounds/hard_fail）。
        options: 重组四选项（A=正解），hard-fail 时为 None。
        answer: 正解字母（恒 "A"），hard-fail 时为 None。
    """

    observation: dict
    options: tuple[str, ...] | None = None
    answer: str | None = None

    @property
    def hard_fail(self) -> bool:
        """是否硬失败（凑不齐 3 个 grounded 干扰项）。"""
        return self.options is None


def _select_in_interval(
    correct_score: float,
    candidates: list[str],
    candidate_scores: list[float],
    delta_low: float,
    delta_high: float,
) -> list[str] | None:
    """从候选中选 3 个视觉分落 [correct-δ_high, correct-δ_low] 区间的干扰项。

    落区间者按分数降序取前 3（分数越高越接近正解=越难）。不足 3 个返回 None。

    参数:
        correct_score: 正解视觉可信度分。
        candidates: 候选干扰项文本列表。
        candidate_scores: 与 candidates 对齐的视觉分列表。
        delta_low: 最小差（上界 = correct - delta_low）。
        delta_high: 最大差（下界 = correct - delta_high）。

    返回:
        选中的 3 个候选文本（降序）或 None（不足 3 个）。
    """
    upper = correct_score - delta_low
    lower = correct_score - delta_high
    eligible = [
        (c, s)
        for c, s in zip(candidates, candidate_scores, strict=True)
        if lower <= s <= upper
    ]
    if len(eligible) < 3:
        return None
    eligible.sort(key=lambda cs: cs[1], reverse=True)
    return [c for c, _ in eligible[:3]]

• Step 4: 跑区间选择测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_distractor_selector.py -v Expected: PASS（3 个区间选择测试）

5.2 版本化 prompt

• Step 5: 建候选池 prompt

新建 store/prompts/question_gen/ar_distractor_pool.md：

You generate hard-negative distractor options for a video Action Recognition
multiple-choice question.

## Given
- The question, the correct answer, subtitle context, and video frames.

## Rules
- Produce distractors that are **grounded near-misses**: each MUST describe an
  action/entity that genuinely appears in the video, differing from the correct
  answer in exactly ONE dimension (timing, subject, manner, or object).
- NEVER invent events absent from the video ("negative space"). A distractor
  that names something not shown is a failure.
- Each distractor must be a plausible answer to the question for someone who
  only skimmed the video.
- Keep each distractor parallel in structure and length to the correct answer.

## Output
Respond with ONLY a JSON object:
```json
{"distractors": ["...", "...", "..."]}

Return exactly N distractors (N is given in the request). No option-letter prefixes, just the raw text.

- [ ] **Step 6: 建打分 prompt**

新建 `store/prompts/question_gen/ar_distractor_score.md`：

```markdown
You are a strict visual grader for a video Action Recognition question.

## Given
- The question, video frames, and a numbered list of candidate answer texts
  (the first is the true answer; the rest are distractor candidates — but you
  are NOT told which is which).

## Task
For EACH candidate, judge how visually credible it is as an answer given ONLY
the frames — i.e. how strongly the frames could be read as supporting it.
Score in [0.0, 1.0]: 1.0 = frames strongly depict this; 0.0 = frames show no
trace of it (pure negative space).

Judge visual groundedness ONLY. Do NOT reward the option for being the
"correct" answer — a good distractor is visually credible yet wrong.

## Output
Respond with ONLY a JSON object mapping 1-based index to score, same order as
input:
```json
{"scores": [0.9, 0.7, 0.6, 0.3, 0.85]}

Return exactly as many scores as candidates, in order.

### 5.3 VLM 编排 + 退火

- [ ] **Step 7: 写失败测试（编排，mock VLM）**

在 `tests/unit/test_distractor_selector.py` 追加：

```python
import pytest
from core.types import LLMResponse


class _FakeVLM:
    """按队列返回预设响应的 mock VLM。"""

    def __init__(self, responses: list[str]):
        self._responses = list(responses)
        self.calls = 0

    async def chat_with_images(self, messages, images, *, session_id=None, parent_call_id=None):
        self.calls += 1
        content = self._responses.pop(0)
        return LLMResponse(
            content=content, thinking="", model="fake", provider="fake",
            prompt_tokens=0, completion_tokens=0, latency_ms=0,
            ttft_ms=None, max_inter_token_ms=None, cache_hit=False, call_id="c",
        )


class _Material:
    subtitle_sentences = ["厨师先炒后蒸"]
    frame_paths = ["/f1.jpg", "/f2.jpg"]
    cross_l2_texts: list = []
    source_nodes = ("n1",)


@pytest.mark.asyncio
async def test_build_grounded_options_happy_path():
    from app.question_gen.distractor_selector import (
        SelectorConfig, build_grounded_options,
    )
    pool = '{"distractors": ["炒", "煮", "炸", "烤"]}'
    scores = '{"scores": [0.90, 0.80, 0.70, 0.60, 0.20]}'  # 正解0.90; 炒0.80 煮0.70 炸0.60 落区间, 烤0.20 剔除
    vlm = _FakeVLM([pool, scores])
    cfg = SelectorConfig(candidate_pool_size=4, delta_low=0.05, delta_high=0.35)
    out = await build_grounded_options(
        vlm=vlm, question="厨师最终用哪种方式？", correct_text="蒸",
        material=_Material(), config=cfg, session_id="s",
    )
    assert out.hard_fail is False
    assert out.answer == "A"
    assert out.options[0] == "A. 蒸"
    assert {o[3:] for o in out.options[1:]} == {"炒", "煮", "炸"}
    assert out.observation["hard_fail"] is False


@pytest.mark.asyncio
async def test_build_grounded_options_hard_fail_keeps_observation():
    from app.question_gen.distractor_selector import (
        SelectorConfig, build_grounded_options,
    )
    # 所有候选都在负空间（分数极低），退火后仍不足 3 个 → hard_fail。
    # VLM 只被调 2 次（首轮 pool+score）+ 1 次退火 pool + 1 次退火 score = 4 次；
    # δ_high 放宽轮次是纯重选，不调 VLM。退火 pool 打分含正解，共 4 个分数。
    pool = '{"distractors": ["x", "y", "z"]}'
    scores = '{"scores": [0.90, 0.05, 0.04, 0.03]}'
    pool2 = '{"distractors": ["p", "q", "r"]}'
    scores2 = '{"scores": [0.90, 0.05, 0.04, 0.03]}'
    vlm = _FakeVLM([pool, scores, pool2, scores2])
    cfg = SelectorConfig(candidate_pool_size=3, delta_low=0.05, delta_high=0.35)
    out = await build_grounded_options(
        vlm=vlm, question="?", correct_text="蒸",
        material=_Material(), config=cfg, session_id="s",
    )
    assert out.hard_fail is True
    assert out.options is None
    assert out.observation["hard_fail"] is True
    assert out.observation["pool_size"] == 6  # 首轮 3 + 退火追加 3

• Step 8: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_distractor_selector.py -v Expected: FAIL（build_grounded_options 未实现）

• Step 9: 实现 pool/score/编排

在 distractor_selector.py 追加。先补 prompt 加载与解析辅助，再 build_grounded_options：

def _load_prompt(name: str) -> str:
    path = _PROMPTS_DIR / name
    if not path.exists():
        msg = f"Prompt 模板不存在: {path}"
        raise FileNotFoundError(msg)
    return path.read_text(encoding="utf-8")


def _material_context_block(question: str, correct_text: str, material: MaterialContext) -> str:
    parts = [f"## Question\n{question}", f"## Correct Answer\n{correct_text}"]
    if material.subtitle_sentences:
        parts.append("## Subtitles")
        parts.extend(f"  - {s}" for s in material.subtitle_sentences)
    if getattr(material, "cross_l2_texts", None):
        parts.append("## Cross-Segment Context")
        parts.extend(f"  - {t}" for t in material.cross_l2_texts)
    return "\n".join(parts)


def _parse_json_object(raw: str) -> dict:
    content = raw.strip()
    if "```" in content:
        for part in content.split("```"):
            stripped = part.strip()
            if stripped.startswith("json"):
                stripped = stripped[4:].strip()
            if stripped.startswith("{"):
                content = stripped
                break
    data = json.loads(repair_json(content, return_objects=False))
    if not isinstance(data, dict):
        msg = f"selector 响应顶层非 JSON 对象: {type(data).__name__}"
        raise ValueError(msg)
    return data


async def _generate_pool(
    vlm: VLMProvider, question: str, correct_text: str,
    material: MaterialContext, n: int, *, session_id: str,
) -> list[str]:
    """VLM 生成 n 个候选干扰项文本。"""
    system = _load_prompt("ar_distractor_pool.md")
    user = _material_context_block(question, correct_text, material) + f"\n## N\nGenerate exactly {n} distractors."
    messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
    resp = await vlm.chat_with_images(messages, list(material.frame_paths), session_id=session_id)
    data = _parse_json_object(resp.content)
    raw = data.get("distractors", [])
    if not isinstance(raw, list):
        return []
    # 防御：去空、去重、剔除与正解字面相同者
    seen: set[str] = set()
    out: list[str] = []
    for item in raw:
        text = str(item).strip()
        if not text or text == correct_text.strip() or text in seen:
            continue
        seen.add(text)
        out.append(text)
    return out


async def _score_options(
    vlm: VLMProvider, question: str, options: list[str],
    material: MaterialContext, *, session_id: str,
) -> list[float]:
    """VLM 对 options（首个为正解）逐一打视觉可信度分 [0,1]，返回对齐分数列表。"""
    system = _load_prompt("ar_distractor_score.md")
    numbered = "\n".join(f"{i}. {opt}" for i, opt in enumerate(options, 1))
    user = f"## Question\n{question}\n\n## Candidates\n{numbered}"
    messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
    resp = await vlm.chat_with_images(messages, list(material.frame_paths), session_id=session_id)
    data = _parse_json_object(resp.content)
    scores_raw = data.get("scores", [])
    if not isinstance(scores_raw, list) or len(scores_raw) != len(options):
        msg = f"打分数量({len(scores_raw) if isinstance(scores_raw, list) else 'NA'}) != 选项数({len(options)})"
        raise ValueError(msg)
    return [max(0.0, min(1.0, float(s))) for s in scores_raw]


async def build_grounded_options(
    vlm: VLMProvider,
    question: str,
    correct_text: str,
    material: MaterialContext,
    config: SelectorConfig,
    *,
    session_id: str,
) -> SelectorOutcome:
    """生成候选池 → 视觉打分 → 区间选 3 干扰项 → 重组四选项。

    退火（凑不齐 3 个时按序）：① 追加 N 个候选使池达 2N 再打分；② 逐步放宽
    δ_high（纯重选，不再调 VLM）；③ 仍不足则 hard_fail（调用方走重出）。

    参数:
        vlm: VLM 端口。
        question: 题干。
        correct_text: 正解文本（无字母前缀）。
        material: 采样素材（提供 frame_paths / subtitles）。
        config: selector 科研参数。
        session_id: 遥测会话 ID。

    返回:
        SelectorOutcome。成功时 options=A 正解+3 grounded 干扰项；hard_fail
        时 options=None，但 observation 始终存在供落库。
    """
    candidates = await _generate_pool(
        vlm, question, correct_text, material, config.candidate_pool_size, session_id=session_id
    )
    # options[0] 恒为正解
    scored = await _score_options(vlm, question, [correct_text, *candidates], material, session_id=session_id)
    correct_score, cand_scores = scored[0], scored[1:]
    anneal_rounds = 0

    chosen = _select_in_interval(
        correct_score, candidates, cand_scores, config.delta_low, config.delta_high
    )

    # 退火 1: 追加 N 个候选使池达 2N（仅对新增候选打分，正解分保持首轮值）
    if chosen is None:
        anneal_rounds += 1
        more = await _generate_pool(
            vlm, question, correct_text, material, config.candidate_pool_size, session_id=session_id
        )
        more = [m for m in more if m not in candidates]
        if more:
            more_scores = await _score_options(
                vlm, question, [correct_text, *more], material, session_id=session_id
            )
            candidates = candidates + more
            cand_scores = cand_scores + more_scores[1:]
        chosen = _select_in_interval(
            correct_score, candidates, cand_scores, config.delta_low, config.delta_high
        )

    # 退火 2: 放宽 δ_high（下界下移，纳入更低分候选），δ_low 不动
    relax = 0
    delta_high = config.delta_high
    while chosen is None and relax < config.max_delta_relax:
        relax += 1
        anneal_rounds += 1
        delta_high = delta_high + config.delta_relax_step
        chosen = _select_in_interval(
            correct_score, candidates, cand_scores, config.delta_low, delta_high
        )

    hard_fail = chosen is None
    observation = {
        "correct_score": correct_score,
        "chosen": [
            cand_scores[candidates.index(c)] for c in (chosen or [])
        ],
        "pool_size": len(candidates),
        "anneal_rounds": anneal_rounds,
        "delta_high_final": delta_high,
        "hard_fail": hard_fail,
    }
    if hard_fail:
        logger.warning(
            "grounded selector 硬失败: correct={:.3f}, pool={}, anneal={}",
            correct_score, len(candidates), anneal_rounds,
        )
        # observation 仍返回，供 pipeline 落 selector_scores（设计 §3.3 退化观测）
        return SelectorOutcome(observation=observation)

    options = (
        f"A. {correct_text}",
        f"B. {chosen[0]}",
        f"C. {chosen[1]}",
        f"D. {chosen[2]}",
    )
    return SelectorOutcome(observation=observation, options=options, answer="A")

• Step 10: 跑全模块测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_distractor_selector.py -v Expected: PASS（区间选择 3 + 编排 2）

• Step 11: 提交

git add app/question_gen/distractor_selector.py store/prompts/question_gen/ar_distractor_pool.md store/prompts/question_gen/ar_distractor_score.md tests/unit/test_distractor_selector.py
git commit -m "feat: add grounded distractor selector with visual scoring"

---

Task 6: selector 接入 _process_one_slot + 配置参数（AR 路径）

把 selector 织入 AR 出题：generate_one_v2 拿正解 → 提取正解文本 → build_grounded_options 重组四选项 → 失败则重出。配置走 PipelineConfig + YAML。

Files:

• Modify: app/question_gen/pipeline_v2.py（PipelineConfig 加 selector 字段；load_pipeline_config 读 YAML；_process_one_slot 织入）

• Modify: config/question_gen_ar30.yaml（补 selector 参数）

• Test: tests/unit/test_pipeline_selector_wiring.py（新建）

• Step 1: 加 PipelineConfig 字段 + 加载

app/question_gen/pipeline_v2.py，PipelineConfig 末尾加（带默认，兼容既有 YAML）：

    seed: int
    output_dir: Path
    candidate_pool_size: int = 24
    selector_delta_low: float = 0.05
    selector_delta_high: float = 0.35

load_pipeline_config 的 return PipelineConfig(...) 补三行（.get 读，缺省用设计默认）：

        seed=int(section["seed"]),
        output_dir=Path(section["output_dir"]),
        candidate_pool_size=int(section.get("candidate_pool_size", 24)),
        selector_delta_low=float(section.get("selector_delta_low", 0.05)),
        selector_delta_high=float(section.get("selector_delta_high", 0.35)),

同步修 CLI seed override：tools/generate_questions.py:900 的 --seed 覆盖手工重建 PipelineConfig，只复制旧字段会把 selector 三参重置为默认。补三行：

        config = PipelineConfig(
            per_type=config.per_type,
            retry_limit=config.retry_limit,
            heavy_sample_rate=config.heavy_sample_rate,
            dedup_threshold=config.dedup_threshold,
            concurrency=config.concurrency,
            seed=args.seed,
            output_dir=config.output_dir,
            candidate_pool_size=config.candidate_pool_size,
            selector_delta_low=config.selector_delta_low,
            selector_delta_high=config.selector_delta_high,
        )

更稳健的等价写法是 dataclasses.replace(config, seed=args.seed)；若采用请在文件顶部 import dataclasses 或 from dataclasses import replace。二选一即可，实现时保持一致。

• Step 2: 写失败测试（正解文本提取 + 织入分流）

新建 tests/unit/test_pipeline_selector_wiring.py。先测纯辅助 _extract_correct_text：

"""selector 织入辅助：正解文本提取 + 分流。"""

from app.question_gen.pipeline_v2 import _extract_correct_text


def test_extract_correct_text_strips_prefix():
    options = ("A. 蒸", "B. 炒", "C. 煮", "D. 炸")
    assert _extract_correct_text(options, "C") == "煮"


def test_extract_correct_text_handles_lowercase_answer():
    options = ("A. run", "B. walk", "C. jump", "D. sit")
    assert _extract_correct_text(options, "b") == "walk"

• Step 3: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_pipeline_selector_wiring.py -v Expected: FAIL（_extract_correct_text 不存在）

• Step 4: 加 _extract_correct_text 辅助

app/question_gen/pipeline_v2.py（_to_generated_question 附近）：

def _extract_correct_text(options: tuple[str, ...], answer: str) -> str:
    """从四选项中取正解文本（去掉 "X. " 字母前缀）。

    参数:
        options: 选项元组，格式 ("A. ...", "B. ...", ...)。
        answer: 正解字母（大小写不敏感）。

    返回:
        正解选项去前缀后的文本。
    """
    idx = ord(answer.strip().upper()) - ord("A")
    if not 0 <= idx < len(options):
        msg = f"answer '{answer}' 超出选项范围 (n={len(options)})"
        raise ValueError(msg)
    opt = options[idx]
    prefix = f"{answer.strip().upper()}. "
    return opt[len(prefix):] if opt.startswith(prefix) else opt

• Step 5: 织入 selector 到 _process_one_slot

在 Phase 2（generate_one_v2 得到 candidate）与 Phase 3（record_item）之间不变；在 Phase 3 之后、Phase 4（postprocess）之前，插入 grounded 分支。用 strategy.uses_grounded_selector 分流；失败走 continue。注意：selector 成功后需用重组选项替换 candidate 的 options/answer 再进 postprocess。

            # Phase 3.5: grounded selector（仅 AR 路径）
            if strategy.uses_grounded_selector:
                from app.question_gen.distractor_selector import (
                    SelectorConfig,
                    build_grounded_options,
                )

                correct_text = _extract_correct_text(candidate.options, candidate.answer)
                selector_cfg = SelectorConfig(
                    candidate_pool_size=config.candidate_pool_size,
                    delta_low=config.selector_delta_low,
                    delta_high=config.selector_delta_high,
                )
                try:
                    outcome = await build_grounded_options(
                        vlm=vlm,
                        question=candidate.question,
                        correct_text=correct_text,
                        material=material,
                        config=selector_cfg,
                        session_id=session_id,
                    )
                except (ValueError, FileNotFoundError) as e:
                    logger.warning("slot {} selector 异常 (attempt {}): {}", slot.slot_id, attempt, e)
                    prev_reason = f"selector_error: {e}"
                    continue

                # observation 始终落库（含 hard-fail），供 EOB 退化观测与调参
                store.update_selector_scores(
                    item_id, json.dumps(outcome.observation, ensure_ascii=False)
                )

                if outcome.hard_fail:
                    prev_reason = "grounded 干扰项不足（selector 硬失败）"
                    store.mark_item_rejected(item_id, prev_reason)
                    logger.info("slot {} selector 硬失败 (attempt {})", slot.slot_id, attempt)
                    continue

                # 用 grounded 四选项替换候选（frozen → 构造新实例）
                candidate = _replace_candidate_options(candidate, outcome.options, outcome.answer)

在文件顶部 import 区补 import json（若未导入）。并加辅助：

def _replace_candidate_options(
    candidate: CandidateQuestion, options: tuple[str, ...], answer: str
) -> CandidateQuestion:
    """用 selector 重组的选项/答案替换候选（CandidateQuestion frozen）。"""
    return CandidateQuestion(
        question_id=candidate.question_id,
        video_id=candidate.video_id,
        task_type=candidate.task_type,
        skill_target=candidate.skill_target,
        question=candidate.question,
        options=options,
        answer=answer,
        source_nodes=candidate.source_nodes,
        difficulty=candidate.difficulty,
        subtitle_sentences=candidate.subtitle_sentences,
        frame_paths=candidate.frame_paths,
    )

• Step 6: 写织入集成测试（mock VLM 分流）

在 tests/unit/test_pipeline_selector_wiring.py 追加一个断言：非 AR 题型不触发 selector（源码守卫——uses_grounded_selector 分支只在 True 时进入）。用轻量单测覆盖 _replace_candidate_options：

def test_replace_candidate_options():
    from app.question_gen.generator_v2 import CandidateQuestion
    from app.question_gen.pipeline_v2 import _replace_candidate_options
    c = CandidateQuestion(
        question_id="q", video_id="v", task_type="Action Recognition",
        skill_target="M1_AR", question="?",
        options=("A. a", "B. b", "C. c", "D. d"), answer="A",
        source_nodes=("n1",), difficulty="hard",
    )
    new = _replace_candidate_options(c, ("A. 蒸", "B. 炒", "C. 煮", "D. 炸"), "A")
    assert new.options == ("A. 蒸", "B. 炒", "C. 煮", "D. 炸")
    assert new.question == "?"  # 其余字段不变
    assert new.source_nodes == ("n1",)

• Step 7: 跑测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_pipeline_selector_wiring.py -v Expected: PASS

• Step 8: 补 YAML 配置

config/question_gen_ar30.yaml 的 question_gen_v2 区段补三行（值用设计默认）：

  candidate_pool_size: 24
  selector_delta_low: 0.05
  selector_delta_high: 0.35

• Step 9: 更新 AR 集成测试 MockVLM（selector 启用后必须能应答 pool/score）

tests/integration/test_pipeline_v2.py 的 MockVLM.chat_with_images 现只区分门控（含 "verdict"）与生成。selector 启用后 AR 路径会额外发 pool 请求（system prompt 含 "distractor"）和 score 请求（含 "grader" / "scores"）。若不识别，pool 会解析成候选 JSON → _generate_pool 得空列表 → hard-fail → AR slot 全拒，破坏既有断言。改 chat_with_images 顶部按 system prompt 关键词分流：

    async def chat_with_images(self, messages, images, *, session_id=None, parent_call_id=None):
        prompt_text = str(messages)
        system_text = messages[0].get("content", "") if messages else ""
        if "verdict" in prompt_text.lower():
            return _make_llm_response(self._gate_response)
        if "distractor" in system_text.lower() and "grader" not in system_text.lower():
            # 候选池请求：返回 4 个 grounded 干扰项
            return _make_llm_response('{"distractors": ["蒸", "煮", "炸", "烤"]}')
        if "grader" in system_text.lower():
            # 打分请求：正解高分、3 个落区间、1 个负空间
            return _make_llm_response('{"scores": [0.90, 0.80, 0.70, 0.60, 0.20]}')
        idx = min(self._gen_count, len(self._responses) - 1)
        self._gen_count += 1
        return _make_llm_response(self._responses[idx])

打分响应长度需匹配"正解 + 候选数"。若某测试自定义候选池大小，须相应调整该 mock（打分列表长度 = pool 返回的干扰项数 + 1）。默认候选 JSON 4 个 → 打分 5 个，与上面一致。

• Step 10: 全量出题相关单测 + AR 集成回归

Run: conda run -n Video-Tree-TRM pytest tests/integration/test_pipeline_v2.py tests/unit/test_generate_questions.py tests/unit/test_families.py tests/unit/test_gates.py -v Expected: PASS（AR 集成经 selector 仍通过；非 AR 路径不受影响）

• Step 11: 提交

git add app/question_gen/pipeline_v2.py config/question_gen_ar30.yaml tools/generate_questions.py tests/unit/test_pipeline_selector_wiring.py tests/integration/test_pipeline_v2.py
git commit -m "feat: wire grounded selector into AR slot processing"

---

Task 7: 单维反事实约束（仅 AR，SubPattern 内容）

改 6 个 SubPattern 的 instruction + distractor_rules，硬约束"干扰项必须是视频中真实发生、仅在单一维度（时点/主体/方式/对象）与正解不同，严禁缺席事件"。纯 prompt 内容，行为由 selector 保障，此处强化生成端引导。

Files:

• Modify: app/question_gen/strategy_action_recognition.py（6 个 SubPattern 的 distractor_rules）

• Test: tests/unit/test_ar_sub_pattern_counterfactual.py（新建）

• Step 1: 写失败测试

"""AR 6 个 SubPattern 的 distractor_rules 含单维反事实约束关键词。"""

from app.question_gen.strategy_action_recognition import AR_SUB_PATTERNS

_REQUIRED = ["真实", "单一维度"]  # 每个 distractor_rules 都需强调 grounded + 单维


def test_all_sub_patterns_enforce_single_dimension_counterfactual():
    for sp in AR_SUB_PATTERNS:
        rules = sp.distractor_rules
        assert "真实" in rules, sp.name  # 干扰项须真实发生
        assert ("单一维度" in rules or "只在" in rules or "仅在" in rules), sp.name


def test_sub_pattern_count_unchanged():
    assert len(AR_SUB_PATTERNS) == 6

• Step 2: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_ar_sub_pattern_counterfactual.py -v Expected: FAIL

• Step 3: 改 6 个 SubPattern 的 distractor_rules

app/question_gen/strategy_action_recognition.py，逐一替换（按设计 §3.4 的反事实维度表）。示例——_TEMPORAL_REASONING_FAILURE（事件顺序维）：

    distractor_rules=(
        "干扰项必须是视频中真实发生的事件，仅在【事件时序/顺序】这一单一维度上与正解不同——"
        "即同一组真实事件的错误排列或错误的第 N 次定位。"
        "严禁使用视频中未出现的缺席事件作为干扰项。"
    ),

_PREMATURE_EVIDENCE_ANCHORING（时点维）：

    distractor_rules=(
        "将视频前段真实出现的局部匹配动作设为强干扰项——它真实发生，"
        "仅在【时点】这一单一维度上与正解不同（前段 vs 最终结论段）。"
        "其余干扰项亦须是视频中真实发生的动作，严禁缺席事件。"
    ),

_SEMANTIC_RIGIDITY（表述维）：

    distractor_rules=(
        "干扰项须基于视频真实内容，仅在【表述/语义】这一单一维度上做文章："
        "保留一个复用视频原始字幕字面、但在题干限定下语义为假的选项作为陷阱，"
        "其余选项描述真实动作的不同同义表述。严禁凭空编造缺席动作。"
    ),

_FINE_GRAINED_VISUAL_ACTION（方式维）：

    distractor_rules=(
        "四个选项须是【同一大类动作】的不同执行方式，全部为视频中真实可见的做法，"
        "仅在【执行方式】这一单一维度上不同（如顺/逆时针、扳手/螺丝刀）。"
        "严禁使用明显不相关或视频中未出现的动作作为干扰项。"
    ),

_CROSS_SEGMENT_ENTITY_TRACKING（主体维）：

    distractor_rules=(
        "干扰项须是视频中【另一真实实体】在相应片段真实做过的动作，"
        "仅在【动作主体】这一单一维度上与正解不同；或构造只覆盖部分片段的真实子集。"
        "严禁编造任何实体未做过的缺席动作。"
    ),

_EVIDENCE_GAP_CONFABULATION（因果完整性维）：

    distractor_rules=(
        "正解仅陈述视频中可观测的事实或诚实承认证据不足；"
        "干扰项在【因果完整性】这一单一维度上越界——补上一段视频未展示的因果链，"
        "但其前提元素仍取自视频真实内容（诱导 Agent 顺势编造），而非完全凭空的缺席事件。"
    ),

同时在每个 SubPattern 的 instruction 末尾（可选）加一句"干扰项遵循单维反事实、不得缺席"的提醒——但测试只校验 distractor_rules，此步以 distractor_rules 为准。

• Step 4: 跑测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_ar_sub_pattern_counterfactual.py -v Expected: PASS

• Step 5: 回归 AR 策略既有测试

Run: conda run -n Video-Tree-TRM pytest tests/unit/ -k "action or strategy or families" -v Expected: PASS

• Step 6: 提交

git add app/question_gen/strategy_action_recognition.py tests/unit/test_ar_sub_pattern_counterfactual.py
git commit -m "feat: enforce single-dimension counterfactual in AR distractor rules"

---

Task 8: multi_true 门 rubric 松绑（公共，12 类统一）

放行"错误选项有局部真实证据、但在题干限定（同主体/时点/方式/对象）下为假"的近似干扰项——否则 grounded 干扰项会被 multi_true 误毙。

这是本计划唯一一处授权的公共路径行为变更（用户在 brainstorming 明确答复"全局松绑"，见设计 §3.5）。它作用于全部 12 题型的 multi_true 门。无回归破坏风险：_gate_multi_true（app/question_gen/gates.py:363）加载 prompt 后调 llm.chat，tests/unit/test_gates.py 用 mock LLM 返回固定 verdict、不校验 prompt 内容，故 rubric 文案变更不会使既有非 AR 门控测试变红。

Files:

• Modify: store/prompts/question_gen/gate_multi_true.md

• Test: tests/unit/test_gate_multi_true_rubric.py（新建）

• Step 1: 写失败测试

"""multi_true rubric 已松绑为题干限定判据。"""

from pathlib import Path

_PROMPT = Path(__file__).resolve().parents[2] / "store" / "prompts" / "question_gen" / "gate_multi_true.md"


def test_rubric_uses_qualifier_constraint():
    text = _PROMPT.read_text(encoding="utf-8")
    # 新 rubric 必须提到"题干限定下同时为真才 fail"（qualifier / under the question's constraints）
    assert "under the question" in text.lower() or "qualifier" in text.lower()
    # 必须显式放行"有局部真实证据但在限定下为假"的干扰项
    assert "partial" in text.lower() or "locally" in text.lower()

• Step 2: 跑测试确认失败

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_gate_multi_true_rubric.py -v Expected: FAIL

• Step 3: 改 rubric

store/prompts/question_gen/gate_multi_true.md 的 ## Instructions 段替换为：

## Instructions

1. Read the source material and the question carefully. Note the question's
   explicit qualifiers (subject, timing, manner, object).
2. For each option, assess whether it is **fully correct under the question's
   qualifiers** — not merely whether it has some partial or local support in
   the source.
3. A good hard-negative distractor MAY have partial/local evidence in the video
   yet be FALSE under the question's constraints. Such an option is NOT a
   second correct answer — do NOT fail the question for it.
4. Verdict "fail" ONLY IF two or more options are each fully correct under the
   question's qualifiers (a genuine ambiguity).
5. Otherwise verdict "pass".

• Step 4: 跑测试确认通过

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_gate_multi_true_rubric.py -v Expected: PASS

• Step 5: 回归 gate 测试

Run: conda run -n Video-Tree-TRM pytest tests/unit/test_gates.py -v Expected: PASS（mock LLM，rubric 文案变更不影响判定断言）

• Step 6: 提交

git add store/prompts/question_gen/gate_multi_true.md tests/unit/test_gate_multi_true_rubric.py
git commit -m "feat: loosen multi_true gate to qualifier-scoped correctness"

---

Task 9: 全量回归 + lint + wiki 收口

Files:

• 无新代码；运行验证 + wiki 登记。

• Step 1: 全量测试

Run: conda run -n Video-Tree-TRM pytest tests/ -q Expected: 全绿（含既有 1200+ 用例，证明 11 非 AR 题型行为不变）。若有红，回到对应 Task 修复。

• Step 2: lint

Run: conda run -n Video-Tree-TRM ruff check app/ core/ --fix && conda run -n Video-Tree-TRM ruff format app/ core/ Expected: 无剩余错误。

• Step 3: wiki 登记 plan 实体

conda run -n Video-Tree-TRM python3 .claude/tools/research_wiki.py add_entity research-wiki/ --type plan --id grounded-question-gen-phaseA --title "Grounded Question-Gen Phase A"
conda run -n Video-Tree-TRM python3 .claude/tools/research_wiki.py add_edge research-wiki/ --from "plan:grounded-question-gen-phaseA" --to "design:grounded-question-gen-phaseA" --type implements --evidence "Phase A 实现计划"
conda run -n Video-Tree-TRM python3 .claude/tools/research_wiki.py rebuild_index research-wiki/

• Step 4: 提交

git add research-wiki/
git commit -m "docs: register Phase A plan in research wiki"

---

Self-Review 与保真校验

核心算法保真：本计划改动局限于出题（question_gen）的候选池/打分/门控 rubric/数据透传，不涉及 research-wiki/ARCHITECTURE.md §6 的 12 项核心算法（建树 4 项 + 训练 8 项）。出题四门 gate 非核心算法清单成员。保真校验不适用。

Spec 覆盖：L0 tree bug→Task1；uses_grounded_selector→Task3；候选池+VLM打分+区间+退火→Task5；单维反事实→Task7；multi_true松绑→Task8；sub_pattern持久化→Task2；selector观测列→Task4；接入+配置→Task6；回归→Task9。设计 §7 三个配置参数→Task6 Step1/8。

路径隔离：Task3 建开关，Task6 用开关分流，Task5 模块只被 AR 分支调用；Task1/2/4/8 为公共纯 bug/数据/rubric。每个改行为的 Task 都含非 AR 回归步骤。