diff --git a/research-wiki/plans/2026-07-07-search-module.md b/research-wiki/plans/2026-07-07-search-module.md index f97e0c5..ddf4cac 100644 --- a/research-wiki/plans/2026-07-07-search-module.md +++ b/research-wiki/plans/2026-07-07-search-module.md @@ -7,3 +7,412 @@ date: 2026-07-07 # app/search/ 搜索 Agent 装配层实现计划 +> **For agentic workers:** REQUIRED SUB-SKILL: Use subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** 完整迁移 TRM4 搜索 Agent 装配层到 TRM5 app/search/,包含 prompt 管理、skill 注册、工具分发、LLM 两轮摘要、VLM 视觉观察和 OCR 支持。 + +**Architecture:** 方案 A 平铺模块(6 个文件 + 1 个 adapter + 1 个 Protocol)。所有 LLM/VLM 调用通过 Protocol 注入,environment 保持纯数据层。详见 `research-wiki/designs/2026-07-07-search-module-design.md`。 + +**Tech Stack:** Python 3.11, asyncio, pluggy, loguru, requests, numpy, pytest + +**核心算法保真声明:** 本计划不涉及 ARCHITECTURE.md §6 核心算法保真清单中的 13 项关键算法迁移。 + +--- + +## 文件结构总览 + +| 操作 | 文件 | 职责 | +|------|------|------| +| Create | `app/search/__init__.py` | 公开 API 重导出 | +| Create | `app/search/skills.py` | SkillRegistry + discover_skills | +| Create | `app/search/summarizer.py` | 两轮 LLM 摘要 + anchor 锚模式 | +| Create | `app/search/vision.py` | observe_frame(VLM 两轮 + OCR) | +| Create | `app/search/tools.py` | SearchToolDispatcher + 工具描述 | +| Create | `app/search/prompt.py` | PromptManager | +| Create | `adapters/ocr.py` | MonkeyOCRClient | +| Modify | `app/ports.py` | 新增 OCRProvider Protocol | +| Modify | `app/tree/environment.py` | 新增 get_node_text / get_children_info | +| Copy | `store/prompts/*.md` × 9 | 从 TRM4 v2 字节级复制 | + +--- + +### Task 1: 复制 Prompt 种子文件 + +**Files:** +- Copy: `store/prompts/` (9 files from TRM4 `store/prompts/v2/`) + +- [ ] **Step 1: 复制全部 prompt 文件** + +```bash +mkdir -p store/prompts +for f in system.md observe_frame_extract.md observe_frame_verify.md view_node_extract.md view_node_verify.md view_node_children_extract.md view_node_children_verify.md search_similar_extract.md search_similar_verify.md; do + cp /home/iomgaa/Projects/Video-Tree-TRM4/store/prompts/v2/$f store/prompts/$f +done +``` + +- [ ] **Step 2: 字节级校验** + +```bash +for f in system.md observe_frame_extract.md observe_frame_verify.md view_node_extract.md view_node_verify.md view_node_children_extract.md view_node_children_verify.md search_similar_extract.md search_similar_verify.md; do + diff /home/iomgaa/Projects/Video-Tree-TRM4/store/prompts/v2/$f store/prompts/$f +done +``` + +Expected: 无输出(全部一致) + +- [ ] **Step 3: Commit** + +```bash +git add store/prompts/ && git commit -m "chore: 复制 TRM4 v2 prompt 种子文件(9 个,字节级一致)" +``` + +--- + +### Task 2: OCRProvider Protocol + MonkeyOCRClient + +**Files:** +- Modify: `app/ports.py` — 新增 OCRProvider +- Create: `adapters/ocr.py` — MonkeyOCRClient +- Create: `tests/unit/test_ocr_adapter.py` + +- [ ] **Step 1: 写 OCR 测试** + +`tests/unit/test_ocr_adapter.py`。测试 Protocol 合规性、单帧转录、失败降级、健康检查、轮询、行去重过滤。使用 `responses` 库 mock HTTP。 + +- [ ] **Step 2: 运行测试确认失败** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_ocr_adapter.py -v +``` + +Expected: ImportError + +- [ ] **Step 3: 实现 OCRProvider Protocol** + +`app/ports.py` 新增 `OCRProvider(Protocol)` + `async def transcribe_frames(self, frame_paths: list[Path]) -> str`。 + +- [ ] **Step 4: 实现 MonkeyOCRClient** + +`adapters/ocr.py` 从 TRM4 `core/tree/ocr.py` 迁移。公开方法改 async(`asyncio.to_thread` 包装同步 HTTP)。构造函数 `ValueError` 替代 `assert`。逻辑与 TRM4 完全一致:多端点轮询、线程安全 Session、单帧失败降级、行去重过滤。 + +- [ ] **Step 5: 运行测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_ocr_adapter.py -v +``` + +Expected: 全部 PASS + +- [ ] **Step 6: Commit** + +```bash +git add app/ports.py adapters/ocr.py tests/unit/test_ocr_adapter.py +git commit -m "feat(adapters): OCRProvider Protocol + MonkeyOCRClient 异步实现" +``` + +--- + +### Task 3: TreeEnvironment 扩展 + +**Files:** +- Modify: `app/tree/environment.py` — 新增 get_node_text + get_children_info +- Modify: `tests/unit/test_tree_environment.py` + +- [ ] **Step 1: 写测试** + +追加 `TestGetNodeText`(正常/anchor 模式/不存在节点)和 `TestGetChildrenInfo`(L1 有子节点/L3 空/不存在节点)到现有测试文件。 + +- [ ] **Step 2: 运行测试确认失败** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_tree_environment.py::TestGetNodeText -v +``` + +Expected: AttributeError + +- [ ] **Step 3: 实现** + +`get_node_text(node_id, *, anchor=False) -> tuple[str, dict[str, str] | None]`:复用已有 `_node_full_text` / `_node_anchored_text`,anchor 模式解析行号构建 anchor_map。 + +`get_children_info(node_id) -> list[dict[str, Any]]`:复用 `_get_children` + `_node_description` + `_format_time_range`。 + +- [ ] **Step 4: 运行测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_tree_environment.py -v +``` + +Expected: 全部 PASS + +- [ ] **Step 5: Commit** + +```bash +git add app/tree/environment.py tests/unit/test_tree_environment.py +git commit -m "feat(tree): TreeEnvironment.get_node_text + get_children_info 结构化查询" +``` + +--- + +### Task 4: app/search/skills.py + +**Files:** +- Create: `app/search/skills.py` +- Create: `tests/unit/test_search_skills.py` + +- [ ] **Step 1: 写测试** + +测试 `parse_frontmatter`(正常/缺结束符/无 frontmatter)、`strip_frontmatter`、`SkillRegistry.read`(正常/未注册 KeyError)、`discover_skills`(always/task_type/catalog 分类 + 空目录)。使用 `tmp_path` 创建临时 .md 文件。 + +- [ ] **Step 2: 运行测试确认失败** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_skills.py -v +``` + +- [ ] **Step 3: 实现** + +从 TRM4 `core/search/skills.py` 保真迁移。逻辑完全一致。 + +- [ ] **Step 4: 运行测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_skills.py -v +``` + +- [ ] **Step 5: Commit** + +```bash +git add app/search/skills.py tests/unit/test_search_skills.py +git commit -m "feat(search): SkillRegistry + discover_skills — skill 扫描与注册" +``` + +--- + +### Task 5: app/search/summarizer.py + +**Files:** +- Create: `app/search/summarizer.py` +- Create: `tests/unit/test_search_summarizer.py` + +- [ ] **Step 1: 写 anchor 工具测试** + +测试 `check_anchors`(合法锚保留/非法锚删除/范围展开/声明句不计数)和 `assemble_anchored_output`(ids/ids_expand/expand_only 三种模式 + 封顶逻辑)。纯函数,无需 mock。 + +- [ ] **Step 2: 运行测试确认失败** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_summarizer.py -v +``` + +- [ ] **Step 3: 实现 anchor 工具** + +从 TRM4 `core/tree/summarizer.py` 保真迁移:`_expand_anchor_ids`, `check_anchors`, `_cited_anchor_ids`, `assemble_anchored_output` + 全部正则常量。逻辑完全一致。 + +- [ ] **Step 4: 运行 anchor 测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_summarizer.py -v +``` + +- [ ] **Step 5: 写 summarize_* 测试** + +测试 `summarize_node`(两轮正常/提取失败/验证失败降级/anchor 模式)、`summarize_children`(正常/失败回退原始列表)、`summarize_nodes_batch`(并发多节点)。使用 FakeLLMProvider mock。 + +- [ ] **Step 6: 实现 summarize_node / summarize_children / summarize_nodes_batch** + +从 TRM4 迁移。有意变更:同步→async;`_call_llm` → `await llm.chat()`(返回 `response.content`);`ThreadPoolExecutor` → `asyncio.gather`;透传 `session_id` / `parent_call_id`。 + +- [ ] **Step 7: 运行全部 summarizer 测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_summarizer.py -v +``` + +- [ ] **Step 8: Commit** + +```bash +git add app/search/summarizer.py tests/unit/test_search_summarizer.py +git commit -m "feat(search): summarizer — 两轮 LLM 摘要 + anchor 锚模式" +``` + +--- + +### Task 6: app/search/vision.py + +**Files:** +- Create: `app/search/vision.py` +- Create: `tests/unit/test_search_vision.py` + +- [ ] **Step 1: 写测试** + +测试 `observe_frame`:两轮正常、verify=False 仅提取、OCR 注入、OCR 失败降级、OCR 为 None、VLM 提取失败、VLM 验证失败降级、帧文件不存在、stats 键完整性。使用 FakeVLMProvider + FakeOCRProvider mock。 + +- [ ] **Step 2: 运行测试确认失败** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_vision.py -v +``` + +- [ ] **Step 3: 实现** + +从 TRM4 `core/tree/vision.py` 迁移。有意变更:`await vlm.chat_with_images(messages, images)` 替代手动 base64 + 同步 `_call_vl`;images 传 Path 列表;OCR `await ocr.transcribe_frames()`;透传遥测字段。输出格式与 TRM4 完全一致。 + +- [ ] **Step 4: 运行测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_vision.py -v +``` + +- [ ] **Step 5: Commit** + +```bash +git add app/search/vision.py tests/unit/test_search_vision.py +git commit -m "feat(search): vision.observe_frame — VLM 两轮 + OCR 异步实现" +``` + +--- + +### Task 7: app/search/tools.py + +**Files:** +- Create: `app/search/tools.py` +- Create: `tests/unit/test_search_tools.py` + +- [ ] **Step 1: 写测试** + +测试 `get_tool_descriptions`(含/不含 read_skill)、`SearchToolDispatcher.dispatch` 五个工具(view_node 调 env+summarizer、search_similar 调 env+summarize_batch、observe_frame 调 env+vision+subtitle 拼接、submit_answer 返回文本、read_skill 调 registry)+ 未知工具 ValueError + 节点不存在错误文本。 + +- [ ] **Step 2: 运行测试确认失败** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_tools.py -v +``` + +- [ ] **Step 3: 实现** + +`get_tool_descriptions()` 工具描述文本与 TRM4 完全一致。`SearchToolDispatcher` 类封装,构造注入全部依赖,`dispatch` 按工具名路由到 `_handle_view_node` / `_handle_search_similar` / `_handle_observe_frame` 私有方法。`ValueError` 直接抛出(未知工具),其他异常捕获返回错误文本。 + +- [ ] **Step 4: 运行测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_tools.py -v +``` + +- [ ] **Step 5: Commit** + +```bash +git add app/search/tools.py tests/unit/test_search_tools.py +git commit -m "feat(search): SearchToolDispatcher — 5 工具分发 + 摘要集成" +``` + +--- + +### Task 8: app/search/prompt.py + +**Files:** +- Create: `app/search/prompt.py` +- Create: `tests/unit/test_search_prompt.py` + +- [ ] **Step 1: 写测试** + +测试 `__init__`(加载 system.md / 不存在抛错)、`build_inference_prompt`(auto/manual/none 三种 skill_mode)、`format_user_prompt`(含/不含 task_type)、`load`(正常/不存在)。使用 `tmp_path` 写入 prompt 文件。 + +- [ ] **Step 2: 运行测试确认失败** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_prompt.py -v +``` + +- [ ] **Step 3: 实现** + +从 TRM4 `core/search/prompt.py` 迁移。有意变更:工具描述从 `app.search.tools.get_tool_descriptions` 获取;`format_user_prompt` 参数显式化(question/options/l1_node_ids/task_type)。 + +- [ ] **Step 4: 运行测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_prompt.py -v +``` + +- [ ] **Step 5: Commit** + +```bash +git add app/search/prompt.py tests/unit/test_search_prompt.py +git commit -m "feat(search): PromptManager — prompt 加载与拼装" +``` + +--- + +### Task 9: app/search/__init__.py + Lint + 全量测试 + +**Files:** +- Create: `app/search/__init__.py` + +- [ ] **Step 1: 创建 __init__.py** + +```python +"""搜索 Agent 装配层 — prompt 管理、skill 注册、工具分发、LLM 摘要、视觉观察。""" +from app.search.prompt import PromptManager +from app.search.skills import SkillRegistry, discover_skills +from app.search.tools import SearchToolDispatcher, get_tool_descriptions + +__all__ = [ + "PromptManager", + "SkillRegistry", + "SearchToolDispatcher", + "discover_skills", + "get_tool_descriptions", +] +``` + +- [ ] **Step 2: Lint** + +```bash +conda activate Video-Tree-TRM & ruff check app/search/ adapters/ocr.py --fix && ruff format app/search/ adapters/ocr.py +``` + +- [ ] **Step 3: 全量 search 测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/unit/test_search_prompt.py tests/unit/test_search_skills.py tests/unit/test_search_tools.py tests/unit/test_search_summarizer.py tests/unit/test_search_vision.py tests/unit/test_ocr_adapter.py -v +``` + +Expected: 全部 PASS + +- [ ] **Step 4: 回归测试** + +```bash +conda activate Video-Tree-TRM & pytest tests/ -v --tb=short +``` + +Expected: 全部 PASS + +- [ ] **Step 5: Commit** + +```bash +git add app/search/__init__.py && git commit -m "feat(search): __init__.py 公开 API + lint 通过" +``` + +--- + +### Task 10: 更新 ARCHITECTURE.md + +**Files:** +- Modify: `research-wiki/ARCHITECTURE.md` + +- [ ] **Step 1: 更新 §2.3 目录树中 app/search/** + +替换为实际 6 个文件。 + +- [ ] **Step 2: 更新 §3.3 ToolDispatcher 实现映射** + +`app/search/skills.py SkillRegistry` → `app/search/tools.py SearchToolDispatcher`。 + +- [ ] **Step 3: 更新 §3.2 OCRProvider 方法签名** + +`recognize(image_path)` → `transcribe_frames(frame_paths: list[Path]) -> str`。 + +- [ ] **Step 4: Commit** + +```bash +git add research-wiki/ARCHITECTURE.md && git commit -m "docs: 同步 app/search/ + OCRProvider 到 ARCHITECTURE.md" +```