diff --git a/research-wiki/designs/2026-07-07-search-module-design.md b/research-wiki/designs/2026-07-07-search-module-design.md new file mode 100644 index 0000000..6f87ea6 --- /dev/null +++ b/research-wiki/designs/2026-07-07-search-module-design.md @@ -0,0 +1,313 @@ +--- +type: design +node_id: design:2026-07-07-search-module-design +title: "搜索 Agent 装配层设计（app/search/）" +date: 2026-07-07 +--- + +# 搜索 Agent 装配层设计（app/search/） + +**日期** 2026-07-07 · **状态** 已批准 · **关联** TRM4 `core/search/` + `core/tree/tools.py` + `core/tree/vision.py` + +--- + +## §1 定位 + +`app/search/` 是搜索 Agent 的"装配层"——为 `core/agent/loop.py` AgentLoop 提供 **prompt 组装**、**skill 管理**、**工具定义/分发** 和 **视觉观察**。它不控制推理循环，只被 AgentLoop 调用；编排责任在 `app/harness/inference.py`。 + +### 与 TRM4 的映射 + +| TRM4 | TRM5 | 变更类型 | +|------|------|---------| +| `core/search/prompt.py` | `app/search/prompt.py` | 保真迁移 + P4 显式参数 | +| `core/search/skills.py` | `app/search/skills.py` | 保真迁移 | +| `core/tree/tools.py` | `app/search/tools.py` | 重组为 `SearchToolDispatcher` 类 | +| `core/tree/vision.py` | `app/search/vision.py` | 异步化 + Protocol 注入 | +| `core/tree/ocr.py` | `adapters/ocr.py` | 异步化 + OCRProvider Protocol | + +--- + +## §2 模块结构 + +``` +app/search/ +├── __init__.py # 公开 API 重导出 +├── prompt.py # PromptManager — prompt 加载与拼装 +├── skills.py # SkillRegistry + discover_skills — skill 扫描与注册 +├── tools.py # SearchToolDispatcher（实现 ToolDispatcher Protocol） +└── vision.py # observe_frame（VLM 两轮 + OCR 注入） + +adapters/ +└── ocr.py # MonkeyOCRClient（实现 OCRProvider Protocol） + +core/protocols.py # 新增 OCRProvider Protocol + +store/prompts/ # 初始种子（从 TRM4 v2 直接复制，不修改） +├── system.md +├── observe_frame_extract.md +└── observe_frame_verify.md +``` + +--- + +## §3 依赖方向 + +```mermaid +flowchart TB + subgraph adapters + OCR_IMPL["adapters/ocr.py\nMonkeyOCRClient"] + end + + subgraph core + PROTO["core/protocols.py\nOCRProvider Protocol"] + AGENT_PROTO["core/agent/protocols.py\nToolDispatcher Protocol"] + end + + subgraph app/search + PROMPT["prompt.py\nPromptManager"] + SKILLS["skills.py\nSkillRegistry"] + TOOLS["tools.py\nSearchToolDispatcher"] + VISION["vision.py\nobserve_frame"] + end + + subgraph app/tree + ENV["environment.py\nTreeEnvironment"] + end + + OCR_IMPL -->|实现| PROTO + TOOLS -->|实现| AGENT_PROTO + TOOLS --> ENV + TOOLS --> SKILLS + TOOLS --> VISION + VISION -->|依赖| PROTO + PROMPT --> SKILLS + PROMPT -.->|读取| STORE["store/prompts/*.md"] +``` + +依赖只向内或同层，`core/` 不认识 `app/search/`。 + +--- + +## §4 公开 API + +### 4.1 PromptManager（prompt.py） + +```python +class PromptManager: + def __init__(self, prompts_dir: Path) -> None: ... + def build_inference_prompt( + self, + skill_mode: str, + task_type: str, + always_skills_text: str, + task_skill_map: dict[str, str], + catalog_text: str, + ) -> str: ... + def format_user_prompt( + self, + question: str, + options: list[str], + l1_node_ids: list[str], + task_type: str | None = None, + ) -> str: ... + def load(self, name: str) -> str: ... +``` + +**与 TRM4 有意变更**： +- 工具描述从 `app/search/tools.py` 的 `get_tool_descriptions()` 获取（职责归属修正） +- `format_user_prompt` 参数从 `dict` 改为显式 `question` / `options` / `l1_node_ids`（P4） + +### 4.2 SkillRegistry + discover_skills（skills.py） + +```python +def parse_frontmatter(text: str) -> dict[str, str]: ... +def strip_frontmatter(text: str) -> str: ... + +class SkillRegistry: + def set_paths(self, mapping: dict[str, Path]) -> None: ... + def read(self, name: str) -> str: ... + +def discover_skills(skills_dir: Path) -> tuple[str, dict[str, str], str, SkillRegistry]: ... +``` + +与 TRM4 逻辑完全一致，无有意变更。 + +### 4.3 SearchToolDispatcher（tools.py） + +```python +def get_tool_descriptions(include_read_skill: bool = False) -> str: ... + +class SearchToolDispatcher: + """实现 core/agent/protocols.ToolDispatcher。""" + def __init__( + self, + env: TreeEnvironment, + vlm: VLMProvider, + ocr: OCRProvider | None, + prompts_dir: Path, + skills: SkillRegistry | None, + *, + embed_fn: Callable[[str | list[str]], np.ndarray], + verify_vision: bool = True, + stats_sink: Callable[[dict[str, int]], None] | None = None, + ) -> None: ... + + async def dispatch( + self, tool_name: str, args: dict[str, Any], *, context: dict[str, Any] + ) -> str: ... +``` + +| 工具 | 实现路径 | +|------|---------| +| `view_node` | → `env.view_node(node_id)` | +| `search_similar` | → `env.search_similar(query, top_k, embed_fn=...)` + 格式化 | +| `observe_frame` | → `env.resolve_frame_paths(...)` + `vision.observe_frame(...)` | +| `submit_answer` | → 返回确认文本 | +| `read_skill` | → `skills.read(name)` | +| 未知工具 | → `raise ValueError`（AgentLoop 捕获，不计步） | + +**与 TRM4 有意变更**： +- 自由函数 + 大量位置参数 → 类封装（构造时注入依赖） +- 工具描述 `get_tool_descriptions()` 移入此文件 +- `search_similar` 结果格式化由 dispatcher 负责（env 返回 `list[tuple[str, float]]`） + +### 4.4 observe_frame（vision.py） + +```python +async def observe_frame( + vlm: VLMProvider, + frame_paths: list[Path], + question: str, + prompts_dir: Path, + *, + ocr: OCRProvider | None, + stats_sink: Callable[[dict[str, int]], None] | None = None, + verify: bool = True, +) -> str: ... +``` + +两轮 VLM 调用保真： + +``` +1. [可选] OCR 转录 → 事前并置到 user_content +2. 提取轮: VLM + observe_frame_extract.md +3. [可选] 验证轮: VLM + observe_frame_verify.md +4. 返回 "[视觉观察] {证据}\n[验证] {核实结果}" +``` + +**与 TRM4 有意变更**： + +| 项目 | TRM4 | TRM5 | +|------|------|------| +| 异步 | `_call_vl` 同步 | `await vlm.chat_with_images()` | +| VLM 接口 | 裸 LLMClient + 手动 base64 | VLMProvider Protocol，images 传 Path | +| OCR 接口 | `Callable[[list[Path]], str]` | `OCRProvider` Protocol（async） | +| Prompt 内容 | store/prompts/v2/ | 原封不动复制 | + +### 4.5 OCRProvider Protocol（core/protocols.py 新增） + +```python +@runtime_checkable +class OCRProvider(Protocol): + """帧文字转录端口。""" + async def transcribe_frames(self, frame_paths: list[Path]) -> str: ... +``` + +### 4.6 MonkeyOCRClient（adapters/ocr.py） + +```python +class MonkeyOCRClient: + """实现 OCRProvider Protocol。多端点轮询 + 单帧降级。""" + def __init__(self, urls: list[str]) -> None: ... + async def check_health(self) -> None: ... + async def transcribe_frames(self, frame_paths: list[Path]) -> str: ... +``` + +内部同步 HTTP 调用通过 `asyncio.to_thread` 包装。端点轮询 + 线程安全 Session 保留 TRM4 逻辑。 + +--- + +## §5 交互流程 + +```mermaid +sequenceDiagram + participant H as harness/inference + participant PM as PromptManager + participant SK as discover_skills + participant AL as AgentLoop + participant TD as SearchToolDispatcher + participant ENV as TreeEnvironment + participant V as vision.observe_frame + participant VLM as VLMProvider + participant OCR as OCRProvider + + H->>SK: discover_skills(skills_dir) + SK-->>H: (always_text, task_skill_map, catalog_text, registry) + H->>PM: build_inference_prompt(...) + PM-->>H: system_prompt + H->>PM: format_user_prompt(question, options, l1_ids) + PM-->>H: user_prompt + H->>TD: 构造(env, vlm, ocr, prompts_dir, registry, embed_fn) + H->>AL: run(system_prompt, user_prompt, tool_dispatcher) + + loop AgentLoop 每步推理 + AL->>TD: dispatch("view_node", args, context) + TD->>ENV: view_node(node_id) + ENV-->>TD: 节点文本 + TD-->>AL: 输出 + + AL->>TD: dispatch("observe_frame", args, context) + TD->>ENV: resolve_frame_paths(node_ids) + ENV-->>TD: list[Path] + TD->>V: observe_frame(vlm, paths, question, ...) + V->>OCR: transcribe_frames(paths) + OCR-->>V: ocr_text + V->>VLM: chat_with_images(extract prompt) + VLM-->>V: raw_evidence + V->>VLM: chat_with_images(verify prompt) + VLM-->>V: verify_result + V-->>TD: "[视觉观察] ...\n[验证] ..." + TD-->>AL: 输出 + + AL->>TD: dispatch("submit_answer", args, context) + TD-->>AL: "[ok] 答案已提交" + end + AL-->>H: LoopResult +``` + +--- + +## §6 错误处理 + +| 场景 | 处理 | 与 TRM4 一致性 | +|------|------|---------------| +| 节点不存在 | env 抛 KeyError，dispatcher 捕获返回错误文本 | 一致 | +| 帧文件不存在 | FileNotFoundError，vision 返回 `[VL错误]` | 一致 | +| VLM 提取轮失败 | 捕获 Exception，返回 `[VL错误]` | 一致 | +| VLM 验证轮失败 | 降级返回 `[验证] 跳过（调用失败）` | 一致 | +| OCR 失败 | 降级不注入，stats `ocr_failed=1` | 一致 | +| 未知工具名 | raise ValueError，AgentLoop 不计步 | 一致 | +| read_skill 未注册 | KeyError 透传，dispatcher 捕获返回错误文本 | 一致 | + +原则：工具执行错误不中断 AgentLoop，所有异常在 dispatcher 层兜底为错误文本。 + +--- + +## §7 测试策略 + +| 测试文件 | 覆盖 | 方法 | +|----------|------|------| +| `test_search_prompt.py` | PromptManager 加载/拼装/格式化 | 临时目录写 prompt 文件 | +| `test_search_skills.py` | frontmatter 解析、discover_skills 分类 | 临时目录写 .md | +| `test_search_tools.py` | SearchToolDispatcher 5 个工具分发 | 假 env/VLM/OCR 通过 Protocol 注入 | +| `test_search_vision.py` | observe_frame 两轮、OCR 注入/降级、stats | 假 VLMProvider + 假 OCRProvider | +| `test_ocr_adapter.py` | MonkeyOCRClient 健康检查/轮询/降级 | responses 库 mock HTTP | + +--- + +## §8 被否决的方案 + +| 方案 | 否决理由 | +|------|---------| +| vision.py 放 app/tree/ | observe_frame 是搜索工具实现，不是建树管线；tree/ 是离线预处理模块 | +| tools/ 子包 | 当前仅 5 个工具，子包过度组织 | diff --git a/research-wiki/graph/edges.json b/research-wiki/graph/edges.json index 497384d..3fa4f3f 100644 --- a/research-wiki/graph/edges.json +++ b/research-wiki/graph/edges.json @@ -25,6 +25,16 @@ "id": "plan:tree-module-vertical-slice", "label": "建树模块竖切实现计划", "type": "plan" + }, + { + "id": "plan:question-gen", + "label": "question_gen 模块实现计划", + "type": "plan" + }, + { + "id": "design:2026-07-07-search-module-design", + "label": "搜索 Agent 装配层设计（app/search/）", + "type": "design" } ], "links": [ @@ -41,6 +51,13 @@ "relation": "implements", "evidence": "计划逐 Task 实现设计文档中的 11 个模块", "added": "2026-07-07T05:27:02.953166+00:00" + }, + { + "source": "plan:question-gen", + "target": "design:question-gen", + "relation": "implements", + "evidence": "实现计划覆盖设计文档的全部需求", + "added": "2026-07-07T08:32:53.887071+00:00" } ] } \ No newline at end of file diff --git a/research-wiki/index.md b/research-wiki/index.md index 9209e92..de34ade 100644 --- a/research-wiki/index.md +++ b/research-wiki/index.md @@ -1,15 +1,19 @@ # Research Wiki 索引 -> 自动生成，更新时间：2026-07-07 05:27 UTC +> 自动生成，更新时间：2026-07-07 09:14 UTC -## design (3) +## design (5) - [2026-07-06-core-agent-adapters-llm-design](designs/2026-07-06-core-agent-adapters-llm-design.md) `design:2026-07-06-core-agent-adapters-llm-design` +- [出题模块迁移设计（question_gen）](designs/2026-07-07-question-gen-design.md) `design:2026-07-07-question-gen-design` - [建树模块竖切设计：数据结构 + 建树 + 修复 + 迁移](designs/2026-07-07-tree-module-design.md) `design:2026-07-07-tree-module-design` - [建树模块竖切设计：数据结构 + 建树 + 修复 + 迁移](designs/tree-module-vertical-slice.md) `design:tree-module-vertical-slice` +- [搜索 Agent 装配层设计（app/search/）](designs/2026-07-07-search-module-design.md) `design:2026-07-07-search-module-design` -## plan (5) +## plan (7) - [2026-07-06-core-agent-adapters-llm](plans/2026-07-06-core-agent-adapters-llm.md) `plan:2026-07-06-core-agent-adapters-llm` +- [2026-07-07-question-gen](plans/2026-07-07-question-gen.md) `plan:2026-07-07-question-gen` - [2026-07-07-tree-module-vertical-slice](plans/2026-07-07-tree-module-vertical-slice.md) `plan:2026-07-07-tree-module-vertical-slice` - [core/agent/ + adapters/llm 基础设施实现计划](plans/core-agent-adapters-llm.md) `plan:core-agent-adapters-llm` +- [question_gen 模块实现计划](plans/question-gen.md) `plan:question-gen` - [建树模块竖切实现计划](plans/tree-module-vertical-slice.md) `plan:tree-module-vertical-slice` - [项目基础设施初始化计划](plans/infrastructure-setup.md) `plan:infrastructure-setup` diff --git a/research-wiki/log.md b/research-wiki/log.md index 9c0ddf0..7922f68 100644 --- a/research-wiki/log.md +++ b/research-wiki/log.md @@ -12,3 +12,8 @@ - [2026-07-07 05:26 UTC] 新增 plan: 建树模块竖切实现计划 (plan:tree-module-vertical-slice) - [2026-07-07 05:27 UTC] 新增边: plan:tree-module-vertical-slice --implements--> design:tree-module-vertical-slice - [2026-07-07 05:27 UTC] 重建索引: 8 篇页面 +- [2026-07-07 08:32 UTC] 新增 plan: question_gen 模块实现计划 (plan:question-gen) +- [2026-07-07 08:32 UTC] 新增边: plan:question-gen --implements--> design:question-gen +- [2026-07-07 08:32 UTC] 重建索引: 11 篇页面 +- [2026-07-07 09:14 UTC] 新增 design: 搜索 Agent 装配层设计（app/search/） (design:2026-07-07-search-module-design) +- [2026-07-07 09:14 UTC] 重建索引: 12 篇页面