docs(design): 搜索 Agent 装配层设计(app/search/)
方案 A 平铺模块:prompt.py / skills.py / tools.py / vision.py 新增 OCRProvider Protocol + adapters/ocr.py Prompt 从 TRM4 store/prompts/v2/ 原封不动复制
This commit is contained in:
@@ -0,0 +1,313 @@
|
|||||||
|
---
|
||||||
|
type: design
|
||||||
|
node_id: design:2026-07-07-search-module-design
|
||||||
|
title: "搜索 Agent 装配层设计(app/search/)"
|
||||||
|
date: 2026-07-07
|
||||||
|
---
|
||||||
|
|
||||||
|
# 搜索 Agent 装配层设计(app/search/)
|
||||||
|
|
||||||
|
**日期** 2026-07-07 · **状态** 已批准 · **关联** TRM4 `core/search/` + `core/tree/tools.py` + `core/tree/vision.py`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## §1 定位
|
||||||
|
|
||||||
|
`app/search/` 是搜索 Agent 的"装配层"——为 `core/agent/loop.py` AgentLoop 提供 **prompt 组装**、**skill 管理**、**工具定义/分发** 和 **视觉观察**。它不控制推理循环,只被 AgentLoop 调用;编排责任在 `app/harness/inference.py`。
|
||||||
|
|
||||||
|
### 与 TRM4 的映射
|
||||||
|
|
||||||
|
| TRM4 | TRM5 | 变更类型 |
|
||||||
|
|------|------|---------|
|
||||||
|
| `core/search/prompt.py` | `app/search/prompt.py` | 保真迁移 + P4 显式参数 |
|
||||||
|
| `core/search/skills.py` | `app/search/skills.py` | 保真迁移 |
|
||||||
|
| `core/tree/tools.py` | `app/search/tools.py` | 重组为 `SearchToolDispatcher` 类 |
|
||||||
|
| `core/tree/vision.py` | `app/search/vision.py` | 异步化 + Protocol 注入 |
|
||||||
|
| `core/tree/ocr.py` | `adapters/ocr.py` | 异步化 + OCRProvider Protocol |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## §2 模块结构
|
||||||
|
|
||||||
|
```
|
||||||
|
app/search/
|
||||||
|
├── __init__.py # 公开 API 重导出
|
||||||
|
├── prompt.py # PromptManager — prompt 加载与拼装
|
||||||
|
├── skills.py # SkillRegistry + discover_skills — skill 扫描与注册
|
||||||
|
├── tools.py # SearchToolDispatcher(实现 ToolDispatcher Protocol)
|
||||||
|
└── vision.py # observe_frame(VLM 两轮 + OCR 注入)
|
||||||
|
|
||||||
|
adapters/
|
||||||
|
└── ocr.py # MonkeyOCRClient(实现 OCRProvider Protocol)
|
||||||
|
|
||||||
|
core/protocols.py # 新增 OCRProvider Protocol
|
||||||
|
|
||||||
|
store/prompts/ # 初始种子(从 TRM4 v2 直接复制,不修改)
|
||||||
|
├── system.md
|
||||||
|
├── observe_frame_extract.md
|
||||||
|
└── observe_frame_verify.md
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## §3 依赖方向
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TB
|
||||||
|
subgraph adapters
|
||||||
|
OCR_IMPL["adapters/ocr.py\nMonkeyOCRClient"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph core
|
||||||
|
PROTO["core/protocols.py\nOCRProvider Protocol"]
|
||||||
|
AGENT_PROTO["core/agent/protocols.py\nToolDispatcher Protocol"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph app/search
|
||||||
|
PROMPT["prompt.py\nPromptManager"]
|
||||||
|
SKILLS["skills.py\nSkillRegistry"]
|
||||||
|
TOOLS["tools.py\nSearchToolDispatcher"]
|
||||||
|
VISION["vision.py\nobserve_frame"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph app/tree
|
||||||
|
ENV["environment.py\nTreeEnvironment"]
|
||||||
|
end
|
||||||
|
|
||||||
|
OCR_IMPL -->|实现| PROTO
|
||||||
|
TOOLS -->|实现| AGENT_PROTO
|
||||||
|
TOOLS --> ENV
|
||||||
|
TOOLS --> SKILLS
|
||||||
|
TOOLS --> VISION
|
||||||
|
VISION -->|依赖| PROTO
|
||||||
|
PROMPT --> SKILLS
|
||||||
|
PROMPT -.->|读取| STORE["store/prompts/*.md"]
|
||||||
|
```
|
||||||
|
|
||||||
|
依赖只向内或同层,`core/` 不认识 `app/search/`。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## §4 公开 API
|
||||||
|
|
||||||
|
### 4.1 PromptManager(prompt.py)
|
||||||
|
|
||||||
|
```python
|
||||||
|
class PromptManager:
|
||||||
|
def __init__(self, prompts_dir: Path) -> None: ...
|
||||||
|
def build_inference_prompt(
|
||||||
|
self,
|
||||||
|
skill_mode: str,
|
||||||
|
task_type: str,
|
||||||
|
always_skills_text: str,
|
||||||
|
task_skill_map: dict[str, str],
|
||||||
|
catalog_text: str,
|
||||||
|
) -> str: ...
|
||||||
|
def format_user_prompt(
|
||||||
|
self,
|
||||||
|
question: str,
|
||||||
|
options: list[str],
|
||||||
|
l1_node_ids: list[str],
|
||||||
|
task_type: str | None = None,
|
||||||
|
) -> str: ...
|
||||||
|
def load(self, name: str) -> str: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
**与 TRM4 有意变更**:
|
||||||
|
- 工具描述从 `app/search/tools.py` 的 `get_tool_descriptions()` 获取(职责归属修正)
|
||||||
|
- `format_user_prompt` 参数从 `dict` 改为显式 `question` / `options` / `l1_node_ids`(P4)
|
||||||
|
|
||||||
|
### 4.2 SkillRegistry + discover_skills(skills.py)
|
||||||
|
|
||||||
|
```python
|
||||||
|
def parse_frontmatter(text: str) -> dict[str, str]: ...
|
||||||
|
def strip_frontmatter(text: str) -> str: ...
|
||||||
|
|
||||||
|
class SkillRegistry:
|
||||||
|
def set_paths(self, mapping: dict[str, Path]) -> None: ...
|
||||||
|
def read(self, name: str) -> str: ...
|
||||||
|
|
||||||
|
def discover_skills(skills_dir: Path) -> tuple[str, dict[str, str], str, SkillRegistry]: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
与 TRM4 逻辑完全一致,无有意变更。
|
||||||
|
|
||||||
|
### 4.3 SearchToolDispatcher(tools.py)
|
||||||
|
|
||||||
|
```python
|
||||||
|
def get_tool_descriptions(include_read_skill: bool = False) -> str: ...
|
||||||
|
|
||||||
|
class SearchToolDispatcher:
|
||||||
|
"""实现 core/agent/protocols.ToolDispatcher。"""
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
env: TreeEnvironment,
|
||||||
|
vlm: VLMProvider,
|
||||||
|
ocr: OCRProvider | None,
|
||||||
|
prompts_dir: Path,
|
||||||
|
skills: SkillRegistry | None,
|
||||||
|
*,
|
||||||
|
embed_fn: Callable[[str | list[str]], np.ndarray],
|
||||||
|
verify_vision: bool = True,
|
||||||
|
stats_sink: Callable[[dict[str, int]], None] | None = None,
|
||||||
|
) -> None: ...
|
||||||
|
|
||||||
|
async def dispatch(
|
||||||
|
self, tool_name: str, args: dict[str, Any], *, context: dict[str, Any]
|
||||||
|
) -> str: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
| 工具 | 实现路径 |
|
||||||
|
|------|---------|
|
||||||
|
| `view_node` | → `env.view_node(node_id)` |
|
||||||
|
| `search_similar` | → `env.search_similar(query, top_k, embed_fn=...)` + 格式化 |
|
||||||
|
| `observe_frame` | → `env.resolve_frame_paths(...)` + `vision.observe_frame(...)` |
|
||||||
|
| `submit_answer` | → 返回确认文本 |
|
||||||
|
| `read_skill` | → `skills.read(name)` |
|
||||||
|
| 未知工具 | → `raise ValueError`(AgentLoop 捕获,不计步) |
|
||||||
|
|
||||||
|
**与 TRM4 有意变更**:
|
||||||
|
- 自由函数 + 大量位置参数 → 类封装(构造时注入依赖)
|
||||||
|
- 工具描述 `get_tool_descriptions()` 移入此文件
|
||||||
|
- `search_similar` 结果格式化由 dispatcher 负责(env 返回 `list[tuple[str, float]]`)
|
||||||
|
|
||||||
|
### 4.4 observe_frame(vision.py)
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def observe_frame(
|
||||||
|
vlm: VLMProvider,
|
||||||
|
frame_paths: list[Path],
|
||||||
|
question: str,
|
||||||
|
prompts_dir: Path,
|
||||||
|
*,
|
||||||
|
ocr: OCRProvider | None,
|
||||||
|
stats_sink: Callable[[dict[str, int]], None] | None = None,
|
||||||
|
verify: bool = True,
|
||||||
|
) -> str: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
两轮 VLM 调用保真:
|
||||||
|
|
||||||
|
```
|
||||||
|
1. [可选] OCR 转录 → 事前并置到 user_content
|
||||||
|
2. 提取轮: VLM + observe_frame_extract.md
|
||||||
|
3. [可选] 验证轮: VLM + observe_frame_verify.md
|
||||||
|
4. 返回 "[视觉观察] {证据}\n[验证] {核实结果}"
|
||||||
|
```
|
||||||
|
|
||||||
|
**与 TRM4 有意变更**:
|
||||||
|
|
||||||
|
| 项目 | TRM4 | TRM5 |
|
||||||
|
|------|------|------|
|
||||||
|
| 异步 | `_call_vl` 同步 | `await vlm.chat_with_images()` |
|
||||||
|
| VLM 接口 | 裸 LLMClient + 手动 base64 | VLMProvider Protocol,images 传 Path |
|
||||||
|
| OCR 接口 | `Callable[[list[Path]], str]` | `OCRProvider` Protocol(async) |
|
||||||
|
| Prompt 内容 | store/prompts/v2/ | 原封不动复制 |
|
||||||
|
|
||||||
|
### 4.5 OCRProvider Protocol(core/protocols.py 新增)
|
||||||
|
|
||||||
|
```python
|
||||||
|
@runtime_checkable
|
||||||
|
class OCRProvider(Protocol):
|
||||||
|
"""帧文字转录端口。"""
|
||||||
|
async def transcribe_frames(self, frame_paths: list[Path]) -> str: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.6 MonkeyOCRClient(adapters/ocr.py)
|
||||||
|
|
||||||
|
```python
|
||||||
|
class MonkeyOCRClient:
|
||||||
|
"""实现 OCRProvider Protocol。多端点轮询 + 单帧降级。"""
|
||||||
|
def __init__(self, urls: list[str]) -> None: ...
|
||||||
|
async def check_health(self) -> None: ...
|
||||||
|
async def transcribe_frames(self, frame_paths: list[Path]) -> str: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
内部同步 HTTP 调用通过 `asyncio.to_thread` 包装。端点轮询 + 线程安全 Session 保留 TRM4 逻辑。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## §5 交互流程
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant H as harness/inference
|
||||||
|
participant PM as PromptManager
|
||||||
|
participant SK as discover_skills
|
||||||
|
participant AL as AgentLoop
|
||||||
|
participant TD as SearchToolDispatcher
|
||||||
|
participant ENV as TreeEnvironment
|
||||||
|
participant V as vision.observe_frame
|
||||||
|
participant VLM as VLMProvider
|
||||||
|
participant OCR as OCRProvider
|
||||||
|
|
||||||
|
H->>SK: discover_skills(skills_dir)
|
||||||
|
SK-->>H: (always_text, task_skill_map, catalog_text, registry)
|
||||||
|
H->>PM: build_inference_prompt(...)
|
||||||
|
PM-->>H: system_prompt
|
||||||
|
H->>PM: format_user_prompt(question, options, l1_ids)
|
||||||
|
PM-->>H: user_prompt
|
||||||
|
H->>TD: 构造(env, vlm, ocr, prompts_dir, registry, embed_fn)
|
||||||
|
H->>AL: run(system_prompt, user_prompt, tool_dispatcher)
|
||||||
|
|
||||||
|
loop AgentLoop 每步推理
|
||||||
|
AL->>TD: dispatch("view_node", args, context)
|
||||||
|
TD->>ENV: view_node(node_id)
|
||||||
|
ENV-->>TD: 节点文本
|
||||||
|
TD-->>AL: 输出
|
||||||
|
|
||||||
|
AL->>TD: dispatch("observe_frame", args, context)
|
||||||
|
TD->>ENV: resolve_frame_paths(node_ids)
|
||||||
|
ENV-->>TD: list[Path]
|
||||||
|
TD->>V: observe_frame(vlm, paths, question, ...)
|
||||||
|
V->>OCR: transcribe_frames(paths)
|
||||||
|
OCR-->>V: ocr_text
|
||||||
|
V->>VLM: chat_with_images(extract prompt)
|
||||||
|
VLM-->>V: raw_evidence
|
||||||
|
V->>VLM: chat_with_images(verify prompt)
|
||||||
|
VLM-->>V: verify_result
|
||||||
|
V-->>TD: "[视觉观察] ...\n[验证] ..."
|
||||||
|
TD-->>AL: 输出
|
||||||
|
|
||||||
|
AL->>TD: dispatch("submit_answer", args, context)
|
||||||
|
TD-->>AL: "[ok] 答案已提交"
|
||||||
|
end
|
||||||
|
AL-->>H: LoopResult
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## §6 错误处理
|
||||||
|
|
||||||
|
| 场景 | 处理 | 与 TRM4 一致性 |
|
||||||
|
|------|------|---------------|
|
||||||
|
| 节点不存在 | env 抛 KeyError,dispatcher 捕获返回错误文本 | 一致 |
|
||||||
|
| 帧文件不存在 | FileNotFoundError,vision 返回 `[VL错误]` | 一致 |
|
||||||
|
| VLM 提取轮失败 | 捕获 Exception,返回 `[VL错误]` | 一致 |
|
||||||
|
| VLM 验证轮失败 | 降级返回 `[验证] 跳过(调用失败)` | 一致 |
|
||||||
|
| OCR 失败 | 降级不注入,stats `ocr_failed=1` | 一致 |
|
||||||
|
| 未知工具名 | raise ValueError,AgentLoop 不计步 | 一致 |
|
||||||
|
| read_skill 未注册 | KeyError 透传,dispatcher 捕获返回错误文本 | 一致 |
|
||||||
|
|
||||||
|
原则:工具执行错误不中断 AgentLoop,所有异常在 dispatcher 层兜底为错误文本。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## §7 测试策略
|
||||||
|
|
||||||
|
| 测试文件 | 覆盖 | 方法 |
|
||||||
|
|----------|------|------|
|
||||||
|
| `test_search_prompt.py` | PromptManager 加载/拼装/格式化 | 临时目录写 prompt 文件 |
|
||||||
|
| `test_search_skills.py` | frontmatter 解析、discover_skills 分类 | 临时目录写 .md |
|
||||||
|
| `test_search_tools.py` | SearchToolDispatcher 5 个工具分发 | 假 env/VLM/OCR 通过 Protocol 注入 |
|
||||||
|
| `test_search_vision.py` | observe_frame 两轮、OCR 注入/降级、stats | 假 VLMProvider + 假 OCRProvider |
|
||||||
|
| `test_ocr_adapter.py` | MonkeyOCRClient 健康检查/轮询/降级 | responses 库 mock HTTP |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## §8 被否决的方案
|
||||||
|
|
||||||
|
| 方案 | 否决理由 |
|
||||||
|
|------|---------|
|
||||||
|
| vision.py 放 app/tree/ | observe_frame 是搜索工具实现,不是建树管线;tree/ 是离线预处理模块 |
|
||||||
|
| tools/ 子包 | 当前仅 5 个工具,子包过度组织 |
|
||||||
@@ -25,6 +25,16 @@
|
|||||||
"id": "plan:tree-module-vertical-slice",
|
"id": "plan:tree-module-vertical-slice",
|
||||||
"label": "建树模块竖切实现计划",
|
"label": "建树模块竖切实现计划",
|
||||||
"type": "plan"
|
"type": "plan"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "plan:question-gen",
|
||||||
|
"label": "question_gen 模块实现计划",
|
||||||
|
"type": "plan"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "design:2026-07-07-search-module-design",
|
||||||
|
"label": "搜索 Agent 装配层设计(app/search/)",
|
||||||
|
"type": "design"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"links": [
|
"links": [
|
||||||
@@ -41,6 +51,13 @@
|
|||||||
"relation": "implements",
|
"relation": "implements",
|
||||||
"evidence": "计划逐 Task 实现设计文档中的 11 个模块",
|
"evidence": "计划逐 Task 实现设计文档中的 11 个模块",
|
||||||
"added": "2026-07-07T05:27:02.953166+00:00"
|
"added": "2026-07-07T05:27:02.953166+00:00"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"source": "plan:question-gen",
|
||||||
|
"target": "design:question-gen",
|
||||||
|
"relation": "implements",
|
||||||
|
"evidence": "实现计划覆盖设计文档的全部需求",
|
||||||
|
"added": "2026-07-07T08:32:53.887071+00:00"
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
@@ -1,15 +1,19 @@
|
|||||||
# Research Wiki 索引
|
# Research Wiki 索引
|
||||||
|
|
||||||
> 自动生成,更新时间:2026-07-07 05:27 UTC
|
> 自动生成,更新时间:2026-07-07 09:14 UTC
|
||||||
|
|
||||||
## design (3)
|
## design (5)
|
||||||
- [2026-07-06-core-agent-adapters-llm-design](designs/2026-07-06-core-agent-adapters-llm-design.md) `design:2026-07-06-core-agent-adapters-llm-design`
|
- [2026-07-06-core-agent-adapters-llm-design](designs/2026-07-06-core-agent-adapters-llm-design.md) `design:2026-07-06-core-agent-adapters-llm-design`
|
||||||
|
- [出题模块迁移设计(question_gen)](designs/2026-07-07-question-gen-design.md) `design:2026-07-07-question-gen-design`
|
||||||
- [建树模块竖切设计:数据结构 + 建树 + 修复 + 迁移](designs/2026-07-07-tree-module-design.md) `design:2026-07-07-tree-module-design`
|
- [建树模块竖切设计:数据结构 + 建树 + 修复 + 迁移](designs/2026-07-07-tree-module-design.md) `design:2026-07-07-tree-module-design`
|
||||||
- [建树模块竖切设计:数据结构 + 建树 + 修复 + 迁移](designs/tree-module-vertical-slice.md) `design:tree-module-vertical-slice`
|
- [建树模块竖切设计:数据结构 + 建树 + 修复 + 迁移](designs/tree-module-vertical-slice.md) `design:tree-module-vertical-slice`
|
||||||
|
- [搜索 Agent 装配层设计(app/search/)](designs/2026-07-07-search-module-design.md) `design:2026-07-07-search-module-design`
|
||||||
|
|
||||||
## plan (5)
|
## plan (7)
|
||||||
- [2026-07-06-core-agent-adapters-llm](plans/2026-07-06-core-agent-adapters-llm.md) `plan:2026-07-06-core-agent-adapters-llm`
|
- [2026-07-06-core-agent-adapters-llm](plans/2026-07-06-core-agent-adapters-llm.md) `plan:2026-07-06-core-agent-adapters-llm`
|
||||||
|
- [2026-07-07-question-gen](plans/2026-07-07-question-gen.md) `plan:2026-07-07-question-gen`
|
||||||
- [2026-07-07-tree-module-vertical-slice](plans/2026-07-07-tree-module-vertical-slice.md) `plan:2026-07-07-tree-module-vertical-slice`
|
- [2026-07-07-tree-module-vertical-slice](plans/2026-07-07-tree-module-vertical-slice.md) `plan:2026-07-07-tree-module-vertical-slice`
|
||||||
- [core/agent/ + adapters/llm 基础设施实现计划](plans/core-agent-adapters-llm.md) `plan:core-agent-adapters-llm`
|
- [core/agent/ + adapters/llm 基础设施实现计划](plans/core-agent-adapters-llm.md) `plan:core-agent-adapters-llm`
|
||||||
|
- [question_gen 模块实现计划](plans/question-gen.md) `plan:question-gen`
|
||||||
- [建树模块竖切实现计划](plans/tree-module-vertical-slice.md) `plan:tree-module-vertical-slice`
|
- [建树模块竖切实现计划](plans/tree-module-vertical-slice.md) `plan:tree-module-vertical-slice`
|
||||||
- [项目基础设施初始化计划](plans/infrastructure-setup.md) `plan:infrastructure-setup`
|
- [项目基础设施初始化计划](plans/infrastructure-setup.md) `plan:infrastructure-setup`
|
||||||
|
|||||||
@@ -12,3 +12,8 @@
|
|||||||
- [2026-07-07 05:26 UTC] 新增 plan: 建树模块竖切实现计划 (plan:tree-module-vertical-slice)
|
- [2026-07-07 05:26 UTC] 新增 plan: 建树模块竖切实现计划 (plan:tree-module-vertical-slice)
|
||||||
- [2026-07-07 05:27 UTC] 新增边: plan:tree-module-vertical-slice --implements--> design:tree-module-vertical-slice
|
- [2026-07-07 05:27 UTC] 新增边: plan:tree-module-vertical-slice --implements--> design:tree-module-vertical-slice
|
||||||
- [2026-07-07 05:27 UTC] 重建索引: 8 篇页面
|
- [2026-07-07 05:27 UTC] 重建索引: 8 篇页面
|
||||||
|
- [2026-07-07 08:32 UTC] 新增 plan: question_gen 模块实现计划 (plan:question-gen)
|
||||||
|
- [2026-07-07 08:32 UTC] 新增边: plan:question-gen --implements--> design:question-gen
|
||||||
|
- [2026-07-07 08:32 UTC] 重建索引: 11 篇页面
|
||||||
|
- [2026-07-07 09:14 UTC] 新增 design: 搜索 Agent 装配层设计(app/search/) (design:2026-07-07-search-module-design)
|
||||||
|
- [2026-07-07 09:14 UTC] 重建索引: 12 篇页面
|
||||||
|
|||||||
Reference in New Issue
Block a user