docs(design): 修复 Codex 审查发现 — 10 项修正
Critical: OCRProvider 移至 app/ports.py;新增 TreeEnvironment 结构化 API;
observe_frame 补充字幕上下文拼接;search_similar 补充节点文本获取
Important: 遥测链路透传 session_id/parent_call_id;异常降级边界明确化;
verify_vision/anchor/assemble_mode 改为必传;Prompt 路径逐文件列出;
测试目录规范化到 tests/unit/
Minor: 依赖图补全 VLMProvider 连线
This commit is contained in:
@@ -42,18 +42,18 @@ app/search/
|
|||||||
adapters/
|
adapters/
|
||||||
└── ocr.py # MonkeyOCRClient(实现 OCRProvider Protocol)
|
└── ocr.py # MonkeyOCRClient(实现 OCRProvider Protocol)
|
||||||
|
|
||||||
core/protocols.py # 新增 OCRProvider Protocol
|
app/ports.py # 新增 OCRProvider Protocol(应用层端口,与 EmbeddingProvider 同级)
|
||||||
|
|
||||||
store/prompts/ # 初始种子(从 TRM4 v2 直接复制,不修改)
|
store/prompts/ # 初始种子,逐文件从 TRM4 store/prompts/v2/ 字节级复制,不修改
|
||||||
├── system.md
|
├── system.md # ← TRM4 store/prompts/v2/system.md
|
||||||
├── observe_frame_extract.md
|
├── observe_frame_extract.md # ← TRM4 store/prompts/v2/observe_frame_extract.md
|
||||||
├── observe_frame_verify.md
|
├── observe_frame_verify.md # ← TRM4 store/prompts/v2/observe_frame_verify.md
|
||||||
├── view_node_extract.md
|
├── view_node_extract.md # ← TRM4 store/prompts/v2/view_node_extract.md
|
||||||
├── view_node_verify.md
|
├── view_node_verify.md # ← TRM4 store/prompts/v2/view_node_verify.md
|
||||||
├── view_node_children_extract.md
|
├── view_node_children_extract.md # ← TRM4 store/prompts/v2/view_node_children_extract.md
|
||||||
├── view_node_children_verify.md
|
├── view_node_children_verify.md # ← TRM4 store/prompts/v2/view_node_children_verify.md
|
||||||
├── search_similar_extract.md
|
├── search_similar_extract.md # ← TRM4 store/prompts/v2/search_similar_extract.md
|
||||||
└── search_similar_verify.md
|
└── search_similar_verify.md # ← TRM4 store/prompts/v2/search_similar_verify.md
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -67,7 +67,7 @@ flowchart TB
|
|||||||
end
|
end
|
||||||
|
|
||||||
subgraph core
|
subgraph core
|
||||||
PROTO["core/protocols.py\nOCRProvider Protocol"]
|
PROTO["app/ports.py\nOCRProvider Protocol"]
|
||||||
AGENT_PROTO["core/agent/protocols.py\nToolDispatcher Protocol"]
|
AGENT_PROTO["core/agent/protocols.py\nToolDispatcher Protocol"]
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -91,6 +91,7 @@ flowchart TB
|
|||||||
TOOLS --> SUMM
|
TOOLS --> SUMM
|
||||||
SUMM -->|依赖| PROTO_LLM["core/protocols.py\nLLMProvider"]
|
SUMM -->|依赖| PROTO_LLM["core/protocols.py\nLLMProvider"]
|
||||||
VISION -->|依赖| PROTO
|
VISION -->|依赖| PROTO
|
||||||
|
VISION -->|依赖| PROTO_VLM["core/protocols.py\nVLMProvider"]
|
||||||
PROMPT --> SKILLS
|
PROMPT --> SKILLS
|
||||||
PROMPT -.->|读取| STORE["store/prompts/*.md"]
|
PROMPT -.->|读取| STORE["store/prompts/*.md"]
|
||||||
SUMM -.->|读取| STORE
|
SUMM -.->|读取| STORE
|
||||||
@@ -161,8 +162,9 @@ class SearchToolDispatcher:
|
|||||||
skills: SkillRegistry | None,
|
skills: SkillRegistry | None,
|
||||||
*,
|
*,
|
||||||
embed_fn: Callable[[str | list[str]], np.ndarray],
|
embed_fn: Callable[[str | list[str]], np.ndarray],
|
||||||
verify_vision: bool = True,
|
verify_vision: bool,
|
||||||
anchor: bool = False,
|
anchor: bool,
|
||||||
|
assemble_mode: str,
|
||||||
stats_sink: Callable[[dict[str, int]], None] | None = None,
|
stats_sink: Callable[[dict[str, int]], None] | None = None,
|
||||||
) -> None: ...
|
) -> None: ...
|
||||||
|
|
||||||
@@ -173,9 +175,9 @@ class SearchToolDispatcher:
|
|||||||
|
|
||||||
| 工具 | 实现路径 |
|
| 工具 | 实现路径 |
|
||||||
|------|---------|
|
|------|---------|
|
||||||
| `view_node` | → `env.view_node(node_id)` 获取原始文本 → `summarizer.summarize_node(...)` 两轮 LLM 摘要 + `summarizer.summarize_children(...)` 子节点标注 |
|
| `view_node` | → `env.get_node_text(node_id)` 获取原始文本 + `env.get_children_info(node_id)` 获取子节点结构化信息 → `summarizer.summarize_node(...)` 两轮 LLM 摘要 + `summarizer.summarize_children(...)` 子节点标注 |
|
||||||
| `search_similar` | → `env.search_similar(query, top_k, embed_fn=...)` → `summarizer.summarize_nodes_batch(...)` 并发两轮 LLM 摘要 + 格式化 |
|
| `search_similar` | → `env.search_similar(query, top_k, embed_fn=...)` 获取 `[(node_id, score)]` → 对每个 node_id 调 `env.get_node_text(node_id)` → `summarizer.summarize_nodes_batch(...)` 并发两轮 LLM 摘要 + 格式化 |
|
||||||
| `observe_frame` | → `env.resolve_frame_paths(...)` + `vision.observe_frame(...)` |
|
| `observe_frame` | → `env.resolve_frame_paths(...)` + `env.get_subtitle(node_ids[0])` 获取字幕 → `vision.observe_frame(...)` → 输出前拼接 `[字幕上下文]`(保真 TRM4 tools.py:136-153) |
|
||||||
| `submit_answer` | → 返回确认文本 |
|
| `submit_answer` | → 返回确认文本 |
|
||||||
| `read_skill` | → `skills.read(name)` |
|
| `read_skill` | → `skills.read(name)` |
|
||||||
| 未知工具 | → `raise ValueError`(AgentLoop 捕获,不计步) |
|
| 未知工具 | → `raise ValueError`(AgentLoop 捕获,不计步) |
|
||||||
@@ -185,6 +187,7 @@ class SearchToolDispatcher:
|
|||||||
- 工具描述 `get_tool_descriptions()` 移入此文件
|
- 工具描述 `get_tool_descriptions()` 移入此文件
|
||||||
- LLM 摘要从 environment 拆出到 `summarizer.py`(environment 回归纯数据层)
|
- LLM 摘要从 environment 拆出到 `summarizer.py`(environment 回归纯数据层)
|
||||||
- `SearchToolDispatcher.__init__` 新增 `tool_llm: LLMProvider` 参数(工具级 LLM,thinking=False,用于 summarizer)
|
- `SearchToolDispatcher.__init__` 新增 `tool_llm: LLMProvider` 参数(工具级 LLM,thinking=False,用于 summarizer)
|
||||||
|
- `dispatch()` 从 `context` 中提取 `session_id` / `parent_call_id`,透传给 summarizer / vision 的 LLM/VLM 调用,确保遥测链路完整
|
||||||
|
|
||||||
### 4.4 summarizer(summarizer.py)
|
### 4.4 summarizer(summarizer.py)
|
||||||
|
|
||||||
@@ -197,9 +200,11 @@ async def summarize_node(
|
|||||||
question: str,
|
question: str,
|
||||||
prompts_dir: Path,
|
prompts_dir: Path,
|
||||||
*,
|
*,
|
||||||
anchor_map: dict[str, str] | None = None,
|
anchor_map: dict[str, str] | None,
|
||||||
assemble_mode: str = "ids_expand",
|
assemble_mode: str,
|
||||||
stats_sink: Callable | None = None,
|
stats_sink: Callable | None = None,
|
||||||
|
session_id: str | None = None,
|
||||||
|
parent_call_id: str | None = None,
|
||||||
) -> str: ...
|
) -> str: ...
|
||||||
|
|
||||||
async def summarize_children(
|
async def summarize_children(
|
||||||
@@ -207,6 +212,9 @@ async def summarize_children(
|
|||||||
children_info: list[dict[str, Any]],
|
children_info: list[dict[str, Any]],
|
||||||
question: str,
|
question: str,
|
||||||
prompts_dir: Path,
|
prompts_dir: Path,
|
||||||
|
*,
|
||||||
|
session_id: str | None = None,
|
||||||
|
parent_call_id: str | None = None,
|
||||||
) -> str: ...
|
) -> str: ...
|
||||||
|
|
||||||
async def summarize_nodes_batch(
|
async def summarize_nodes_batch(
|
||||||
@@ -214,6 +222,9 @@ async def summarize_nodes_batch(
|
|||||||
items: list[tuple[str, str, str]],
|
items: list[tuple[str, str, str]],
|
||||||
question: str,
|
question: str,
|
||||||
prompts_dir: Path,
|
prompts_dir: Path,
|
||||||
|
*,
|
||||||
|
session_id: str | None = None,
|
||||||
|
parent_call_id: str | None = None,
|
||||||
) -> list[tuple[str, str]]: ...
|
) -> list[tuple[str, str]]: ...
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -246,8 +257,10 @@ async def observe_frame(
|
|||||||
prompts_dir: Path,
|
prompts_dir: Path,
|
||||||
*,
|
*,
|
||||||
ocr: OCRProvider | None,
|
ocr: OCRProvider | None,
|
||||||
|
verify: bool,
|
||||||
stats_sink: Callable[[dict[str, int]], None] | None = None,
|
stats_sink: Callable[[dict[str, int]], None] | None = None,
|
||||||
verify: bool = True,
|
session_id: str | None = None,
|
||||||
|
parent_call_id: str | None = None,
|
||||||
) -> str: ...
|
) -> str: ...
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -269,7 +282,7 @@ async def observe_frame(
|
|||||||
| OCR 接口 | `Callable[[list[Path]], str]` | `OCRProvider` Protocol(async) |
|
| OCR 接口 | `Callable[[list[Path]], str]` | `OCRProvider` Protocol(async) |
|
||||||
| Prompt 内容 | store/prompts/v2/ | 原封不动复制 |
|
| Prompt 内容 | store/prompts/v2/ | 原封不动复制 |
|
||||||
|
|
||||||
### 4.6 OCRProvider Protocol(core/protocols.py 新增)
|
### 4.6 OCRProvider Protocol(app/ports.py 新增)
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@runtime_checkable
|
@runtime_checkable
|
||||||
@@ -278,6 +291,8 @@ class OCRProvider(Protocol):
|
|||||||
async def transcribe_frames(self, frame_paths: list[Path]) -> str: ...
|
async def transcribe_frames(self, frame_paths: list[Path]) -> str: ...
|
||||||
```
|
```
|
||||||
|
|
||||||
|
放置在 `app/ports.py`(与 `EmbeddingProvider` 同级),而非 `core/protocols.py`——OCR 只被 `app/search/` 使用,不是 core 共享端口。
|
||||||
|
|
||||||
### 4.7 MonkeyOCRClient(adapters/ocr.py)
|
### 4.7 MonkeyOCRClient(adapters/ocr.py)
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@@ -290,6 +305,22 @@ class MonkeyOCRClient:
|
|||||||
|
|
||||||
内部同步 HTTP 调用通过 `asyncio.to_thread` 包装。端点轮询 + 线程安全 Session 保留 TRM4 逻辑。
|
内部同步 HTTP 调用通过 `asyncio.to_thread` 包装。端点轮询 + 线程安全 Session 保留 TRM4 逻辑。
|
||||||
|
|
||||||
|
### 4.8 TreeEnvironment 新增 API(app/tree/environment.py 扩展)
|
||||||
|
|
||||||
|
现有 `view_node()` 返回格式化字符串,不适合 summarizer 消费。需新增结构化查询方法:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def get_node_text(self, node_id: str, *, anchor: bool = False) -> tuple[str, dict[str, str] | None]:
|
||||||
|
"""返回节点原始文本(或带行号锚的文本)+ anchor_map。"""
|
||||||
|
...
|
||||||
|
|
||||||
|
def get_children_info(self, node_id: str) -> list[dict[str, Any]]:
|
||||||
|
"""返回子节点结构化信息列表 [{id, time_range, summary}, ...]。"""
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
现有 `view_node()` 和 `search_similar()` 保持不变(向后兼容),新方法专供 `SearchToolDispatcher` 使用。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## §5 交互流程
|
## §5 交互流程
|
||||||
@@ -374,7 +405,14 @@ sequenceDiagram
|
|||||||
| 未知工具名 | raise ValueError,AgentLoop 不计步 | 一致 |
|
| 未知工具名 | raise ValueError,AgentLoop 不计步 | 一致 |
|
||||||
| read_skill 未注册 | KeyError 透传,dispatcher 捕获返回错误文本 | 一致 |
|
| read_skill 未注册 | KeyError 透传,dispatcher 捕获返回错误文本 | 一致 |
|
||||||
|
|
||||||
原则:工具执行错误不中断 AgentLoop,所有异常在 dispatcher 层兜底为错误文本。
|
**原则**:工具执行错误不中断 AgentLoop。未知工具名 `raise ValueError`(由 AgentLoop 捕获不计步);已知工具的运行时错误在 dispatcher 层转为错误文本返回。
|
||||||
|
|
||||||
|
**允许的降级边界**(刻意宽泛捕获,与 TRM4 一致):
|
||||||
|
- OCR 转录失败 → 降级不注入(`ocr_fn` 是外部依赖,任何异常不得中断工具主流程)
|
||||||
|
- VLM 验证轮失败 → 降级跳过验证(提取结果仍然有效)
|
||||||
|
- summarize_children 失败 → 回退原始子节点列表
|
||||||
|
|
||||||
|
其他异常(如节点不存在、帧文件缺失)捕获特定异常类型,不做宽泛降级。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -382,12 +420,12 @@ sequenceDiagram
|
|||||||
|
|
||||||
| 测试文件 | 覆盖 | 方法 |
|
| 测试文件 | 覆盖 | 方法 |
|
||||||
|----------|------|------|
|
|----------|------|------|
|
||||||
| `test_search_prompt.py` | PromptManager 加载/拼装/格式化 | 临时目录写 prompt 文件 |
|
| `tests/unit/test_search_prompt.py` | PromptManager 加载/拼装/格式化 | 临时目录写真实 TRM4 v2 prompt 文件 |
|
||||||
| `test_search_skills.py` | frontmatter 解析、discover_skills 分类 | 临时目录写 .md |
|
| `tests/unit/test_search_skills.py` | frontmatter 解析、discover_skills 分类 | 临时目录写 .md |
|
||||||
| `test_search_tools.py` | SearchToolDispatcher 5 个工具分发 + 摘要集成 | 假 env/LLM/VLM/OCR 通过 Protocol 注入 |
|
| `tests/unit/test_search_tools.py` | SearchToolDispatcher 5 个工具分发 + 摘要集成 | 假 env/LLM/VLM/OCR 通过 Protocol 注入 |
|
||||||
| `test_search_summarizer.py` | summarize_node(含 anchor 模式)、summarize_children、summarize_nodes_batch | 假 LLMProvider;check_anchors / assemble 用真实输入 |
|
| `tests/unit/test_search_summarizer.py` | summarize_node(含 anchor 模式)、summarize_children、summarize_nodes_batch;check_anchors / assemble 纯函数用真实输入 | 假 LLMProvider |
|
||||||
| `test_search_vision.py` | observe_frame 两轮、OCR 注入/降级、stats | 假 VLMProvider + 假 OCRProvider |
|
| `tests/unit/test_search_vision.py` | observe_frame 两轮、OCR 注入/降级、stats、字幕拼接 | 假 VLMProvider + 假 OCRProvider |
|
||||||
| `test_ocr_adapter.py` | MonkeyOCRClient 健康检查/轮询/降级 | `responses` 库 mock HTTP |
|
| `tests/unit/test_ocr_adapter.py` | MonkeyOCRClient 健康检查/轮询/降级 | `responses` 库 mock HTTP |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user