feat: 增加评审文档机器投影与 JSON 报告
This commit is contained in:
@@ -3,8 +3,9 @@
|
|||||||
`mdpolish` 是实验室共用的、项目无关的 Python Markdown 修改库。它提供函数式 `Modifier`、精确文本编辑执行器、
|
`mdpolish` 是实验室共用的、项目无关的 Python Markdown 修改库。它提供函数式 `Modifier`、精确文本编辑执行器、
|
||||||
有序 `Pipeline`、正则修改器工厂，以及少量可以用合成样例完整说明的通用修改器。
|
有序 `Pipeline`、正则修改器工厂，以及少量可以用合成样例完整说明的通用修改器。
|
||||||
|
|
||||||
当前发布版本是 [`v0.4.0`](https://github.com/Bepr4/mdpolish/releases/tag/v0.4.0)。库只处理内存中的 Markdown 字符串，
|
当前发布版本是 [`v0.4.0`](https://github.com/Bepr4/mdpolish/releases/tag/v0.4.0)，当前源码树的下一候选版本是
|
||||||
不读取或写入文件，不提供默认流水线，也不包含任何项目的规则集合、数据清单、实验脚本或评审界面。
|
`0.5.0`。库只处理内存中的 Markdown 字符串，不读取或写入文件，不提供默认流水线，也不包含任何项目的规则集合、
|
||||||
|
数据清单、实验脚本或评审界面。
|
||||||
|
|
||||||
## 当前能力
|
## 当前能力
|
||||||
|
|
||||||
@@ -15,6 +16,8 @@
|
|||||||
| `Pipeline` | 按调用方顺序运行修改器，并对最终快照做只读稳定性复查 | 不自动选规则、不重排、不循环执行 |
|
| `Pipeline` | 按调用方顺序运行修改器，并对最终快照做只读稳定性复查 | 不自动选规则、不重排、不循环执行 |
|
||||||
| `build_review_document()` | 验证并重放已有结果，提供可信阶段、位置、全文和错误/残留证据 | 不重新运行修改器，不猜测损坏或不完整的结果 |
|
| `build_review_document()` | 验证并重放已有结果，提供可信阶段、位置、全文和错误/残留证据 | 不重新运行修改器，不猜测损坏或不完整的结果 |
|
||||||
| `render_markdown_report()` | 把评审视图编排成完整 Markdown 源码报告字符串 | 只返回内存字符串，不创建文件或业务页面 |
|
| `render_markdown_report()` | 把评审视图编排成完整 Markdown 源码报告字符串 | 只返回内存字符串，不创建文件或业务页面 |
|
||||||
|
| `review_document_to_dict()` | 按 schema `1.0` 把评审视图投影成普通 JSON 基本值 | 单向投影，不反序列化或重新应用修改 |
|
||||||
|
| `render_json_report()` | 复用正式 dict 投影生成确定的内存 JSON 字符串 | 不创建文件；默认摘要不等于公开安全日志 |
|
||||||
| `regex_replace()` | 把非空正则匹配转换为精确编辑 | 不提供规则注册表、配置加载或默认模式 |
|
| `regex_replace()` | 把非空正则匹配转换为精确编辑 | 不提供规则注册表、配置加载或默认模式 |
|
||||||
| `mapped_line_join()` | 用精确、正则或可选本地词典规则合并跨行片段 | 无默认规则；代码、表格、未知结构和歧义失败关闭 |
|
| `mapped_line_join()` | 用精确、正则或可选本地词典规则合并跨行片段 | 无默认规则；代码、表格、未知结构和歧义失败关闭 |
|
||||||
| HTML 表格修改器 | 处理严格表格子集的实体和单行布局 | 不是完整 HTML parser，也不是 HTML→GFM 转换器 |
|
| HTML 表格修改器 | 处理严格表格子集的实体和单行布局 | 不是完整 HTML parser，也不是 HTML→GFM 转换器 |
|
||||||
@@ -45,7 +48,8 @@ python -m pip install 'mdpolish[lexical] @ https://github.com/Bepr4/mdpolish/rel
|
|||||||
```
|
```
|
||||||
|
|
||||||
Release 页面同时提供 wheel 的 SHA-256 校验值。仓库或 Release 如果是私有的，调用方需要自行配置 GitHub 访问权限；
|
Release 页面同时提供 wheel 的 SHA-256 校验值。仓库或 Release 如果是私有的，调用方需要自行配置 GitHub 访问权限；
|
||||||
库不会保存凭据。开发环境仍从本地工作树安装：
|
库不会保存凭据。以上命令当前安装的是已发布的 `v0.4.0`，不包含本源码树尚未发布的 `0.5.0` 机器投影。开发环境仍从
|
||||||
|
本地工作树安装：
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python -m venv .venv
|
python -m venv .venv
|
||||||
@@ -158,23 +162,33 @@ if result.status is RunStatus.SUCCESS and result.output_markdown is not None:
|
|||||||
output_path.write_text(result.output_markdown, encoding="utf-8")
|
output_path.write_text(result.output_markdown, encoding="utf-8")
|
||||||
```
|
```
|
||||||
|
|
||||||
文件读取、输出命名、覆盖策略、批处理和 CLI 都属于调用项目。`mdpolish` 可以生成通用的内存评审视图与 Markdown 报告
|
文件读取、输出命名、覆盖策略、批处理和 CLI 都属于调用项目。`mdpolish` 可以生成通用的内存评审视图、机器投影、JSON
|
||||||
字符串，但不会自动保存它们，也不知道报告来自哪个文件。
|
字符串与 Markdown 报告字符串，但不会自动保存它们，也不知道报告来自哪个文件。
|
||||||
|
|
||||||
## 构建内存评审视图和报告
|
## 构建内存评审视图和报告
|
||||||
|
|
||||||
调用方保留原始 Markdown，并把它与 `TransformResult` 一起传给评审构建函数：
|
调用方保留原始 Markdown，并把它与 `TransformResult` 一起传给评审构建函数：
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from mdpolish.review import build_review_document, render_markdown_report
|
from mdpolish.review import (
|
||||||
|
build_review_document,
|
||||||
|
render_json_report,
|
||||||
|
render_markdown_report,
|
||||||
|
review_document_to_dict,
|
||||||
|
)
|
||||||
|
|
||||||
input_markdown = "an exam-\nple text"
|
input_markdown = "an exam-\nple text"
|
||||||
result = pipeline.transform(input_markdown)
|
result = pipeline.transform(input_markdown)
|
||||||
|
|
||||||
review = build_review_document(input_markdown, result)
|
review = build_review_document(input_markdown, result)
|
||||||
|
review_summary = review_document_to_dict(review)
|
||||||
|
report_json = render_json_report(review, detail="changes")
|
||||||
report_markdown = render_markdown_report(review)
|
report_markdown = render_markdown_report(review)
|
||||||
|
|
||||||
assert review.current_markdown == "an example text"
|
assert review.current_markdown == "an example text"
|
||||||
|
assert review_summary["schema_version"] == "1.0"
|
||||||
|
assert review_summary["detail"] == "summary"
|
||||||
|
assert report_json.startswith('{\n "schema_name": "mdpolish.review"')
|
||||||
assert report_markdown.startswith("# mdpolish review report\n")
|
assert report_markdown.startswith("# mdpolish review report\n")
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -188,6 +202,16 @@ Markdown reporter 会包含完整输入、当前全文、统一 diff 以及实
|
|||||||
前 20 条残留候选，并且不重复输出残留候选正文；调用项目如果保存报告，仍需负责路径、权限、脱敏和保留周期。每个完整
|
前 20 条残留候选，并且不重复输出残留候选正文；调用项目如果保存报告，仍需负责路径、权限、脱敏和保留周期。每个完整
|
||||||
阶段都会保留前后快照，当前版本没有承诺无限文档长度或修改器数量下的内存上限。
|
阶段都会保留前后快照，当前版本没有承诺无限文档长度或修改器数量下的内存上限。
|
||||||
|
|
||||||
|
机器投影使用独立于包版本的 schema `1.0`，并提供三个显式 detail：
|
||||||
|
|
||||||
|
- `summary`：默认值；只含状态、哈希、计数、modifier 身份、阶段摘要和稳定错误代码，不含正文承载字段；
|
||||||
|
- `changes`：再加入 modifier 参数、适用说明、实际修改片段、错误消息和未应用残留候选；
|
||||||
|
- `full`：再加入完整输入、当前全文和每个阶段的完整前后全文。
|
||||||
|
|
||||||
|
高 detail 可能还原敏感内容。即使 `summary` 不含正文，它仍然携带项目元数据和哈希，不能自动视为匿名或适合公开传播。
|
||||||
|
dict 和 JSON 都是单向派生视图，不用于恢复 `ReviewDocument` 或重新应用修改。完整 schema、坐标、哈希和兼容口径见
|
||||||
|
[`review-projection-schema-v1.md`](research-wiki/reference/review-projection-schema-v1.md)。
|
||||||
|
|
||||||
## 编写项目自己的修改器
|
## 编写项目自己的修改器
|
||||||
|
|
||||||
复杂规则使用普通函数返回精确候选修改，不需要继承库基类：
|
复杂规则使用普通函数返回精确候选修改，不需要继承库基类：
|
||||||
@@ -262,7 +286,7 @@ src/mdpolish/
|
|||||||
├── modifier.py # 函数式 Modifier 契约
|
├── modifier.py # 函数式 Modifier 契约
|
||||||
├── edits.py # 批次验证与原子应用
|
├── edits.py # 批次验证与原子应用
|
||||||
├── pipeline.py # 有序执行与最终稳定性复查
|
├── pipeline.py # 有序执行与最终稳定性复查
|
||||||
├── review.py # 可信评审投影与内存 Markdown reporter
|
├── review.py # 可信评审视图、机器投影及内存 JSON/Markdown reporter
|
||||||
├── regex.py # 正则修改器工厂
|
├── regex.py # 正则修改器工厂
|
||||||
└── modifiers/ # 少量项目无关的通用修改器
|
└── modifiers/ # 少量项目无关的通用修改器
|
||||||
tests/ # 只使用虚构文本的核心与通用修改器测试
|
tests/ # 只使用虚构文本的核心与通用修改器测试
|
||||||
@@ -279,14 +303,17 @@ research-wiki/
|
|||||||
- 文件适配器、公共 CLI、配置文件、profile 或批处理协议；
|
- 文件适配器、公共 CLI、配置文件、profile 或批处理协议；
|
||||||
- 自动规则发现、注册表或默认流水线；
|
- 自动规则发现、注册表或默认流水线；
|
||||||
- Markdown AST、完整 HTML parser 或必装的第三方运行依赖；
|
- Markdown AST、完整 HTML parser 或必装的第三方运行依赖；
|
||||||
- artifact、报告文件、JSON/HTML reporter、Web/桌面评审器或项目审核流程；
|
- artifact、自动保存的报告文件、正式 JSON Schema 文件、HTML reporter、Web/桌面评审器或项目审核流程；
|
||||||
- 任何业务项目的规则、固定参数、文档 ID、数据或验收统计。
|
- 任何业务项目的规则、固定参数、文档 ID、数据或验收统计。
|
||||||
|
|
||||||
公共边界与原因见
|
公共边界与原因见
|
||||||
[`0008-generic-functional-library-boundary.md`](research-wiki/design/0008-generic-functional-library-boundary.md)，当前机制见
|
[`0008-generic-functional-library-boundary.md`](research-wiki/design/0008-generic-functional-library-boundary.md)，当前机制见
|
||||||
[`functional-modifier-core.md`](research-wiki/explanation/functional-modifier-core.md)。通用内存评审能力的批准边界见
|
[`functional-modifier-core.md`](research-wiki/explanation/functional-modifier-core.md)。通用内存评审能力的批准边界见
|
||||||
[`0011-generic-review-projection-and-reporting.md`](research-wiki/design/0011-generic-review-projection-and-reporting.md)，当前机制见
|
[`0011-generic-review-projection-and-reporting.md`](research-wiki/design/0011-generic-review-projection-and-reporting.md)，当前机制见
|
||||||
[`review-projection.md`](research-wiki/explanation/review-projection.md)。旧 design 只保存历史决策，不代表当前交付能力。
|
[`review-projection.md`](research-wiki/explanation/review-projection.md)。正式机器投影的批准边界见
|
||||||
|
[`0012-review-document-machine-projection.md`](research-wiki/design/0012-review-document-machine-projection.md)，schema `1.0` 的稳定
|
||||||
|
查询口径见 [`review-projection-schema-v1.md`](research-wiki/reference/review-projection-schema-v1.md)。旧 design 只保存历史
|
||||||
|
决策，不代表当前交付能力。
|
||||||
|
|
||||||
## 当前可用检查
|
## 当前可用检查
|
||||||
|
|
||||||
@@ -316,3 +343,11 @@ Python 标准库。Release wheel 的 SHA-256 是
|
|||||||
`12e24863314958130ed082f78e89ab8bc0dad39a3848f2ae42e273d19a409693`。
|
`12e24863314958130ed082f78e89ab8bc0dad39a3848f2ae42e273d19a409693`。
|
||||||
|
|
||||||
上述结果证明当前版本可安装并按合成契约运行，不代表任意词典阈值已经在真实业务语料上达到生产准确率。
|
上述结果证明当前版本可安装并按合成契约运行，不代表任意词典阈值已经在真实业务语料上达到生产准确率。
|
||||||
|
|
||||||
|
`0.5.0` 候选于 2026-08-28 在 Python 3.13.11 开发环境中实际得到：mypy 通过，pytest 为
|
||||||
|
`228 passed, 3 skipped`；三个 skip 仍是没有安装的 optional backend。除工作区已有的 `src/mdpolish/regex.py` 中文注释
|
||||||
|
改动外，Ruff 全部通过；未排除该文件的全仓 Ruff 因其中 30 个 `RUF002` / `RUF003` 失败，本轮没有擅自修改该用户改动。
|
||||||
|
|
||||||
|
候选 `mdpolish-0.5.0-py3-none-any.whl` 构建成功，共 17 个文件，包含更新后的 `review.py` 和 `py.typed`，不包含 tests、Wiki、
|
||||||
|
报告或真实数据；在仓库外全新虚拟环境中无依赖安装后，dict 投影与 JSON reporter smoke test 通过。该临时 wheel 不是 Release
|
||||||
|
资产，其哈希不构成发布身份；完成全仓 Ruff、提交、合并、tag 和 Release 仍需要分别确认。
|
||||||
|
|||||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "mdpolish"
|
name = "mdpolish"
|
||||||
version = "0.4.0"
|
version = "0.5.0"
|
||||||
description = "Deterministic functional core for composing exact Markdown modifiers"
|
description = "Deterministic functional core for composing exact Markdown modifiers"
|
||||||
requires-python = ">=3.11"
|
requires-python = ">=3.11"
|
||||||
dependencies = []
|
dependencies = []
|
||||||
|
|||||||
@@ -0,0 +1,703 @@
|
|||||||
|
# 0012：ReviewDocument 的正式机器投影
|
||||||
|
|
||||||
|
## 状态
|
||||||
|
|
||||||
|
已于 2026-08-28 获用户明确批准，按本文第 16 节实施。本文自批准起冻结；后续改变决策需新增 design 并使用
|
||||||
|
`supersedes` 指向本文。
|
||||||
|
|
||||||
|
`supersedes: 0011`（范围有限）：本文只替代 `0011` 第 8 节“第一版不提供 JSON reporter”和其中把稳定序列化继续留给
|
||||||
|
调用项目的决定。`0011` 已批准并实现的可信重放、`ReviewDocument`、阶段坐标、Markdown reporter、无文件 I/O 和项目拥有
|
||||||
|
持久化决定权继续有效。
|
||||||
|
|
||||||
|
本文不改变 `Pipeline`、`TransformResult`、清洗语义或评审重放结果。它只定义怎样把已经建立并验证的
|
||||||
|
`ReviewDocument` 转成稳定的普通 Python 数据，再按同一结构编码为 JSON。
|
||||||
|
|
||||||
|
## 1. 问题与可观察现象
|
||||||
|
|
||||||
|
`v0.4.0` 已经可以在同一个 Python 进程中这样消费评审结果：
|
||||||
|
|
||||||
|
```python
|
||||||
|
review = build_review_document(input_markdown, result)
|
||||||
|
|
||||||
|
if review.status is RunStatus.SUCCESS:
|
||||||
|
print(review.current_sha256)
|
||||||
|
```
|
||||||
|
|
||||||
|
这时调用方直接使用不可变的 `ReviewDocument`、`ReviewStage` 和 `ReviewChange`，不需要序列化。
|
||||||
|
|
||||||
|
一旦评审结果需要经过 Web、数据库、消息队列、JSON 文件或其他语言，内部 Python 对象就不能直接作为契约。当前项目端只能
|
||||||
|
自己决定怎样处理 dataclass、`StrEnum`、tuple、半开范围、阶段哈希和部分输出。最短的做法看似是：
|
||||||
|
|
||||||
|
```python
|
||||||
|
payload = dataclasses.asdict(review)
|
||||||
|
```
|
||||||
|
|
||||||
|
但这会产生四类问题：
|
||||||
|
|
||||||
|
1. 内部 dataclass 字段会在未经评审的情况下变成外部协议；以后正常的 Python 重构也会破坏消费者；
|
||||||
|
2. enum、tuple、递归参数值和位置单位没有正式 JSON 表达，消费者容易形成不同解释；
|
||||||
|
3. `ReviewDocument` 包含完整输入、当前全文、每阶段全文和修改片段，机械展开会默认暴露全部正文；
|
||||||
|
4. 当前 `RunError.error_type` 是诊断用 Python 异常类名，不能被项目端误当成稳定错误代码。
|
||||||
|
|
||||||
|
因此，需要由 `mdpolish` 自己提供一个经过选择和转换的对外视图，而不是让每个项目根据内部字段猜一个版本。
|
||||||
|
|
||||||
|
本文把这个视图称为“机器投影”：它是 `ReviewDocument` 的有损、单向、稳定表示，不是内部对象的镜像，也不是可用于重新
|
||||||
|
应用修改的序列化快照。
|
||||||
|
|
||||||
|
## 2. 外部规范带来的约束
|
||||||
|
|
||||||
|
2026-08-28 查阅的官方规范给出以下直接约束：
|
||||||
|
|
||||||
|
- [RFC 8259](https://www.rfc-editor.org/rfc/rfc8259.html) 把 JSON object 定义为无序名称/值集合，把 array 定义为有序序列；
|
||||||
|
对外语义不能依赖 object 成员顺序，但修改器、阶段、修改和残留候选的顺序必须用 array 保存；
|
||||||
|
- RFC 8259 要求开放系统中的 JSON 文本使用 UTF-8，成员名应唯一，并指出超出 IEEE 754 binary64 精确整数范围的数字会降低
|
||||||
|
互操作性；本投影只发出唯一键、可 UTF-8 编码的字符串和安全范围整数；
|
||||||
|
- [Python `json` 文档](https://docs.python.org/3/library/json.html) 显示 `allow_nan` 默认允许非标准的 `NaN` / `Infinity`，
|
||||||
|
`ensure_ascii` 默认转义非 ASCII；官方 reporter 必须显式使用 `allow_nan=False` 和 `ensure_ascii=False`；
|
||||||
|
- [JSON Schema 2020-12](https://json-schema.org/draft/2020-12/json-schema-core.html) 区分 schema 版本和实例内容，并允许通过
|
||||||
|
schema 约束对象、数组和 enum；本轮先固定实例 schema 和兼容策略，不引入运行时 validator 或第三方 schema 依赖；
|
||||||
|
- [SARIF 2.1.0](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html) 区分稳定标识符与给人看的消息，并明确
|
||||||
|
不应从没有稳定标识的来源中猜造精细规则 ID；本投影只为现有 `ErrorStage` 提供粗粒度稳定代码，不从异常类名或文案
|
||||||
|
推断更细错误原因。
|
||||||
|
|
||||||
|
这些参考不表示 `mdpolish` 要实现 SARIF 或 JSON Schema validator。它们只帮助确定 JSON 互操作、顺序、版本和错误身份的
|
||||||
|
边界。
|
||||||
|
|
||||||
|
## 3. 目标与非目标
|
||||||
|
|
||||||
|
### 3.1 目标
|
||||||
|
|
||||||
|
- 提供官方 `ReviewDocument -> dict` 投影，返回只含 JSON 基本值的全新普通数据结构；
|
||||||
|
- 提供使用同一投影的确定性 JSON reporter，不建立第二套字段；
|
||||||
|
- 每个结果都携带独立于包版本的 `schema_version`；
|
||||||
|
- 固定 enum、array、哈希和位置的外部表达，避免调用方解释 Python 内部对象；
|
||||||
|
- 提供 `summary`、`changes`、`full` 三个单调增加的正文暴露等级，默认不暴露正文；
|
||||||
|
- 给当前三个错误阶段提供粗粒度、稳定、可供程序判断的代码，同时保留诊断字段的非稳定身份；
|
||||||
|
- 明确 schema 的兼容升级规则，并让旧消费者可以安全忽略同一主版本新增的可选字段；
|
||||||
|
- 保持纯函数、无文件 I/O、无网络、零第三方核心运行依赖；
|
||||||
|
- 只用合成内容验证 Unicode、换行、部分输出、残留候选和内容泄漏边界。
|
||||||
|
|
||||||
|
### 3.2 非目标
|
||||||
|
|
||||||
|
- 不提供 `dict` / JSON 到 `ReviewDocument` 的反序列化或 round-trip；
|
||||||
|
- 不把 JSON 投影作为重新应用 `Change`、恢复 Pipeline 或验证原始结果的权威输入；
|
||||||
|
- 不生成或保存 `.json` 文件，不接收路径，不决定目录、权限、覆盖或保留周期；
|
||||||
|
- 不提供 CLI、Web API、数据库模型、消息队列协议、OpenAPI、HTML 或项目页面；
|
||||||
|
- 不发布 SARIF、JSON Lines、JSON Patch、JSON-LD 或项目 artifact 格式；
|
||||||
|
- 不改变 `ReviewDocument`、`ReviewStage`、`Change`、`RunError` 或 `TransformResult` 的字段；
|
||||||
|
- 不增加细粒度的 Pipeline 错误原因。当前结果没有保存足够的稳定原因身份，本轮不解析异常消息来猜；
|
||||||
|
- 不把 Python 码点坐标转换成 UTF-16、UTF-8 byte offset、LSP 位置或终端显示宽度；
|
||||||
|
- 不承诺 `summary` 是可以公开传播的“安全日志”。它不含正文，但 modifier id、版本、计数和哈希仍可能属于项目元数据；
|
||||||
|
- 不读取、复制或修改项目端测试仓、外部真实文档、历史报告或数据库。
|
||||||
|
|
||||||
|
## 4. 方案比较
|
||||||
|
|
||||||
|
| 方案 | 优点 | 问题 | 选择 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| 各项目继续 `dataclasses.asdict()` | 上游零工作 | 内部结构意外变成协议；enum、tuple、正文和兼容策略失控 | 不采用 |
|
||||||
|
| 使用 Pydantic / Marshmallow 建模 | schema 和校验工具成熟 | 给零依赖核心增加运行依赖，并形成第二套评审模型 | 不采用 |
|
||||||
|
| 只提供 JSON reporter | 调用入口短 | Web 或数据库仍要解析 JSON 才能得到 Python 基本值 | 不采用 |
|
||||||
|
| 只提供 dict，不提供 reporter | 表面积最小 | 各项目会重复 JSON 编码选项，可能生成 NaN、ASCII 转义或不同格式 | 不采用 |
|
||||||
|
| 手写窄投影，JSON reporter 复用它 | 字段和暴露级别可审计；零依赖；dict 与 JSON 只有一个语义来源 | 需要长期维护 schema 兼容 | 采用 |
|
||||||
|
| 第一版同时发布 JSON Schema 文件 | 其他语言可直接验证 | 增加一份必须与代码同步的公共文件和打包契约；当前尚无独立 validator 需求 | 本轮不采用 |
|
||||||
|
|
||||||
|
如果真实消费者以后需要脱离 Python wheel 独立验证 payload，再新增 design 决定是否把 JSON Schema 文件作为 Release 资产或
|
||||||
|
包资源发布，不能根据本文自动补一个未维护的 schema 文件。
|
||||||
|
|
||||||
|
## 5. 职责与数据流
|
||||||
|
|
||||||
|
```text
|
||||||
|
Pipeline.transform()
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
TransformResult + 原始 Markdown
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
build_review_document() 0011：验证和可信重放
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
ReviewDocument
|
||||||
|
│
|
||||||
|
├── render_markdown_report() 人工完整评审
|
||||||
|
│
|
||||||
|
└── review_document_to_dict(detail=...)
|
||||||
|
│
|
||||||
|
├── 项目 Web / 数据库 / 其他语言
|
||||||
|
│
|
||||||
|
└── render_json_report() 内存 JSON 字符串
|
||||||
|
```
|
||||||
|
|
||||||
|
`build_review_document()` 仍是建立可信评审对象的唯一入口。机器投影不重新运行 modifier，也不重新实现阶段重放、冲突检查或
|
||||||
|
哈希证明。
|
||||||
|
|
||||||
|
投影函数负责：
|
||||||
|
|
||||||
|
- 检查输入确实是 `ReviewDocument`；
|
||||||
|
- 把已知 enum 转成规定字符串；
|
||||||
|
- 把 tuple 和内部值转成规定 array/object；
|
||||||
|
- 选择当前 detail 允许的字段；
|
||||||
|
- 拒绝不能安全进入标准 JSON 的值；
|
||||||
|
- 返回一个与原对象不共享 dict/list 容器的全新结果。
|
||||||
|
|
||||||
|
投影函数不负责：
|
||||||
|
|
||||||
|
- 修复手工伪造或语义矛盾的 `ReviewDocument`；
|
||||||
|
- 再次应用修改或重新计算阶段；
|
||||||
|
- 对正文脱敏、截断或摘要生成；
|
||||||
|
- 保存或发送结果。
|
||||||
|
|
||||||
|
调用方应把 `build_review_document()` 返回的对象传给投影。手工构造的对象即使恰好通过结构检查，也不获得可信重放保证。
|
||||||
|
|
||||||
|
## 6. 第一版公共接口
|
||||||
|
|
||||||
|
公共名称继续位于 `mdpolish.review`，不在包根 `mdpolish.__init__` 重新导出：
|
||||||
|
|
||||||
|
```python
|
||||||
|
from enum import StrEnum
|
||||||
|
from typing import TypeAlias
|
||||||
|
|
||||||
|
|
||||||
|
JsonScalar: TypeAlias = str | int | float | bool | None
|
||||||
|
JsonValue: TypeAlias = JsonScalar | list["JsonValue"] | dict[str, "JsonValue"]
|
||||||
|
ReviewProjection: TypeAlias = dict[str, JsonValue]
|
||||||
|
|
||||||
|
|
||||||
|
class ReviewDetail(StrEnum):
|
||||||
|
SUMMARY = "summary"
|
||||||
|
CHANGES = "changes"
|
||||||
|
FULL = "full"
|
||||||
|
|
||||||
|
|
||||||
|
class ReviewProjectionError(ValueError):
|
||||||
|
"""ReviewDocument 不能按正式机器契约投影。"""
|
||||||
|
|
||||||
|
|
||||||
|
def review_document_to_dict(
|
||||||
|
review: ReviewDocument,
|
||||||
|
*,
|
||||||
|
detail: ReviewDetail | str = ReviewDetail.SUMMARY,
|
||||||
|
) -> ReviewProjection:
|
||||||
|
...
|
||||||
|
|
||||||
|
|
||||||
|
def render_json_report(
|
||||||
|
review: ReviewDocument,
|
||||||
|
*,
|
||||||
|
detail: ReviewDetail | str = ReviewDetail.SUMMARY,
|
||||||
|
) -> str:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
允许字符串形式是为了让项目配置和 Web 层直接传入 `"summary"`、`"changes"` 或 `"full"`。其他字符串、非字符串且非
|
||||||
|
`ReviewDetail` 的值直接抛出 `ReviewProjectionError`，不能回退到默认值。
|
||||||
|
|
||||||
|
默认 `summary` 是有意的隐私边界：调用方必须显式选择 `changes` 或 `full` 才能得到 Markdown 正文或修改片段。
|
||||||
|
|
||||||
|
返回类型是普通可变 dict/list，因为目标就是 Python 和 JSON 生态通用的数据载体；权威 `ReviewDocument` 仍然不可变。
|
||||||
|
每次调用都返回新的递归容器，修改返回值不能反向改变 review，也不能影响下一次投影。
|
||||||
|
|
||||||
|
## 7. Schema 身份与顶层结构
|
||||||
|
|
||||||
|
第一版 schema 名称和版本固定为：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"schema_name": "mdpolish.review",
|
||||||
|
"schema_version": "1.0"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`schema_version` 是数据格式版本，不是 `mdpolish` 包版本，也不是 modifier 版本。第一版完整顶层结构为：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"schema_name": "mdpolish.review",
|
||||||
|
"schema_version": "1.0",
|
||||||
|
"detail": "summary",
|
||||||
|
"status": "success",
|
||||||
|
"current_kind": "success_output",
|
||||||
|
"stages_complete": true,
|
||||||
|
"hash_contract": {
|
||||||
|
"algorithm": "sha256",
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"normalization": "none"
|
||||||
|
},
|
||||||
|
"coordinate_contract": {
|
||||||
|
"offset_unit": "unicode_code_point",
|
||||||
|
"span_index_base": 0,
|
||||||
|
"span_end": "exclusive",
|
||||||
|
"location_index_base": 1,
|
||||||
|
"physical_line_endings": ["lf", "crlf", "cr"]
|
||||||
|
},
|
||||||
|
"input": {},
|
||||||
|
"current": {},
|
||||||
|
"counts": {},
|
||||||
|
"modifiers": [],
|
||||||
|
"stages": [],
|
||||||
|
"errors": []
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`residual_proposals` 只在 `changes` 和 `full` 中出现；`summary` 通过 `counts.residual_proposal_count` 报告总数。
|
||||||
|
|
||||||
|
所有 object 键必须唯一。上面的成员排列是官方 renderer 的可读输出顺序，但 JSON object 本身无序，消费者不得根据键顺序
|
||||||
|
解释语义。所有 array 顺序都有意义，必须保持 `ReviewDocument` 的权威顺序，不按 ID、哈希或文本重新排序。
|
||||||
|
|
||||||
|
### 7.1 enum 表达
|
||||||
|
|
||||||
|
enum 一律投影为已批准的 `.value` 小写字符串，不输出 Python 类名、`repr()` 或整数序号：
|
||||||
|
|
||||||
|
| Python enum | 第一版允许值 |
|
||||||
|
| --- | --- |
|
||||||
|
| `RunStatus` | `success`、`failed`、`unstable` |
|
||||||
|
| `ReviewCurrentKind` | `success_output`、`partial_output` |
|
||||||
|
| `ErrorStage` | `preflight`、`transform`、`final_review` |
|
||||||
|
| `ReviewDetail` | `summary`、`changes`、`full` |
|
||||||
|
|
||||||
|
投影遇到未知 enum 实例或未知值时失败，不把 `str(value)` 当作向前兼容。
|
||||||
|
|
||||||
|
### 7.2 哈希口径
|
||||||
|
|
||||||
|
所有 `sha256` 字段都是对对应精确 Markdown 字符串执行：
|
||||||
|
|
||||||
|
```python
|
||||||
|
sha256(markdown.encode("utf-8")).hexdigest()
|
||||||
|
```
|
||||||
|
|
||||||
|
结果为 64 位小写十六进制字符串。不规范化 Unicode，不统一 CR/LF/CRLF，不添加或删除 BOM、空白或末尾换行。
|
||||||
|
`Change.before_sha256` / `after_sha256` 和阶段哈希指向完整阶段快照，不是片段哈希。
|
||||||
|
|
||||||
|
### 7.3 位置口径
|
||||||
|
|
||||||
|
- `span.start` / `span.end` 是所属 `ReviewStage.before_markdown` 中的 0-based Unicode code point 半开范围；
|
||||||
|
- `location.line` / `location.column` 指向 `span.start`，是同一阶段修改前全文中的 1-based 人类位置；
|
||||||
|
- LF、CRLF、CR 都形成一个物理换行；CRLF 在 offset 中仍占两个 code point；
|
||||||
|
- 不提供最终全文坐标、UTF-16、byte offset 或显示列宽。
|
||||||
|
|
||||||
|
所有计数、位置和长度必须是 `0..2**53-1` 范围内的 JSON integer；超出时投影失败，避免其他语言使用 binary64 数字时静默
|
||||||
|
丢失整数精度。现实内存文档远小于该上限，因此这不是实际文档规模承诺。
|
||||||
|
|
||||||
|
## 8. 三个正文暴露等级
|
||||||
|
|
||||||
|
三个等级必须满足单调关系：
|
||||||
|
|
||||||
|
```text
|
||||||
|
summary 的字段 ⊂ changes 的字段 ⊂ full 的字段
|
||||||
|
```
|
||||||
|
|
||||||
|
高等级只能增加正文相关字段，不能改变低等级已有字段的值、顺序或语义。
|
||||||
|
|
||||||
|
| 内容 | `summary` | `changes` | `full` |
|
||||||
|
| --- | ---: | ---: | ---: |
|
||||||
|
| 状态、哈希、坐标契约、计数 | 是 | 是 | 是 |
|
||||||
|
| modifier id / version / position | 是 | 是 | 是 |
|
||||||
|
| modifier parameters / applicability | 否 | 是 | 是 |
|
||||||
|
| 阶段前后哈希和修改数 | 是 | 是 | 是 |
|
||||||
|
| 实际修改位置、理由、`before` / `after` | 否 | 是 | 是 |
|
||||||
|
| 稳定错误代码和错误位置 | 是 | 是 | 是 |
|
||||||
|
| Python 诊断类型和错误消息 | 否 | 是 | 是 |
|
||||||
|
| 残留候选理由、范围、`expected_text` / `replacement` | 否 | 是 | 是 |
|
||||||
|
| 完整输入和当前 Markdown | 否 | 否 | 是 |
|
||||||
|
| 每个阶段的完整 before / after Markdown | 否 | 否 | 是 |
|
||||||
|
|
||||||
|
低等级不允许用 `null` 或空字符串代替被隐藏的正文，而是完全省略对应键。这样消费者能够区分“字段因 detail 未暴露”和“原文
|
||||||
|
本来就是空字符串”。`detail` 顶层字段说明当前投影使用的等级。
|
||||||
|
|
||||||
|
`summary` 只承诺不包含以下正文承载字段：`markdown`、`before`、`after`、`expected_text`、`replacement`、`reason`、
|
||||||
|
`message`、`parameters`、`applicability`。它仍含 modifier identity、哈希、位置和计数，不能在不了解项目数据政策的情况下
|
||||||
|
称为匿名、脱敏或可公开日志。
|
||||||
|
|
||||||
|
`changes` 会暴露实际修改和未应用残留候选中的片段，也会暴露项目 modifier 配置和诊断消息。它可能足以还原敏感局部内容。
|
||||||
|
|
||||||
|
`full` 还会重复保存输入、当前全文及每个完整阶段的前后全文，内存和 JSON 大小可能随修改器数量线性增长。调用方必须显式
|
||||||
|
选择，库不截断、不脱敏，也不自动落盘。
|
||||||
|
|
||||||
|
## 9. 各对象的正式投影
|
||||||
|
|
||||||
|
以下字段名称、类型和层级属于 schema `1.0`。示例中的省略号只为文档可读，正式输出不得包含省略号。
|
||||||
|
|
||||||
|
### 9.1 输入和当前文本
|
||||||
|
|
||||||
|
三个 detail 都输出：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"input": {
|
||||||
|
"sha256": "...",
|
||||||
|
"code_point_length": 123
|
||||||
|
},
|
||||||
|
"current": {
|
||||||
|
"sha256": "...",
|
||||||
|
"code_point_length": 120
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`full` 分别增加：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"markdown": "完整文本"
|
||||||
|
```
|
||||||
|
|
||||||
|
`current.markdown` 的性质必须结合根字段 `current_kind` 判断。`partial_output` 永远不能因为进入 JSON 而改名为 cleaned、final
|
||||||
|
或 successful。
|
||||||
|
|
||||||
|
### 9.2 计数
|
||||||
|
|
||||||
|
```json
|
||||||
|
"counts": {
|
||||||
|
"modifier_count": 3,
|
||||||
|
"completed_stage_count": 2,
|
||||||
|
"change_count": 5,
|
||||||
|
"error_count": 1,
|
||||||
|
"residual_proposal_count": 0
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
这些值是投影时从权威 array 计算的派生摘要，必须与 `ReviewDocument` 一致。`change_count` 是所有完整阶段实际
|
||||||
|
`ReviewChange` 的总数，不包含 residual proposal edit；零修改阶段仍计入 `completed_stage_count`。
|
||||||
|
|
||||||
|
### 9.3 Modifier
|
||||||
|
|
||||||
|
所有 detail 都输出所有 modifier 的稳定身份：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"modifiers": [
|
||||||
|
{
|
||||||
|
"position": 0,
|
||||||
|
"modifier_id": "example.normalize",
|
||||||
|
"version": "1.0.0"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
`changes` 和 `full` 增加：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"parameters": [
|
||||||
|
["pattern", " {2,}"],
|
||||||
|
["replacement", " "]
|
||||||
|
],
|
||||||
|
"applicability": "调用方声明的适用范围"
|
||||||
|
```
|
||||||
|
|
||||||
|
`parameters` 不投影成 JSON object。当前内部 `ParameterValue` 会把 mapping 和 sequence 都冻结成 tuple；某些嵌套值在运行时
|
||||||
|
无法可靠区分原来是 mapping 还是二元组 sequence。第一版忠实投影冻结后的结构：顶层及所有 tuple 都变成有序 array，
|
||||||
|
标量保持 `str`、`int`、有限 `float`、`bool` 或 `null`。投影不得根据“看起来像键值对”猜成 object。
|
||||||
|
|
||||||
|
### 9.4 完整阶段
|
||||||
|
|
||||||
|
所有 detail 都输出：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"stages": [
|
||||||
|
{
|
||||||
|
"modifier_position": 0,
|
||||||
|
"before": {
|
||||||
|
"sha256": "...",
|
||||||
|
"code_point_length": 123
|
||||||
|
},
|
||||||
|
"after": {
|
||||||
|
"sha256": "...",
|
||||||
|
"code_point_length": 120
|
||||||
|
},
|
||||||
|
"change_count": 2
|
||||||
|
}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
`changes` 和 `full` 增加 `changes` array；`full` 再给 `before`、`after` 增加 `markdown`。阶段通过
|
||||||
|
`modifier_position` 引用根 `modifiers`，不复制第二份 modifier 身份。
|
||||||
|
|
||||||
|
阶段顺序与 `ReviewDocument.stages` 相同。不得从 `change_count` 猜测阶段是否完整；完整性继续由根字段
|
||||||
|
`stages_complete` 和已有阶段边界表达。
|
||||||
|
|
||||||
|
### 9.5 已应用修改
|
||||||
|
|
||||||
|
只在 `changes` 和 `full` 中出现：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"changes": [
|
||||||
|
{
|
||||||
|
"proposal_index": 0,
|
||||||
|
"edit_index": 0,
|
||||||
|
"reason": "应用调用方声明的替换",
|
||||||
|
"location": {
|
||||||
|
"line": 3,
|
||||||
|
"column": 7
|
||||||
|
},
|
||||||
|
"span": {
|
||||||
|
"start": 24,
|
||||||
|
"end": 31
|
||||||
|
},
|
||||||
|
"before": "exam-\nple",
|
||||||
|
"after": "example",
|
||||||
|
"before_sha256": "...",
|
||||||
|
"after_sha256": "..."
|
||||||
|
}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
修改所在的 modifier 由外层 stage 唯一确定，因此不重复输出 `modifier_id`、版本和位置。`proposal_index` / `edit_index` 保留
|
||||||
|
原权威引用顺序；`before_sha256` / `after_sha256` 是完整阶段快照哈希。
|
||||||
|
|
||||||
|
### 9.6 错误
|
||||||
|
|
||||||
|
所有 detail 都输出稳定身份和位置：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"errors": [
|
||||||
|
{
|
||||||
|
"code": "run.transform_failed",
|
||||||
|
"stage": "transform",
|
||||||
|
"modifier_position": 1,
|
||||||
|
"modifier_id": "example.normalize",
|
||||||
|
"modifier_version": "1.0.0"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
`changes` 和 `full` 增加：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"diagnostic_type": "ModifierContractError",
|
||||||
|
"message": "modifier proposal failed"
|
||||||
|
```
|
||||||
|
|
||||||
|
稳定代码只按已经存在的 `ErrorStage` 映射：
|
||||||
|
|
||||||
|
| `ErrorStage` | `code` |
|
||||||
|
| --- | --- |
|
||||||
|
| `preflight` | `run.preflight_failed` |
|
||||||
|
| `transform` | `run.transform_failed` |
|
||||||
|
| `final_review` | `run.final_review_failed` |
|
||||||
|
|
||||||
|
`diagnostic_type` 是当前 Python 异常类名，`message` 是给人排障的消息；两者都不属于稳定程序分支条件。消费者只能使用
|
||||||
|
`code` 和 `stage` 做稳定判断。
|
||||||
|
|
||||||
|
这些代码有意保持粗粒度。当前 `RunError` 没有保存“元数据变化”“propose 失败”“批次验证失败”等稳定原因，投影不得解析
|
||||||
|
`error_type` 或 `message` 猜出更细代码。未来要增加精细代码，必须先用另一份 design 改变 Pipeline 的错误事实来源。
|
||||||
|
|
||||||
|
### 9.7 残留候选
|
||||||
|
|
||||||
|
`summary` 只输出总数。`changes` 和 `full` 输出完整 residual proposal：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"residual_proposals": [
|
||||||
|
{
|
||||||
|
"modifier_position": 0,
|
||||||
|
"proposal_index": 0,
|
||||||
|
"snapshot_sha256": "...",
|
||||||
|
"reason": "仍可应用的候选",
|
||||||
|
"edits": [
|
||||||
|
{
|
||||||
|
"edit_index": 0,
|
||||||
|
"span": {
|
||||||
|
"start": 10,
|
||||||
|
"end": 15
|
||||||
|
},
|
||||||
|
"expected_text": "exam-",
|
||||||
|
"replacement": "example"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
modifier id 和版本通过 `modifier_position` 引用根 `modifiers`。array 顺序严格保持 `ReviewDocument.residual_proposals` 和
|
||||||
|
`ProposedChange.edits` 的顺序。残留候选仍未应用；投影不能把它放进 stages 或 change count。
|
||||||
|
|
||||||
|
## 10. JSON 基本值和失败关闭
|
||||||
|
|
||||||
|
`review_document_to_dict()` 只能发出：
|
||||||
|
|
||||||
|
```text
|
||||||
|
object / array / string / integer / finite number / boolean / null
|
||||||
|
```
|
||||||
|
|
||||||
|
它不能依赖 `json.dumps(default=...)` 临时处理未知对象。每种公共模型和 enum 都要显式转换；遇到未知类型立即抛出
|
||||||
|
`ReviewProjectionError`。
|
||||||
|
|
||||||
|
投影时至少检查：
|
||||||
|
|
||||||
|
- `review` 是 `ReviewDocument`；
|
||||||
|
- detail 类型和值有效；
|
||||||
|
- enum 是 schema `1.0` 明确支持的成员；
|
||||||
|
- array 中的公共模型类型符合预期；
|
||||||
|
- 字符串可以无损 UTF-8 编码，不含孤立 UTF-16 surrogate；
|
||||||
|
- 整数不是 `bool` 且在安全范围内；
|
||||||
|
- float 有限，不含 NaN 或正负 Infinity；
|
||||||
|
- `modifier_position` 能引用根 `modifiers`；
|
||||||
|
- summary 中没有任何正文承载键；
|
||||||
|
- changes/full 的附加字段符合第 8、9 节。
|
||||||
|
|
||||||
|
这组检查保证序列化结构，不复制 `build_review_document()` 的哈希重放和修改契约。如果调用方绕过 builder 手工构造了
|
||||||
|
语义矛盾但结构合法的 review，投影不声称能恢复可信性。
|
||||||
|
|
||||||
|
`ReviewProjectionError` 的消息只说明字段路径和契约类别，不拼入具体正文、modifier 参数、reason、error message 或周边
|
||||||
|
文本。底层异常可以作为 `__cause__` 保留，但顶层消息不能泄漏被拒绝值。
|
||||||
|
|
||||||
|
## 11. JSON reporter
|
||||||
|
|
||||||
|
`render_json_report()` 必须只做两步：
|
||||||
|
|
||||||
|
1. 调用 `review_document_to_dict(review, detail=detail)`；
|
||||||
|
2. 使用标准库 `json.dumps()` 编码这个返回值。
|
||||||
|
|
||||||
|
固定编码行为：
|
||||||
|
|
||||||
|
```python
|
||||||
|
dumps(
|
||||||
|
projection,
|
||||||
|
ensure_ascii=False,
|
||||||
|
allow_nan=False,
|
||||||
|
indent=2,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
第一版不开放 `indent`、`sort_keys`、encoder、`default` 或文件对象参数，避免把 JSON 编码器的全部表面积变成库契约。需要紧凑
|
||||||
|
JSON 的项目可以对官方 dict 投影自行调用 `json.dumps()`，但不能改变字段语义。
|
||||||
|
|
||||||
|
reporter 返回 Python `str`，不写文件、不添加 UTF-8 BOM，也不在末尾额外添加换行。调用方通过文件、HTTP 或数据库发送时
|
||||||
|
负责按 UTF-8 编码并设置正确媒体类型。
|
||||||
|
|
||||||
|
相同值的 `ReviewDocument` 和相同 detail 必须得到相等 dict 和完全相同的 JSON 字符串。官方实现会使用固定插入顺序方便
|
||||||
|
diff 和测试，但消费者仍不得把 object 键顺序当成语义。
|
||||||
|
|
||||||
|
## 12. Schema 兼容策略
|
||||||
|
|
||||||
|
`schema_version` 使用 `MAJOR.MINOR`：
|
||||||
|
|
||||||
|
- `MAJOR` 改变表示现有消费者可能误读或无法读取；
|
||||||
|
- `MINOR` 只允许旧消费者在忽略未知字段时仍能正确理解的加法变化；
|
||||||
|
- 文案修正、实现重构和使输出重新符合既有契约的 bug fix 不改变 schema 版本；
|
||||||
|
- schema 版本与 Python 包版本分别管理。`mdpolish 0.6.0` 可以继续输出 schema `1.0`。
|
||||||
|
|
||||||
|
下列变化必须提升 schema major：
|
||||||
|
|
||||||
|
- 删除或改名已有字段；
|
||||||
|
- 改变字段类型、坐标、哈希或 array 顺序语义；
|
||||||
|
- 改变已有 enum 或稳定错误代码的含义；
|
||||||
|
- 删除 enum 值，或让生产者在既有字段中自动输出消费者不认识的新 enum 值；
|
||||||
|
- 在相同 detail 下新增正文承载字段，导致原暴露等级泄漏更多内容；
|
||||||
|
- 把可选字段改为必需，或改变字段缺失与空值的区别。
|
||||||
|
|
||||||
|
下列变化可以提升 schema minor：
|
||||||
|
|
||||||
|
- 增加不改变现有字段含义的非正文可选字段；
|
||||||
|
- 增加只有调用方显式请求才会返回的新 detail；
|
||||||
|
- 增加一个新的可选顶层摘要对象，同时保留既有对象。
|
||||||
|
|
||||||
|
同一 major 的消费者必须忽略未知 object 字段，但必须保留 array 顺序；不得接受未知 major。消费者应对自己依赖的 enum 值
|
||||||
|
显式处理未知情况，不能把未知状态当成 `success`。
|
||||||
|
|
||||||
|
正文暴露是安全边界：即使新增字段通常属于 minor，在 `summary` 或 `changes` 中新增更高等级正文也必须升 major，或新增一个
|
||||||
|
需要调用方显式选择的 detail。
|
||||||
|
|
||||||
|
第一版只提供生产，不提供兼容读取器。历史 JSON 的迁移、数据库 schema 和多版本读取由实际跨进程需求触发下一份 design。
|
||||||
|
|
||||||
|
## 13. 源码、文档与版本边界
|
||||||
|
|
||||||
|
批准后计划修改：
|
||||||
|
|
||||||
|
```text
|
||||||
|
src/mdpolish/review.py
|
||||||
|
tests/test_review_projection.py
|
||||||
|
README.md
|
||||||
|
research-wiki/explanation/review-projection.md
|
||||||
|
research-wiki/reference/review-projection-schema-v1.md
|
||||||
|
pyproject.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
- `review.py` 增加第 6 节的公共类型、投影和 JSON reporter，并更新模块 `__all__`；
|
||||||
|
- 独立测试文件固定 schema 和内容暴露边界，避免继续扩大现有 700 行的 `test_review.py`；
|
||||||
|
- explanation 只在实现完成后更新当前机制；
|
||||||
|
- reference 记录代码难以完整表达的 schema `1.0`、兼容和正文暴露契约，不复制 README 的当前进度；
|
||||||
|
- README 在实现完成并验证后才增加能力、示例和真实检查结果；
|
||||||
|
- 不把新接口导出到包根，不新增依赖或源码包目录。
|
||||||
|
|
||||||
|
这是新的公共接口和跨进程数据契约，计划包版本为 `0.5.0`。批准 design 不自动改变当前 `v0.4.0` 事实，也不授权创建 tag、
|
||||||
|
GitHub Release 或发布 wheel。
|
||||||
|
|
||||||
|
当前工作区已有不属于本文的 `AGENTS.md`、`CLAUDE.md` 和 `src/mdpolish/regex.py` 修改。后续实施必须继续保留并隔离这些改动，
|
||||||
|
不能把它们混入本功能的 diff 或提交。
|
||||||
|
|
||||||
|
## 14. 测试与验收
|
||||||
|
|
||||||
|
### 14.1 Schema 和 detail
|
||||||
|
|
||||||
|
合成测试至少覆盖：
|
||||||
|
|
||||||
|
- 空文档、空流水线和零修改结果；
|
||||||
|
- `success`、`failed`、`unstable`，以及完整和不完整 stages；
|
||||||
|
- 三个 detail 的精确顶层键、嵌套键、enum 字符串和 array 顺序；
|
||||||
|
- `summary` 的递归结果中不存在第 8 节列出的任何正文承载键，也找不到专门放入原文、reason、message 和参数的哨兵字符串；
|
||||||
|
- `changes` 包含实际及残留修改片段，但不包含输入、当前和阶段完整 `markdown`；
|
||||||
|
- `full` 包含完整输入、当前文本、阶段全文、修改片段和残留候选；
|
||||||
|
- 空字符串正文通过 `markdown: ""` 与字段未暴露清楚区分；
|
||||||
|
- detail 之间共同字段的值和 array 顺序完全一致；
|
||||||
|
- 返回 dict/list 是新容器，修改一次投影不影响 review 或下一次投影。
|
||||||
|
|
||||||
|
### 14.2 坐标、哈希和参数
|
||||||
|
|
||||||
|
- 中文、补充平面字符、组合字符、BOM、LF、CRLF、CR 和无末尾换行；
|
||||||
|
- span 的 0-based 半开码点范围和 location 的 1-based 码点行列保持现有口径；
|
||||||
|
- 输入、当前、阶段和 change 哈希字段指向正确的精确全文；
|
||||||
|
- tuple、顶层参数对、嵌套二元组、空 tuple、bool、null、int 和有限 float 都按第 9.3 节投影；
|
||||||
|
- mapping 形状的 tuple 不被启发式改成 JSON object；
|
||||||
|
- 计数与权威 array 一致，残留 edit 不计入实际 change count。
|
||||||
|
|
||||||
|
### 14.3 错误与失败关闭
|
||||||
|
|
||||||
|
- 三种 `ErrorStage` 分别得到固定稳定代码；
|
||||||
|
- summary 不含 `diagnostic_type` 和 `message`，changes/full 原样包含；
|
||||||
|
- 未知 detail、错误 review 类型、未知 enum、错误嵌套模型、孤立 surrogate、非有限 float、越界整数和无效引用都失败；
|
||||||
|
- 失败异常为 `ReviewProjectionError`，消息不包含测试正文、参数、reason 或 error message 哨兵；
|
||||||
|
- 投影不调用 modifier、不读文件、不访问网络、不修复非法值、不静默省略错误字段。
|
||||||
|
|
||||||
|
### 14.4 JSON reporter
|
||||||
|
|
||||||
|
- `json.loads(render_json_report(...))` 与同 detail 的官方 dict 投影值相等；
|
||||||
|
- 相同输入重复调用得到逐字符相同的 JSON；
|
||||||
|
- Unicode 正文不被强制写成 `\uXXXX`，控制字符仍由标准 JSON 正确转义；
|
||||||
|
- 输出没有 BOM、没有尾随换行、没有 NaN / Infinity，也不依赖 object 键顺序解释；
|
||||||
|
- summary JSON 中不存在正文哨兵，changes/full 的暴露边界与 dict 完全一致；
|
||||||
|
- reporter 不接受自定义 encoder 或文件对象，不写入磁盘。
|
||||||
|
|
||||||
|
### 14.5 回归和交付检查
|
||||||
|
|
||||||
|
实施完成后实际运行根 README 当时列出的全部检查，并确认：
|
||||||
|
|
||||||
|
- 现有 `Pipeline`、编辑执行器、modifier、`ReviewDocument` 和 Markdown reporter 行为不变；
|
||||||
|
- mypy strict、Ruff 和全部 pytest 通过；
|
||||||
|
- 核心安装仍然没有第三方运行依赖；
|
||||||
|
- wheel 包含更新后的 `review.py` 和 `py.typed`，不包含 tests、Wiki、JSON 报告、真实数据或项目文件；
|
||||||
|
- README 示例只处理内存对象和字符串，不暗示 JSON 已经保存；
|
||||||
|
- `AGENTS.md` 与 `CLAUDE.md` 除标题外正文一致；
|
||||||
|
- Git diff 不混入用户现有改动、真实文本、大文件或生成产物。
|
||||||
|
|
||||||
|
## 15. 风险与代价
|
||||||
|
|
||||||
|
- **公共 schema 需要长期维护：** 内部模型以后可以重构，但 schema `1.x` 不能跟着任意改变；这是正式跨进程接口的必要成本。
|
||||||
|
- **三个 detail 增加测试矩阵：** 每个字段都要证明在哪些等级出现；换来的是正文暴露由调用方显式决定。
|
||||||
|
- **`changes` 仍可能泄漏大量内容：** 多条修改和 residual proposal 能覆盖文档大部分区域；它不是脱敏模式。
|
||||||
|
- **`full` 重复全文：** `ReviewDocument` 已持有阶段快照，投影和 JSON 会再次分配；本轮不做流式或惰性序列化。
|
||||||
|
- **错误代码较粗：** 它只能稳定表达失败阶段，不能区分具体原因；精细化必须先改善 `RunError` 的事实来源。
|
||||||
|
- **参数 tuple 表达不够自然：** array-of-pairs 比 JSON object 更啰嗦，但不会猜错已经丢失的 mapping/sequence 身份。
|
||||||
|
- **没有正式 JSON Schema 文件：** 第一版依靠代码、严格测试和 reference 契约；真正出现独立 validator 需求后再增加发布资产。
|
||||||
|
- **summary 可能被误称为安全日志：** 它只排除正文承载字段，不替代项目的数据分类、访问控制和哈希治理。
|
||||||
|
|
||||||
|
## 16. 批准后的实施边界
|
||||||
|
|
||||||
|
用户明确批准本文后，只授权：
|
||||||
|
|
||||||
|
1. 在 `mdpolish.review` 实现第 6 至 12 节的公共类型、dict 投影和 JSON reporter；
|
||||||
|
2. 新增合成测试并按第 14 节验证，不接触真实文档；
|
||||||
|
3. 更新第 13 节列出的 README、explanation、reference 和包版本；
|
||||||
|
4. 在功能 diff 中隔离并保留工作区已有的其他修改；
|
||||||
|
5. 报告实际测试、wheel 内容和 Git diff，不把设计批准描述成已经发布。
|
||||||
|
|
||||||
|
批准本文不授权：
|
||||||
|
|
||||||
|
- 提交、push、创建 PR、tag、GitHub Release 或上传 wheel；
|
||||||
|
- 修改 `Pipeline`、清洗规则、错误捕获语义、`ReviewDocument` 字段或其他仓库；
|
||||||
|
- 创建 CLI、文件适配器、Web 服务、数据库表、JSON Schema 发布资产或反序列化器；
|
||||||
|
- 读取、复制、修改或公开真实文档和外部数据。
|
||||||
@@ -16,10 +16,11 @@
|
|||||||
│ 验证并重放，不运行 Modifier
|
│ 验证并重放，不运行 Modifier
|
||||||
▼
|
▼
|
||||||
ReviewDocument
|
ReviewDocument
|
||||||
│ │
|
├── render_markdown_report() ─────────────► 内存 Markdown 字符串
|
||||||
│ └── render_markdown_report() ──► 内存 Markdown 字符串
|
│
|
||||||
▼
|
└── review_document_to_dict(detail=...)
|
||||||
项目自己的界面或转换层
|
├───────────────────────────► 项目自己的界面或转换层
|
||||||
|
└── render_json_report() ──► 内存 JSON 字符串
|
||||||
```
|
```
|
||||||
|
|
||||||
文件读取、保存位置、HTML 页面、权限和审核流程仍由调用项目决定。
|
文件读取、保存位置、HTML 页面、权限和审核流程仍由调用项目决定。
|
||||||
@@ -85,3 +86,30 @@ span 和修改前后长度，不重复输出 `expected_text`、`replacement` 或
|
|||||||
报告有意包含完整文档、实际修改的 `before` / `after` 和 diff，可能还原敏感内容。库不会自动打印、保存、上传或缓存
|
报告有意包含完整文档、实际修改的 `before` / `after` 和 diff，可能还原敏感内容。库不会自动打印、保存、上传或缓存
|
||||||
报告；持久化后的路径、访问权限、脱敏和保留周期属于调用项目。每个阶段保存完整前后快照，当前实现优先保证可复核性，
|
报告；持久化后的路径、访问权限、脱敏和保留周期属于调用项目。每个阶段保存完整前后快照，当前实现优先保证可复核性，
|
||||||
没有声称适合无限长度文档或无限修改器链。
|
没有声称适合无限长度文档或无限修改器链。
|
||||||
|
|
||||||
|
## 6. 机器投影为什么不是 `dataclasses.asdict()`
|
||||||
|
|
||||||
|
`review_document_to_dict()` 是 `ReviewDocument` 的单向公共视图，不是内部 dataclass 的机械展开。它显式转换 enum、tuple、
|
||||||
|
modifier 参数、位置和哈希，并在根对象写入 `schema_name=mdpolish.review` 与独立的 `schema_version=1.0`。这样内部 Python
|
||||||
|
结构可以在不改变 schema 的前提下重构，调用项目也不需要猜测 enum、数组和阶段坐标。
|
||||||
|
|
||||||
|
投影有三个内容等级：
|
||||||
|
|
||||||
|
| detail | 增加的内容 |
|
||||||
|
| --- | --- |
|
||||||
|
| `summary` | 状态、哈希、计数、modifier 身份、阶段摘要和稳定错误代码；默认不含正文承载字段 |
|
||||||
|
| `changes` | modifier 参数和适用说明、实际修改片段、错误诊断、残留候选片段 |
|
||||||
|
| `full` | 完整输入、当前全文和每个完整阶段的前后全文 |
|
||||||
|
|
||||||
|
高等级只增加字段，不改变低等级已有字段的值和顺序。被 detail 隐藏的字段直接不存在，不使用 `null` 或空字符串假装隐藏，
|
||||||
|
因此空文档在 `full` 中仍能明确表示为 `markdown: ""`。
|
||||||
|
|
||||||
|
`summary` 不含正文，但仍包含 modifier identity、哈希和计数，只能称为“无正文投影”，不能称为脱敏或公开安全日志。
|
||||||
|
`changes` 和 `full` 都可能还原敏感内容；库不会自动打印、保存或发送任何投影。
|
||||||
|
|
||||||
|
`render_json_report()` 只把同 detail 的正式 dict 投影用标准 JSON 编码，不维护第二套字段。它返回内存字符串，使用 Unicode、
|
||||||
|
拒绝 NaN / Infinity，不写文件或添加 BOM。Markdown reporter 继续直接读取 `ReviewDocument`：它面向人类排版并包含 diff、
|
||||||
|
动态围栏和 residual 展示限额，不依赖机器 schema。
|
||||||
|
|
||||||
|
机器投影只生产，不提供 JSON 到 `ReviewDocument` 的反序列化，也不能用于重新应用修改。完整字段、坐标、错误代码和兼容规则
|
||||||
|
见 [`review-projection-schema-v1.md`](../reference/review-projection-schema-v1.md)。
|
||||||
|
|||||||
@@ -0,0 +1,250 @@
|
|||||||
|
# ReviewDocument 机器投影 schema 1.0
|
||||||
|
|
||||||
|
本文记录 `mdpolish.review.review_document_to_dict()` 和 `render_json_report()` 当前稳定的跨进程查询口径。公共 Python 类型和
|
||||||
|
运行校验以 `src/mdpolish/review.py` 与测试为准；设计理由和批准边界见
|
||||||
|
[`0012-review-document-machine-projection.md`](../design/0012-review-document-machine-projection.md)。
|
||||||
|
|
||||||
|
## 1. Schema 身份与入口
|
||||||
|
|
||||||
|
每个投影都包含：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"schema_name": "mdpolish.review",
|
||||||
|
"schema_version": "1.0"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
schema 版本独立于 `mdpolish` 包版本和 modifier 版本。公共入口是：
|
||||||
|
|
||||||
|
```python
|
||||||
|
from mdpolish.review import render_json_report, review_document_to_dict
|
||||||
|
|
||||||
|
payload = review_document_to_dict(review, detail="summary")
|
||||||
|
json_text = render_json_report(review, detail="full")
|
||||||
|
```
|
||||||
|
|
||||||
|
两者只接受内存中的 `ReviewDocument`。JSON reporter 编码同 detail 的正式 dict，不定义另一套字段，也不读写文件。
|
||||||
|
|
||||||
|
## 2. 顶层字段
|
||||||
|
|
||||||
|
| 字段 | JSON 类型 | 口径 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `schema_name` | string | 固定为 `mdpolish.review` |
|
||||||
|
| `schema_version` | string | 当前固定为 `1.0` |
|
||||||
|
| `detail` | string | `summary`、`changes`、`full` |
|
||||||
|
| `status` | string | `success`、`failed`、`unstable` |
|
||||||
|
| `current_kind` | string | `success_output`、`partial_output` |
|
||||||
|
| `stages_complete` | boolean | 是否能证明全部 modifier transform 阶段完成 |
|
||||||
|
| `hash_contract` | object | 所有 Markdown 哈希的算法、编码和规范化口径 |
|
||||||
|
| `coordinate_contract` | object | span、行列和物理换行口径 |
|
||||||
|
| `input` | object | 输入文本摘要；`full` 增加正文 |
|
||||||
|
| `current` | object | 当前文本摘要；`full` 增加正文 |
|
||||||
|
| `counts` | object | modifier、完成阶段、实际修改、错误和残留候选数量 |
|
||||||
|
| `modifiers` | array | 全部 modifier 的权威顺序 |
|
||||||
|
| `stages` | array | 已完成 transform 阶段的权威顺序 |
|
||||||
|
| `errors` | array | Pipeline 记录的错误顺序 |
|
||||||
|
| `residual_proposals` | array | 只在 `changes` / `full` 出现；候选未应用 |
|
||||||
|
|
||||||
|
JSON object 成员顺序不构成语义。所有 array 顺序构成语义，消费者不得重新按 ID、哈希或文本排序后再解释位置。
|
||||||
|
|
||||||
|
## 3. 内容暴露等级
|
||||||
|
|
||||||
|
| 内容 | `summary` | `changes` | `full` |
|
||||||
|
| --- | ---: | ---: | ---: |
|
||||||
|
| 状态、哈希、位置口径和计数 | 是 | 是 | 是 |
|
||||||
|
| modifier id、版本和位置 | 是 | 是 | 是 |
|
||||||
|
| modifier 参数和 applicability | 否 | 是 | 是 |
|
||||||
|
| 阶段前后哈希、长度和修改数 | 是 | 是 | 是 |
|
||||||
|
| 实际修改的 reason、位置、`before` / `after` | 否 | 是 | 是 |
|
||||||
|
| 稳定错误代码和错误位置 | 是 | 是 | 是 |
|
||||||
|
| Python 诊断类型和错误消息 | 否 | 是 | 是 |
|
||||||
|
| 残留候选的 reason、`expected_text` / `replacement` | 否 | 是 | 是 |
|
||||||
|
| 输入、当前和阶段完整 Markdown | 否 | 否 | 是 |
|
||||||
|
|
||||||
|
默认是 `summary`。被隐藏的内容键不存在；`null` 和空字符串都不是“已隐藏”的替代值。`summary` 不含直接正文，但仍含
|
||||||
|
modifier identity、哈希和计数，不自动等于匿名、脱敏或适合公开传播。
|
||||||
|
|
||||||
|
## 4. 固定口径
|
||||||
|
|
||||||
|
### 4.1 哈希
|
||||||
|
|
||||||
|
```json
|
||||||
|
"hash_contract": {
|
||||||
|
"algorithm": "sha256",
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"normalization": "none"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
哈希是精确 Markdown 的 `sha256(markdown.encode("utf-8")).hexdigest()`。不规范化 Unicode、BOM、空白、末尾换行或
|
||||||
|
CR/LF/CRLF。`Change.before_sha256` / `after_sha256` 是完整阶段快照哈希，不是修改片段哈希。
|
||||||
|
|
||||||
|
### 4.2 坐标
|
||||||
|
|
||||||
|
```json
|
||||||
|
"coordinate_contract": {
|
||||||
|
"offset_unit": "unicode_code_point",
|
||||||
|
"span_index_base": 0,
|
||||||
|
"span_end": "exclusive",
|
||||||
|
"location_index_base": 1,
|
||||||
|
"physical_line_endings": ["lf", "crlf", "cr"]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- `span` 相对于所属 `ReviewStage.before_markdown`，使用 0-based Unicode code point 半开范围；
|
||||||
|
- `location.line` / `column` 指向 `span.start`，使用同一阶段修改前文本中的 1-based Unicode code point 位置；
|
||||||
|
- CRLF 形成一个物理换行，但在 offset 中占两个 code point；
|
||||||
|
- 不提供最终全文坐标、UTF-16、byte offset 或终端显示宽度。
|
||||||
|
|
||||||
|
所有整数都在 `0..2**53-1`；参数中的有符号整数在 `-(2**53-1)..2**53-1`。所有 float 必须有限。
|
||||||
|
|
||||||
|
## 5. 嵌套对象
|
||||||
|
|
||||||
|
### 5.1 文本摘要
|
||||||
|
|
||||||
|
`input`、`current` 以及 stage 的 `before` / `after` 都至少包含：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"sha256": "64 位小写十六进制",
|
||||||
|
"code_point_length": 123
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`full` 增加 `markdown`。`current.markdown` 是否为正式输出必须看根字段 `current_kind`；`partial_output` 不能当作成功结果。
|
||||||
|
|
||||||
|
### 5.2 Counts
|
||||||
|
|
||||||
|
```json
|
||||||
|
"counts": {
|
||||||
|
"modifier_count": 3,
|
||||||
|
"completed_stage_count": 2,
|
||||||
|
"change_count": 5,
|
||||||
|
"error_count": 1,
|
||||||
|
"residual_proposal_count": 0
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`change_count` 只统计已完成 stage 中的实际 `ReviewChange`，不含 residual proposal edit。零修改的完整 stage 仍计入
|
||||||
|
`completed_stage_count`。
|
||||||
|
|
||||||
|
### 5.3 Modifier
|
||||||
|
|
||||||
|
所有 detail：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"position": 0,
|
||||||
|
"modifier_id": "example.normalize",
|
||||||
|
"version": "1.0.0"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`changes` / `full` 增加 `parameters` 和 `applicability`。`parameters` 是有序 array-of-pairs；所有内部 tuple 继续投影成
|
||||||
|
array，不根据二元组外形猜成 JSON object：
|
||||||
|
|
||||||
|
```json
|
||||||
|
"parameters": [
|
||||||
|
["pattern", " {2,}"],
|
||||||
|
["replacement", " "]
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.4 Stage
|
||||||
|
|
||||||
|
所有 detail：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"modifier_position": 0,
|
||||||
|
"before": {"sha256": "...", "code_point_length": 20},
|
||||||
|
"after": {"sha256": "...", "code_point_length": 18},
|
||||||
|
"change_count": 1
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`changes` / `full` 增加 `changes` array；`full` 还给两个文本摘要增加 `markdown`。`modifier_position` 引用根
|
||||||
|
`modifiers[position]`，不复制 modifier 元数据。
|
||||||
|
|
||||||
|
### 5.5 实际 Change
|
||||||
|
|
||||||
|
只在 `changes` / `full` 出现：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"proposal_index": 0,
|
||||||
|
"edit_index": 0,
|
||||||
|
"reason": "修改原因",
|
||||||
|
"location": {"line": 3, "column": 7},
|
||||||
|
"span": {"start": 24, "end": 31},
|
||||||
|
"before": "原片段",
|
||||||
|
"after": "新片段",
|
||||||
|
"before_sha256": "修改前完整阶段哈希",
|
||||||
|
"after_sha256": "修改后完整阶段哈希"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.6 Error
|
||||||
|
|
||||||
|
所有 detail 都含 `code`、`stage`、modifier 位置与身份。`changes` / `full` 再增加诊断用的 `diagnostic_type` 和
|
||||||
|
`message`。
|
||||||
|
|
||||||
|
| `stage` | 稳定 `code` |
|
||||||
|
| --- | --- |
|
||||||
|
| `preflight` | `run.preflight_failed` |
|
||||||
|
| `transform` | `run.transform_failed` |
|
||||||
|
| `final_review` | `run.final_review_failed` |
|
||||||
|
|
||||||
|
代码只稳定表达失败阶段。`diagnostic_type` 是 Python 异常类名，`message` 是人类消息，不能作为稳定程序分支条件。当前
|
||||||
|
`RunError` 没有保存更细的稳定原因；不能从类名或消息猜造更细代码。
|
||||||
|
|
||||||
|
### 5.7 Residual proposal
|
||||||
|
|
||||||
|
只在 `changes` / `full` 出现：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"modifier_position": 0,
|
||||||
|
"proposal_index": 0,
|
||||||
|
"snapshot_sha256": "当前完整快照哈希",
|
||||||
|
"reason": "候选原因",
|
||||||
|
"edits": [
|
||||||
|
{
|
||||||
|
"edit_index": 0,
|
||||||
|
"span": {"start": 10, "end": 15},
|
||||||
|
"expected_text": "原片段",
|
||||||
|
"replacement": "候选片段"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
它是 final review 证据，没有应用，不进入 stage 或实际 change count。
|
||||||
|
|
||||||
|
## 6. JSON 编码
|
||||||
|
|
||||||
|
`render_json_report()` 固定使用标准库 JSON 的以下语义：
|
||||||
|
|
||||||
|
- `ensure_ascii=False`；
|
||||||
|
- `allow_nan=False`；
|
||||||
|
- `indent=2`；
|
||||||
|
- 无 UTF-8 BOM；
|
||||||
|
- 返回字符串末尾不额外添加换行。
|
||||||
|
|
||||||
|
返回值是 Python `str`。调用方保存或发送时负责 UTF-8 编码、媒体类型、权限和保留周期。库不接收路径或文件对象。
|
||||||
|
|
||||||
|
## 7. 兼容策略
|
||||||
|
|
||||||
|
`schema_version` 使用 `MAJOR.MINOR`：
|
||||||
|
|
||||||
|
- 删除、改名、改类型、改变位置/哈希/顺序语义、改变已有 enum 或错误代码含义，需要提升 major；
|
||||||
|
- 在相同 detail 中新增更高敏感度的正文承载字段，需要提升 major，或增加必须显式请求的新 detail；
|
||||||
|
- 不改变旧字段解释的非正文可选字段，可以提升 minor；
|
||||||
|
- 实现修正为重新符合已有契约，不改变 schema 版本；
|
||||||
|
- schema 版本不跟随 Python 包版本自动变化。
|
||||||
|
|
||||||
|
同一 major 的消费者必须忽略未知 object 字段，但必须保持 array 顺序；不得把未知状态当成 `success`。消费者应拒绝自己不
|
||||||
|
支持的 schema major。
|
||||||
|
|
||||||
|
schema `1.0` 是单向生产契约。当前没有官方反序列化器、JSON Schema 文件、历史迁移器或数据库 schema。
|
||||||
+554
-2
@@ -1,4 +1,4 @@
|
|||||||
"""可信解释 TransformResult 的内存评审视图与 Markdown 报告。"""
|
"""可信解释 TransformResult 的评审视图、机器投影与 Markdown 报告。"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -7,7 +7,8 @@ from difflib import unified_diff
|
|||||||
from enum import StrEnum
|
from enum import StrEnum
|
||||||
from itertools import pairwise
|
from itertools import pairwise
|
||||||
from json import dumps
|
from json import dumps
|
||||||
from typing import Never
|
from math import isfinite
|
||||||
|
from typing import Never, TypeAlias
|
||||||
|
|
||||||
from mdpolish.edits import _apply_validated_edits, _ordered_indexed_edits, validate_modifier_batch
|
from mdpolish.edits import _apply_validated_edits, _ordered_indexed_edits, validate_modifier_batch
|
||||||
from mdpolish.models import (
|
from mdpolish.models import (
|
||||||
@@ -15,11 +16,13 @@ from mdpolish.models import (
|
|||||||
DocumentSnapshot,
|
DocumentSnapshot,
|
||||||
ErrorStage,
|
ErrorStage,
|
||||||
ModifierInfo,
|
ModifierInfo,
|
||||||
|
ProposalReference,
|
||||||
ProposedChange,
|
ProposedChange,
|
||||||
ResidualProposal,
|
ResidualProposal,
|
||||||
RunError,
|
RunError,
|
||||||
RunStatus,
|
RunStatus,
|
||||||
TextEdit,
|
TextEdit,
|
||||||
|
TextSpan,
|
||||||
TransformResult,
|
TransformResult,
|
||||||
markdown_sha256,
|
markdown_sha256,
|
||||||
)
|
)
|
||||||
@@ -29,6 +32,10 @@ class ReviewBuildError(ValueError):
|
|||||||
"""TransformResult 不能被安全地解释为评审视图。"""
|
"""TransformResult 不能被安全地解释为评审视图。"""
|
||||||
|
|
||||||
|
|
||||||
|
class ReviewProjectionError(ValueError):
|
||||||
|
"""ReviewDocument 不能按正式机器契约投影。"""
|
||||||
|
|
||||||
|
|
||||||
class ReviewCurrentKind(StrEnum):
|
class ReviewCurrentKind(StrEnum):
|
||||||
"""评审视图中当前全文的结果性质。"""
|
"""评审视图中当前全文的结果性质。"""
|
||||||
|
|
||||||
@@ -36,6 +43,19 @@ class ReviewCurrentKind(StrEnum):
|
|||||||
PARTIAL_OUTPUT = "partial_output"
|
PARTIAL_OUTPUT = "partial_output"
|
||||||
|
|
||||||
|
|
||||||
|
class ReviewDetail(StrEnum):
|
||||||
|
"""机器投影允许暴露的正文详细程度。"""
|
||||||
|
|
||||||
|
SUMMARY = "summary"
|
||||||
|
CHANGES = "changes"
|
||||||
|
FULL = "full"
|
||||||
|
|
||||||
|
|
||||||
|
JsonScalar: TypeAlias = str | int | float | bool | None
|
||||||
|
JsonValue: TypeAlias = JsonScalar | list["JsonValue"] | dict[str, "JsonValue"]
|
||||||
|
ReviewProjection: TypeAlias = dict[str, JsonValue]
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
@dataclass(frozen=True, slots=True)
|
||||||
class ReviewLocation:
|
class ReviewLocation:
|
||||||
"""修改前快照中的 1-based Python 码点位置。"""
|
"""修改前快照中的 1-based Python 码点位置。"""
|
||||||
@@ -485,6 +505,531 @@ def build_review_document(input_markdown: str, result: TransformResult) -> Revie
|
|||||||
raise ReviewBuildError("review build failed: result structure is malformed") from None
|
raise ReviewBuildError("review build failed: result structure is malformed") from None
|
||||||
|
|
||||||
|
|
||||||
|
_SCHEMA_NAME = "mdpolish.review"
|
||||||
|
_SCHEMA_VERSION = "1.0"
|
||||||
|
_MAX_SAFE_JSON_INTEGER = 2**53 - 1
|
||||||
|
_BODY_FIELD_NAMES = frozenset(
|
||||||
|
{
|
||||||
|
"applicability",
|
||||||
|
"expected_text",
|
||||||
|
"markdown",
|
||||||
|
"message",
|
||||||
|
"parameters",
|
||||||
|
"reason",
|
||||||
|
"replacement",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
_ERROR_CODES = {
|
||||||
|
ErrorStage.PREFLIGHT: "run.preflight_failed",
|
||||||
|
ErrorStage.TRANSFORM: "run.transform_failed",
|
||||||
|
ErrorStage.FINAL_REVIEW: "run.final_review_failed",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_fail(path: str, contract: str) -> Never:
|
||||||
|
raise ReviewProjectionError(f"review projection failed at {path}: {contract}")
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_string(value: object, path: str) -> str:
|
||||||
|
if not isinstance(value, str):
|
||||||
|
_projection_fail(path, "value must be a string")
|
||||||
|
try:
|
||||||
|
encoded = value.encode("utf-8")
|
||||||
|
except UnicodeEncodeError:
|
||||||
|
_projection_fail(path, "string must be valid Unicode encodable as UTF-8")
|
||||||
|
return encoded.decode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_nonempty_string(value: object, path: str) -> str:
|
||||||
|
text = _projection_string(value, path)
|
||||||
|
if not text:
|
||||||
|
_projection_fail(path, "string must not be empty")
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_integer(value: object, path: str, *, minimum: int = 0) -> int:
|
||||||
|
if type(value) is not int:
|
||||||
|
_projection_fail(path, "value must be an integer")
|
||||||
|
if value < minimum or value > _MAX_SAFE_JSON_INTEGER:
|
||||||
|
_projection_fail(path, "integer is outside the interoperable JSON range")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_signed_integer(value: object, path: str) -> int:
|
||||||
|
if type(value) is not int:
|
||||||
|
_projection_fail(path, "value must be an integer")
|
||||||
|
if value < -_MAX_SAFE_JSON_INTEGER or value > _MAX_SAFE_JSON_INTEGER:
|
||||||
|
_projection_fail(path, "integer is outside the interoperable JSON range")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_sha256(value: object, path: str) -> str:
|
||||||
|
digest = _projection_string(value, path)
|
||||||
|
if len(digest) != 64 or any(character not in "0123456789abcdef" for character in digest):
|
||||||
|
_projection_fail(path, "value must be a lowercase SHA-256 digest")
|
||||||
|
return digest
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_detail(value: object) -> ReviewDetail:
|
||||||
|
if not isinstance(value, (ReviewDetail, str)):
|
||||||
|
_projection_fail("detail", "value must be summary, changes, or full")
|
||||||
|
try:
|
||||||
|
return ReviewDetail(value)
|
||||||
|
except ValueError:
|
||||||
|
_projection_fail("detail", "value must be summary, changes, or full")
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_parameter(value: object, path: str) -> JsonValue:
|
||||||
|
if value is None or isinstance(value, bool):
|
||||||
|
return value
|
||||||
|
if isinstance(value, str):
|
||||||
|
return _projection_string(value, path)
|
||||||
|
if type(value) is int:
|
||||||
|
return _projection_signed_integer(value, path)
|
||||||
|
if isinstance(value, float):
|
||||||
|
if not isfinite(value):
|
||||||
|
_projection_fail(path, "float must be finite")
|
||||||
|
return float(value)
|
||||||
|
if isinstance(value, tuple):
|
||||||
|
return [_projection_parameter(item, f"{path}[{index}]") for index, item in enumerate(value)]
|
||||||
|
_projection_fail(path, "parameter contains an unsupported value")
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_parameters(value: object, path: str) -> list[JsonValue]:
|
||||||
|
if not isinstance(value, tuple):
|
||||||
|
_projection_fail(path, "parameters must be a tuple")
|
||||||
|
projected: list[JsonValue] = []
|
||||||
|
seen_keys: set[str] = set()
|
||||||
|
for index, pair in enumerate(value):
|
||||||
|
pair_path = f"{path}[{index}]"
|
||||||
|
if not isinstance(pair, tuple) or len(pair) != 2:
|
||||||
|
_projection_fail(pair_path, "parameter entry must be a key-value tuple")
|
||||||
|
key = _projection_nonempty_string(pair[0], f"{pair_path}[0]")
|
||||||
|
if key in seen_keys:
|
||||||
|
_projection_fail(pair_path, "parameter keys must be unique")
|
||||||
|
seen_keys.add(key)
|
||||||
|
projected.append([key, _projection_parameter(pair[1], f"{pair_path}[1]")])
|
||||||
|
return projected
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_text(
|
||||||
|
markdown: object,
|
||||||
|
digest: object,
|
||||||
|
*,
|
||||||
|
include_markdown: bool,
|
||||||
|
path: str,
|
||||||
|
) -> ReviewProjection:
|
||||||
|
text = _projection_string(markdown, f"{path}.markdown")
|
||||||
|
projection: ReviewProjection = {
|
||||||
|
"sha256": _projection_sha256(digest, f"{path}.sha256"),
|
||||||
|
"code_point_length": _projection_integer(len(text), f"{path}.code_point_length"),
|
||||||
|
}
|
||||||
|
if include_markdown:
|
||||||
|
projection["markdown"] = text
|
||||||
|
return projection
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_modifier(value: object, position: int, detail: ReviewDetail) -> ReviewProjection:
|
||||||
|
path = f"modifiers[{position}]"
|
||||||
|
if not isinstance(value, ModifierInfo):
|
||||||
|
_projection_fail(path, "value must be ModifierInfo")
|
||||||
|
modifier_id = _projection_nonempty_string(value.modifier_id, f"{path}.modifier_id")
|
||||||
|
version = _projection_nonempty_string(value.version, f"{path}.version")
|
||||||
|
parameters = _projection_parameters(value.parameters, f"{path}.parameters")
|
||||||
|
applicability = _projection_nonempty_string(value.applicability, f"{path}.applicability")
|
||||||
|
projection: ReviewProjection = {
|
||||||
|
"position": _projection_integer(position, f"{path}.position"),
|
||||||
|
"modifier_id": modifier_id,
|
||||||
|
"version": version,
|
||||||
|
}
|
||||||
|
if detail is not ReviewDetail.SUMMARY:
|
||||||
|
projection["parameters"] = parameters
|
||||||
|
projection["applicability"] = applicability
|
||||||
|
return projection
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_span(value: object, path: str) -> ReviewProjection:
|
||||||
|
if not isinstance(value, TextSpan):
|
||||||
|
_projection_fail(path, "value must be TextSpan")
|
||||||
|
start = _projection_integer(value.start, f"{path}.start")
|
||||||
|
end = _projection_integer(value.end, f"{path}.end")
|
||||||
|
if end < start:
|
||||||
|
_projection_fail(path, "span end must not precede start")
|
||||||
|
return {"start": start, "end": end}
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_location(value: object, path: str) -> ReviewProjection:
|
||||||
|
if not isinstance(value, ReviewLocation):
|
||||||
|
_projection_fail(path, "value must be ReviewLocation")
|
||||||
|
return {
|
||||||
|
"line": _projection_integer(value.line, f"{path}.line", minimum=1),
|
||||||
|
"column": _projection_integer(value.column, f"{path}.column", minimum=1),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_change(
|
||||||
|
value: object,
|
||||||
|
*,
|
||||||
|
stage: ReviewStage,
|
||||||
|
stage_modifier: ModifierInfo,
|
||||||
|
path: str,
|
||||||
|
) -> ReviewProjection:
|
||||||
|
if not isinstance(value, ReviewChange):
|
||||||
|
_projection_fail(path, "value must be ReviewChange")
|
||||||
|
change = value.change
|
||||||
|
if not isinstance(change, Change):
|
||||||
|
_projection_fail(f"{path}.change", "value must be Change")
|
||||||
|
if (
|
||||||
|
change.modifier_position != stage.modifier_position
|
||||||
|
or change.modifier_id != stage_modifier.modifier_id
|
||||||
|
or change.modifier_version != stage_modifier.version
|
||||||
|
):
|
||||||
|
_projection_fail(path, "change identity does not match its stage")
|
||||||
|
|
||||||
|
proposal_ref = change.proposal_ref
|
||||||
|
if not isinstance(proposal_ref, ProposalReference):
|
||||||
|
_projection_fail(f"{path}.proposal_ref", "value must be ProposalReference")
|
||||||
|
proposal_index = _projection_integer(
|
||||||
|
proposal_ref.proposal_index,
|
||||||
|
f"{path}.proposal_ref.proposal_index",
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
proposal_ref.modifier_position != stage.modifier_position
|
||||||
|
or proposal_ref.snapshot_sha256 != stage.before_sha256
|
||||||
|
):
|
||||||
|
_projection_fail(f"{path}.proposal_ref", "reference does not match its stage")
|
||||||
|
|
||||||
|
edit_index = _projection_integer(change.edit_index, f"{path}.edit_index")
|
||||||
|
reason = _projection_nonempty_string(change.reason, f"{path}.reason")
|
||||||
|
span = _projection_span(change.span, f"{path}.span")
|
||||||
|
before = _projection_string(change.before, f"{path}.before")
|
||||||
|
after = _projection_string(change.after, f"{path}.after")
|
||||||
|
if len(before) != change.span.end - change.span.start or before == after:
|
||||||
|
_projection_fail(path, "change text does not satisfy its span contract")
|
||||||
|
before_sha256 = _projection_sha256(change.before_sha256, f"{path}.before_sha256")
|
||||||
|
after_sha256 = _projection_sha256(change.after_sha256, f"{path}.after_sha256")
|
||||||
|
if before_sha256 != stage.before_sha256 or after_sha256 != stage.after_sha256:
|
||||||
|
_projection_fail(path, "change hashes do not match its stage")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"proposal_index": proposal_index,
|
||||||
|
"edit_index": edit_index,
|
||||||
|
"reason": reason,
|
||||||
|
"location": _projection_location(value.location, f"{path}.location"),
|
||||||
|
"span": span,
|
||||||
|
"before": before,
|
||||||
|
"after": after,
|
||||||
|
"before_sha256": before_sha256,
|
||||||
|
"after_sha256": after_sha256,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_stage(
|
||||||
|
value: object,
|
||||||
|
*,
|
||||||
|
expected_position: int,
|
||||||
|
modifiers: tuple[ModifierInfo, ...],
|
||||||
|
detail: ReviewDetail,
|
||||||
|
) -> tuple[ReviewProjection, int]:
|
||||||
|
path = f"stages[{expected_position}]"
|
||||||
|
if not isinstance(value, ReviewStage):
|
||||||
|
_projection_fail(path, "value must be ReviewStage")
|
||||||
|
position = _projection_integer(value.modifier_position, f"{path}.modifier_position")
|
||||||
|
if position != expected_position or position >= len(modifiers):
|
||||||
|
_projection_fail(path, "stage position does not reference the expected modifier")
|
||||||
|
modifier = modifiers[position]
|
||||||
|
if value.modifier != modifier:
|
||||||
|
_projection_fail(path, "stage modifier does not match root modifier metadata")
|
||||||
|
if not isinstance(value.changes, tuple):
|
||||||
|
_projection_fail(f"{path}.changes", "changes must be a tuple")
|
||||||
|
|
||||||
|
before = _projection_text(
|
||||||
|
value.before_markdown,
|
||||||
|
value.before_sha256,
|
||||||
|
include_markdown=detail is ReviewDetail.FULL,
|
||||||
|
path=f"{path}.before",
|
||||||
|
)
|
||||||
|
after = _projection_text(
|
||||||
|
value.after_markdown,
|
||||||
|
value.after_sha256,
|
||||||
|
include_markdown=detail is ReviewDetail.FULL,
|
||||||
|
path=f"{path}.after",
|
||||||
|
)
|
||||||
|
projected_changes: list[JsonValue] = [
|
||||||
|
_projection_change(
|
||||||
|
review_change,
|
||||||
|
stage=value,
|
||||||
|
stage_modifier=modifier,
|
||||||
|
path=f"{path}.changes[{index}]",
|
||||||
|
)
|
||||||
|
for index, review_change in enumerate(value.changes)
|
||||||
|
]
|
||||||
|
projection: ReviewProjection = {
|
||||||
|
"modifier_position": position,
|
||||||
|
"before": before,
|
||||||
|
"after": after,
|
||||||
|
"change_count": _projection_integer(len(value.changes), f"{path}.change_count"),
|
||||||
|
}
|
||||||
|
if detail is not ReviewDetail.SUMMARY:
|
||||||
|
projection["changes"] = projected_changes
|
||||||
|
return projection, len(value.changes)
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_error(
|
||||||
|
value: object,
|
||||||
|
*,
|
||||||
|
error_index: int,
|
||||||
|
modifiers: tuple[ModifierInfo, ...],
|
||||||
|
detail: ReviewDetail,
|
||||||
|
) -> ReviewProjection:
|
||||||
|
path = f"errors[{error_index}]"
|
||||||
|
if not isinstance(value, RunError):
|
||||||
|
_projection_fail(path, "value must be RunError")
|
||||||
|
if not isinstance(value.stage, ErrorStage) or value.stage not in _ERROR_CODES:
|
||||||
|
_projection_fail(f"{path}.stage", "error stage is not supported by schema 1.0")
|
||||||
|
position = _projection_integer(value.modifier_position, f"{path}.modifier_position")
|
||||||
|
modifier_id = _projection_nonempty_string(value.modifier_id, f"{path}.modifier_id")
|
||||||
|
modifier_version = _projection_nonempty_string(value.modifier_version, f"{path}.modifier_version")
|
||||||
|
if position < len(modifiers):
|
||||||
|
modifier = modifiers[position]
|
||||||
|
if modifier_id != modifier.modifier_id or modifier_version != modifier.version:
|
||||||
|
_projection_fail(path, "error identity does not match root modifier metadata")
|
||||||
|
elif position != len(modifiers) or value.stage is not ErrorStage.PREFLIGHT:
|
||||||
|
_projection_fail(path, "error position is outside root modifier metadata")
|
||||||
|
|
||||||
|
diagnostic_type = _projection_nonempty_string(value.error_type, f"{path}.diagnostic_type")
|
||||||
|
message = _projection_nonempty_string(value.message, f"{path}.message")
|
||||||
|
projection: ReviewProjection = {
|
||||||
|
"code": _ERROR_CODES[value.stage],
|
||||||
|
"stage": value.stage.value,
|
||||||
|
"modifier_position": position,
|
||||||
|
"modifier_id": modifier_id,
|
||||||
|
"modifier_version": modifier_version,
|
||||||
|
}
|
||||||
|
if detail is not ReviewDetail.SUMMARY:
|
||||||
|
projection["diagnostic_type"] = diagnostic_type
|
||||||
|
projection["message"] = message
|
||||||
|
return projection
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_residual_edit(value: object, *, edit_index: int, path: str) -> ReviewProjection:
|
||||||
|
if not isinstance(value, TextEdit):
|
||||||
|
_projection_fail(path, "value must be TextEdit")
|
||||||
|
span = _projection_span(value.span, f"{path}.span")
|
||||||
|
expected_text = _projection_string(value.expected_text, f"{path}.expected_text")
|
||||||
|
replacement = _projection_string(value.replacement, f"{path}.replacement")
|
||||||
|
if len(expected_text) != value.span.end - value.span.start or expected_text == replacement:
|
||||||
|
_projection_fail(path, "residual edit text does not satisfy its span contract")
|
||||||
|
return {
|
||||||
|
"edit_index": _projection_integer(edit_index, f"{path}.edit_index"),
|
||||||
|
"span": span,
|
||||||
|
"expected_text": expected_text,
|
||||||
|
"replacement": replacement,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _projection_residual(
|
||||||
|
value: object,
|
||||||
|
*,
|
||||||
|
residual_index: int,
|
||||||
|
modifiers: tuple[ModifierInfo, ...],
|
||||||
|
current_sha256: str,
|
||||||
|
) -> ReviewProjection:
|
||||||
|
path = f"residual_proposals[{residual_index}]"
|
||||||
|
if not isinstance(value, ResidualProposal):
|
||||||
|
_projection_fail(path, "value must be ResidualProposal")
|
||||||
|
position = _projection_integer(value.modifier_position, f"{path}.modifier_position")
|
||||||
|
if position >= len(modifiers):
|
||||||
|
_projection_fail(path, "residual position is outside root modifier metadata")
|
||||||
|
modifier = modifiers[position]
|
||||||
|
if value.modifier_id != modifier.modifier_id or value.modifier_version != modifier.version:
|
||||||
|
_projection_fail(path, "residual identity does not match root modifier metadata")
|
||||||
|
|
||||||
|
proposal_ref = value.proposal_ref
|
||||||
|
if not isinstance(proposal_ref, ProposalReference):
|
||||||
|
_projection_fail(f"{path}.proposal_ref", "value must be ProposalReference")
|
||||||
|
proposal_index = _projection_integer(
|
||||||
|
proposal_ref.proposal_index,
|
||||||
|
f"{path}.proposal_ref.proposal_index",
|
||||||
|
)
|
||||||
|
if proposal_ref.modifier_position != position or proposal_ref.snapshot_sha256 != current_sha256:
|
||||||
|
_projection_fail(f"{path}.proposal_ref", "reference does not target the current snapshot")
|
||||||
|
|
||||||
|
proposal = value.proposal
|
||||||
|
if not isinstance(proposal, ProposedChange):
|
||||||
|
_projection_fail(f"{path}.proposal", "value must be ProposedChange")
|
||||||
|
snapshot_sha256 = _projection_sha256(proposal.snapshot_sha256, f"{path}.snapshot_sha256")
|
||||||
|
if snapshot_sha256 != current_sha256:
|
||||||
|
_projection_fail(path, "residual proposal does not target the current snapshot")
|
||||||
|
reason = _projection_nonempty_string(proposal.reason, f"{path}.reason")
|
||||||
|
if not isinstance(proposal.edits, tuple) or not proposal.edits:
|
||||||
|
_projection_fail(f"{path}.edits", "edits must be a non-empty tuple")
|
||||||
|
edits: list[JsonValue] = [
|
||||||
|
_projection_residual_edit(edit, edit_index=index, path=f"{path}.edits[{index}]")
|
||||||
|
for index, edit in enumerate(proposal.edits)
|
||||||
|
]
|
||||||
|
for index, edit in enumerate(proposal.edits):
|
||||||
|
if edit.snapshot_sha256 != current_sha256:
|
||||||
|
_projection_fail(f"{path}.edits[{index}]", "edit does not target the current snapshot")
|
||||||
|
return {
|
||||||
|
"modifier_position": position,
|
||||||
|
"proposal_index": proposal_index,
|
||||||
|
"snapshot_sha256": snapshot_sha256,
|
||||||
|
"reason": reason,
|
||||||
|
"edits": edits,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _require_no_body_fields(value: JsonValue, path: str = "root") -> None:
|
||||||
|
if isinstance(value, dict):
|
||||||
|
for key, nested in value.items():
|
||||||
|
if key in _BODY_FIELD_NAMES:
|
||||||
|
_projection_fail(path, "summary contains a body-bearing field")
|
||||||
|
_require_no_body_fields(nested, f"{path}.{key}")
|
||||||
|
elif isinstance(value, list):
|
||||||
|
for index, nested in enumerate(value):
|
||||||
|
_require_no_body_fields(nested, f"{path}[{index}]")
|
||||||
|
|
||||||
|
|
||||||
|
def review_document_to_dict(
|
||||||
|
review: ReviewDocument,
|
||||||
|
*,
|
||||||
|
detail: ReviewDetail | str = ReviewDetail.SUMMARY,
|
||||||
|
) -> ReviewProjection:
|
||||||
|
"""把可信评审对象投影为 schema 1.0 的普通 JSON 基本值。"""
|
||||||
|
if not isinstance(review, ReviewDocument):
|
||||||
|
_projection_fail("review", "value must be a ReviewDocument")
|
||||||
|
resolved_detail = _projection_detail(detail)
|
||||||
|
if not isinstance(review.status, RunStatus):
|
||||||
|
_projection_fail("review.status", "status is not supported by schema 1.0")
|
||||||
|
if not isinstance(review.current_kind, ReviewCurrentKind):
|
||||||
|
_projection_fail("review.current_kind", "current kind is not supported by schema 1.0")
|
||||||
|
if (review.status is RunStatus.SUCCESS) != (
|
||||||
|
review.current_kind is ReviewCurrentKind.SUCCESS_OUTPUT
|
||||||
|
):
|
||||||
|
_projection_fail("review.current_kind", "current kind does not match run status")
|
||||||
|
if type(review.stages_complete) is not bool:
|
||||||
|
_projection_fail("review.stages_complete", "value must be a boolean")
|
||||||
|
if not isinstance(review.modifiers, tuple):
|
||||||
|
_projection_fail("review.modifiers", "modifiers must be a tuple")
|
||||||
|
if not isinstance(review.stages, tuple):
|
||||||
|
_projection_fail("review.stages", "stages must be a tuple")
|
||||||
|
if not isinstance(review.errors, tuple):
|
||||||
|
_projection_fail("review.errors", "errors must be a tuple")
|
||||||
|
if not isinstance(review.residual_proposals, tuple):
|
||||||
|
_projection_fail("review.residual_proposals", "residual proposals must be a tuple")
|
||||||
|
|
||||||
|
input_projection = _projection_text(
|
||||||
|
review.input_markdown,
|
||||||
|
review.input_sha256,
|
||||||
|
include_markdown=resolved_detail is ReviewDetail.FULL,
|
||||||
|
path="input",
|
||||||
|
)
|
||||||
|
current_sha256 = _projection_sha256(review.current_sha256, "current.sha256")
|
||||||
|
current_projection = _projection_text(
|
||||||
|
review.current_markdown,
|
||||||
|
current_sha256,
|
||||||
|
include_markdown=resolved_detail is ReviewDetail.FULL,
|
||||||
|
path="current",
|
||||||
|
)
|
||||||
|
|
||||||
|
modifier_projections: list[JsonValue] = [
|
||||||
|
_projection_modifier(modifier, position, resolved_detail)
|
||||||
|
for position, modifier in enumerate(review.modifiers)
|
||||||
|
]
|
||||||
|
stage_projections: list[JsonValue] = []
|
||||||
|
total_changes = 0
|
||||||
|
for expected_position, stage in enumerate(review.stages):
|
||||||
|
stage_projection, stage_change_count = _projection_stage(
|
||||||
|
stage,
|
||||||
|
expected_position=expected_position,
|
||||||
|
modifiers=review.modifiers,
|
||||||
|
detail=resolved_detail,
|
||||||
|
)
|
||||||
|
stage_projections.append(stage_projection)
|
||||||
|
total_changes += stage_change_count
|
||||||
|
|
||||||
|
error_projections: list[JsonValue] = [
|
||||||
|
_projection_error(
|
||||||
|
error,
|
||||||
|
error_index=index,
|
||||||
|
modifiers=review.modifiers,
|
||||||
|
detail=resolved_detail,
|
||||||
|
)
|
||||||
|
for index, error in enumerate(review.errors)
|
||||||
|
]
|
||||||
|
residual_projections: list[JsonValue] = [
|
||||||
|
_projection_residual(
|
||||||
|
residual,
|
||||||
|
residual_index=index,
|
||||||
|
modifiers=review.modifiers,
|
||||||
|
current_sha256=current_sha256,
|
||||||
|
)
|
||||||
|
for index, residual in enumerate(review.residual_proposals)
|
||||||
|
]
|
||||||
|
|
||||||
|
hash_contract: ReviewProjection = {
|
||||||
|
"algorithm": "sha256",
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"normalization": "none",
|
||||||
|
}
|
||||||
|
coordinate_contract: ReviewProjection = {
|
||||||
|
"offset_unit": "unicode_code_point",
|
||||||
|
"span_index_base": 0,
|
||||||
|
"span_end": "exclusive",
|
||||||
|
"location_index_base": 1,
|
||||||
|
"physical_line_endings": ["lf", "crlf", "cr"],
|
||||||
|
}
|
||||||
|
counts: ReviewProjection = {
|
||||||
|
"modifier_count": _projection_integer(len(review.modifiers), "counts.modifier_count"),
|
||||||
|
"completed_stage_count": _projection_integer(
|
||||||
|
len(review.stages),
|
||||||
|
"counts.completed_stage_count",
|
||||||
|
),
|
||||||
|
"change_count": _projection_integer(total_changes, "counts.change_count"),
|
||||||
|
"error_count": _projection_integer(len(review.errors), "counts.error_count"),
|
||||||
|
"residual_proposal_count": _projection_integer(
|
||||||
|
len(review.residual_proposals),
|
||||||
|
"counts.residual_proposal_count",
|
||||||
|
),
|
||||||
|
}
|
||||||
|
projection: ReviewProjection = {
|
||||||
|
"schema_name": _SCHEMA_NAME,
|
||||||
|
"schema_version": _SCHEMA_VERSION,
|
||||||
|
"detail": resolved_detail.value,
|
||||||
|
"status": review.status.value,
|
||||||
|
"current_kind": review.current_kind.value,
|
||||||
|
"stages_complete": review.stages_complete,
|
||||||
|
"hash_contract": hash_contract,
|
||||||
|
"coordinate_contract": coordinate_contract,
|
||||||
|
"input": input_projection,
|
||||||
|
"current": current_projection,
|
||||||
|
"counts": counts,
|
||||||
|
"modifiers": modifier_projections,
|
||||||
|
"stages": stage_projections,
|
||||||
|
"errors": error_projections,
|
||||||
|
}
|
||||||
|
if resolved_detail is not ReviewDetail.SUMMARY:
|
||||||
|
projection["residual_proposals"] = residual_projections
|
||||||
|
else:
|
||||||
|
_require_no_body_fields(projection)
|
||||||
|
return projection
|
||||||
|
|
||||||
|
|
||||||
|
def render_json_report(
|
||||||
|
review: ReviewDocument,
|
||||||
|
*,
|
||||||
|
detail: ReviewDetail | str = ReviewDetail.SUMMARY,
|
||||||
|
) -> str:
|
||||||
|
"""把正式机器投影确定地编码为内存 JSON 字符串。"""
|
||||||
|
return dumps(
|
||||||
|
review_document_to_dict(review, detail=detail),
|
||||||
|
ensure_ascii=False,
|
||||||
|
allow_nan=False,
|
||||||
|
indent=2,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _maximum_run(text: str, character: str) -> int:
|
def _maximum_run(text: str, character: str) -> int:
|
||||||
longest = 0
|
longest = 0
|
||||||
current = 0
|
current = 0
|
||||||
@@ -714,12 +1259,19 @@ def render_markdown_report(review: ReviewDocument, *, residual_limit: int = 20)
|
|||||||
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
|
"JsonScalar",
|
||||||
|
"JsonValue",
|
||||||
"ReviewBuildError",
|
"ReviewBuildError",
|
||||||
"ReviewChange",
|
"ReviewChange",
|
||||||
"ReviewCurrentKind",
|
"ReviewCurrentKind",
|
||||||
|
"ReviewDetail",
|
||||||
"ReviewDocument",
|
"ReviewDocument",
|
||||||
"ReviewLocation",
|
"ReviewLocation",
|
||||||
|
"ReviewProjection",
|
||||||
|
"ReviewProjectionError",
|
||||||
"ReviewStage",
|
"ReviewStage",
|
||||||
"build_review_document",
|
"build_review_document",
|
||||||
|
"render_json_report",
|
||||||
"render_markdown_report",
|
"render_markdown_report",
|
||||||
|
"review_document_to_dict",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -0,0 +1,512 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import replace
|
||||||
|
from json import loads
|
||||||
|
from typing import cast
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from mdpolish import (
|
||||||
|
DocumentSnapshot,
|
||||||
|
Modifier,
|
||||||
|
ModifierInfo,
|
||||||
|
Pipeline,
|
||||||
|
ProposedChange,
|
||||||
|
RunStatus,
|
||||||
|
TextEdit,
|
||||||
|
TextSpan,
|
||||||
|
)
|
||||||
|
from mdpolish.review import (
|
||||||
|
JsonValue,
|
||||||
|
ReviewCurrentKind,
|
||||||
|
ReviewDetail,
|
||||||
|
ReviewDocument,
|
||||||
|
ReviewLocation,
|
||||||
|
ReviewProjection,
|
||||||
|
ReviewProjectionError,
|
||||||
|
build_review_document,
|
||||||
|
render_json_report,
|
||||||
|
review_document_to_dict,
|
||||||
|
)
|
||||||
|
|
||||||
|
_MAX_SAFE_JSON_INTEGER = 2**53 - 1
|
||||||
|
|
||||||
|
|
||||||
|
def _replace_modifier(
|
||||||
|
needle: str,
|
||||||
|
replacement: str,
|
||||||
|
*,
|
||||||
|
modifier_id: str,
|
||||||
|
reason: str,
|
||||||
|
parameters: dict[str, object] | None = None,
|
||||||
|
applicability: str = "只处理虚构投影测试标记。",
|
||||||
|
) -> Modifier:
|
||||||
|
def propose(snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]:
|
||||||
|
position = snapshot.markdown.find(needle)
|
||||||
|
if position < 0:
|
||||||
|
return ()
|
||||||
|
edit = TextEdit(
|
||||||
|
snapshot_sha256=snapshot.sha256,
|
||||||
|
span=TextSpan(position, position + len(needle)),
|
||||||
|
expected_text=needle,
|
||||||
|
replacement=replacement,
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
ProposedChange(
|
||||||
|
snapshot_sha256=snapshot.sha256,
|
||||||
|
reason=reason,
|
||||||
|
edits=(edit,),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
return Modifier(
|
||||||
|
modifier_id=modifier_id,
|
||||||
|
version="1.2.3",
|
||||||
|
parameters=parameters or {"needle": needle, "replacement": replacement},
|
||||||
|
applicability=applicability,
|
||||||
|
propose=propose,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _rich_success_review() -> ReviewDocument:
|
||||||
|
modifier = _replace_modifier(
|
||||||
|
"BEFORE_SECRET",
|
||||||
|
"AFTER_SECRET",
|
||||||
|
modifier_id="test.machine-projection",
|
||||||
|
reason="REASON_SECRET",
|
||||||
|
parameters={
|
||||||
|
"enabled": True,
|
||||||
|
"finite": 1.25,
|
||||||
|
"mapping": {"alpha": 1},
|
||||||
|
"none": None,
|
||||||
|
"sequence": (("alpha", 1),),
|
||||||
|
},
|
||||||
|
applicability="APPLICABILITY_SECRET",
|
||||||
|
)
|
||||||
|
input_markdown = "中文_SOURCE_ONLY\r\nBEFORE_SECRET\tTAIL"
|
||||||
|
result = Pipeline((modifier,)).transform(input_markdown)
|
||||||
|
assert result.status is RunStatus.SUCCESS
|
||||||
|
return build_review_document(input_markdown, result)
|
||||||
|
|
||||||
|
|
||||||
|
def _unstable_review() -> ReviewDocument:
|
||||||
|
def propose_residual(snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]:
|
||||||
|
needle = "RESIDUAL_EXPECTED_SECRET"
|
||||||
|
position = snapshot.markdown.find(needle)
|
||||||
|
if position < 0:
|
||||||
|
return ()
|
||||||
|
edit = TextEdit(
|
||||||
|
snapshot_sha256=snapshot.sha256,
|
||||||
|
span=TextSpan(position, position + len(needle)),
|
||||||
|
expected_text=needle,
|
||||||
|
replacement="RESIDUAL_REPLACEMENT_SECRET",
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
ProposedChange(
|
||||||
|
snapshot_sha256=snapshot.sha256,
|
||||||
|
reason="RESIDUAL_REASON_SECRET",
|
||||||
|
edits=(edit,),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
residual = Modifier(
|
||||||
|
modifier_id="test.residual",
|
||||||
|
version="1.0.0",
|
||||||
|
parameters=(),
|
||||||
|
applicability="只在最终复查中产生虚构候选。",
|
||||||
|
propose=propose_residual,
|
||||||
|
)
|
||||||
|
producer = _replace_modifier(
|
||||||
|
"start",
|
||||||
|
"RESIDUAL_EXPECTED_SECRET",
|
||||||
|
modifier_id="test.producer",
|
||||||
|
reason="produce residual marker",
|
||||||
|
)
|
||||||
|
result = Pipeline((residual, producer)).transform("start")
|
||||||
|
assert result.status is RunStatus.UNSTABLE
|
||||||
|
return build_review_document("start", result)
|
||||||
|
|
||||||
|
|
||||||
|
def _exploding_modifier(modifier_id: str, *, trigger: str | None = None) -> Modifier:
|
||||||
|
def propose(snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]:
|
||||||
|
if trigger is None or snapshot.markdown == trigger:
|
||||||
|
raise RuntimeError(f"ERROR_MESSAGE_SECRET: {snapshot.markdown}")
|
||||||
|
return ()
|
||||||
|
|
||||||
|
return Modifier(
|
||||||
|
modifier_id=modifier_id,
|
||||||
|
version="1.0.0",
|
||||||
|
parameters=(),
|
||||||
|
applicability="只测试机器错误投影。",
|
||||||
|
propose=propose,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _object(value: JsonValue) -> ReviewProjection:
|
||||||
|
assert isinstance(value, dict)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _array(value: JsonValue) -> list[JsonValue]:
|
||||||
|
assert isinstance(value, list)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _contains_key(value: JsonValue, key: str) -> bool:
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return key in value or any(_contains_key(nested, key) for nested in value.values())
|
||||||
|
if isinstance(value, list):
|
||||||
|
return any(_contains_key(nested, key) for nested in value)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_summary_projection_has_stable_schema_without_body_content() -> None:
|
||||||
|
review = _rich_success_review()
|
||||||
|
|
||||||
|
projection = review_document_to_dict(review)
|
||||||
|
|
||||||
|
assert set(projection) == {
|
||||||
|
"schema_name",
|
||||||
|
"schema_version",
|
||||||
|
"detail",
|
||||||
|
"status",
|
||||||
|
"current_kind",
|
||||||
|
"stages_complete",
|
||||||
|
"hash_contract",
|
||||||
|
"coordinate_contract",
|
||||||
|
"input",
|
||||||
|
"current",
|
||||||
|
"counts",
|
||||||
|
"modifiers",
|
||||||
|
"stages",
|
||||||
|
"errors",
|
||||||
|
}
|
||||||
|
assert projection["schema_name"] == "mdpolish.review"
|
||||||
|
assert projection["schema_version"] == "1.0"
|
||||||
|
assert projection["detail"] == "summary"
|
||||||
|
assert projection["status"] == "success"
|
||||||
|
assert projection["current_kind"] == "success_output"
|
||||||
|
assert projection["stages_complete"] is True
|
||||||
|
assert projection["hash_contract"] == {
|
||||||
|
"algorithm": "sha256",
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"normalization": "none",
|
||||||
|
}
|
||||||
|
assert projection["coordinate_contract"] == {
|
||||||
|
"offset_unit": "unicode_code_point",
|
||||||
|
"span_index_base": 0,
|
||||||
|
"span_end": "exclusive",
|
||||||
|
"location_index_base": 1,
|
||||||
|
"physical_line_endings": ["lf", "crlf", "cr"],
|
||||||
|
}
|
||||||
|
assert projection["counts"] == {
|
||||||
|
"modifier_count": 1,
|
||||||
|
"completed_stage_count": 1,
|
||||||
|
"change_count": 1,
|
||||||
|
"error_count": 0,
|
||||||
|
"residual_proposal_count": 0,
|
||||||
|
}
|
||||||
|
|
||||||
|
input_projection = _object(projection["input"])
|
||||||
|
current_projection = _object(projection["current"])
|
||||||
|
assert input_projection == {
|
||||||
|
"sha256": review.input_sha256,
|
||||||
|
"code_point_length": len(review.input_markdown),
|
||||||
|
}
|
||||||
|
assert current_projection == {
|
||||||
|
"sha256": review.current_sha256,
|
||||||
|
"code_point_length": len(review.current_markdown),
|
||||||
|
}
|
||||||
|
|
||||||
|
modifiers = _array(projection["modifiers"])
|
||||||
|
assert modifiers == [
|
||||||
|
{
|
||||||
|
"position": 0,
|
||||||
|
"modifier_id": "test.machine-projection",
|
||||||
|
"version": "1.2.3",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
stages = _array(projection["stages"])
|
||||||
|
stage = _object(stages[0])
|
||||||
|
assert set(stage) == {"modifier_position", "before", "after", "change_count"}
|
||||||
|
assert stage["modifier_position"] == 0
|
||||||
|
assert stage["change_count"] == 1
|
||||||
|
assert stage["before"] == {
|
||||||
|
"sha256": review.stages[0].before_sha256,
|
||||||
|
"code_point_length": len(review.stages[0].before_markdown),
|
||||||
|
}
|
||||||
|
assert stage["after"] == {
|
||||||
|
"sha256": review.stages[0].after_sha256,
|
||||||
|
"code_point_length": len(review.stages[0].after_markdown),
|
||||||
|
}
|
||||||
|
|
||||||
|
rendered = render_json_report(review)
|
||||||
|
for secret in (
|
||||||
|
"中文_SOURCE_ONLY",
|
||||||
|
"BEFORE_SECRET",
|
||||||
|
"AFTER_SECRET",
|
||||||
|
"REASON_SECRET",
|
||||||
|
"APPLICABILITY_SECRET",
|
||||||
|
"ERROR_MESSAGE_SECRET",
|
||||||
|
):
|
||||||
|
assert secret not in rendered
|
||||||
|
for body_key in (
|
||||||
|
"markdown",
|
||||||
|
"expected_text",
|
||||||
|
"replacement",
|
||||||
|
"reason",
|
||||||
|
"message",
|
||||||
|
"parameters",
|
||||||
|
"applicability",
|
||||||
|
):
|
||||||
|
assert not _contains_key(projection, body_key)
|
||||||
|
|
||||||
|
|
||||||
|
def test_changes_and_full_details_add_content_monotonically() -> None:
|
||||||
|
review = _rich_success_review()
|
||||||
|
|
||||||
|
summary = review_document_to_dict(review, detail=ReviewDetail.SUMMARY)
|
||||||
|
changes = review_document_to_dict(review, detail="changes")
|
||||||
|
full = review_document_to_dict(review, detail=ReviewDetail.FULL)
|
||||||
|
|
||||||
|
assert set(summary) < set(changes)
|
||||||
|
assert set(changes) == set(full)
|
||||||
|
assert changes["detail"] == "changes"
|
||||||
|
assert full["detail"] == "full"
|
||||||
|
for key in set(summary) - {"detail", "input", "current", "modifiers", "stages"}:
|
||||||
|
assert summary[key] == changes[key] == full[key]
|
||||||
|
|
||||||
|
changes_modifier = _object(_array(changes["modifiers"])[0])
|
||||||
|
full_modifier = _object(_array(full["modifiers"])[0])
|
||||||
|
assert changes_modifier == full_modifier
|
||||||
|
assert changes_modifier["applicability"] == "APPLICABILITY_SECRET"
|
||||||
|
assert changes_modifier["parameters"] == [
|
||||||
|
["enabled", True],
|
||||||
|
["finite", 1.25],
|
||||||
|
["mapping", [["alpha", 1]]],
|
||||||
|
["none", None],
|
||||||
|
["sequence", [["alpha", 1]]],
|
||||||
|
]
|
||||||
|
|
||||||
|
changes_stage = _object(_array(changes["stages"])[0])
|
||||||
|
full_stage = _object(_array(full["stages"])[0])
|
||||||
|
assert "markdown" not in _object(changes_stage["before"])
|
||||||
|
assert "markdown" not in _object(changes_stage["after"])
|
||||||
|
assert not _contains_key(changes, "markdown")
|
||||||
|
assert _object(full_stage["before"])["markdown"] == review.input_markdown
|
||||||
|
assert _object(full_stage["after"])["markdown"] == review.current_markdown
|
||||||
|
assert _object(full["input"])["markdown"] == review.input_markdown
|
||||||
|
assert _object(full["current"])["markdown"] == review.current_markdown
|
||||||
|
|
||||||
|
change = _object(_array(changes_stage["changes"])[0])
|
||||||
|
assert change == {
|
||||||
|
"proposal_index": 0,
|
||||||
|
"edit_index": 0,
|
||||||
|
"reason": "REASON_SECRET",
|
||||||
|
"location": {"line": 2, "column": 1},
|
||||||
|
"span": {
|
||||||
|
"start": review.input_markdown.index("BEFORE_SECRET"),
|
||||||
|
"end": review.input_markdown.index("BEFORE_SECRET") + len("BEFORE_SECRET"),
|
||||||
|
},
|
||||||
|
"before": "BEFORE_SECRET",
|
||||||
|
"after": "AFTER_SECRET",
|
||||||
|
"before_sha256": review.stages[0].before_sha256,
|
||||||
|
"after_sha256": review.stages[0].after_sha256,
|
||||||
|
}
|
||||||
|
assert "中文_SOURCE_ONLY" not in render_json_report(review, detail="changes")
|
||||||
|
|
||||||
|
|
||||||
|
def test_residual_content_requires_changes_or_full_detail() -> None:
|
||||||
|
review = _unstable_review()
|
||||||
|
|
||||||
|
summary = review_document_to_dict(review, detail="summary")
|
||||||
|
changes = review_document_to_dict(review, detail="changes")
|
||||||
|
|
||||||
|
assert summary["status"] == "unstable"
|
||||||
|
assert summary["current_kind"] == "partial_output"
|
||||||
|
assert _object(summary["counts"])["residual_proposal_count"] == 1
|
||||||
|
assert "residual_proposals" not in summary
|
||||||
|
assert "RESIDUAL_EXPECTED_SECRET" not in render_json_report(review, detail="summary")
|
||||||
|
|
||||||
|
residuals = _array(changes["residual_proposals"])
|
||||||
|
assert len(residuals) == 1
|
||||||
|
residual = _object(residuals[0])
|
||||||
|
assert residual["modifier_position"] == 0
|
||||||
|
assert residual["proposal_index"] == 0
|
||||||
|
assert residual["snapshot_sha256"] == review.current_sha256
|
||||||
|
assert residual["reason"] == "RESIDUAL_REASON_SECRET"
|
||||||
|
edit = _object(_array(residual["edits"])[0])
|
||||||
|
assert edit["edit_index"] == 0
|
||||||
|
assert edit["expected_text"] == "RESIDUAL_EXPECTED_SECRET"
|
||||||
|
assert edit["replacement"] == "RESIDUAL_REPLACEMENT_SECRET"
|
||||||
|
|
||||||
|
|
||||||
|
def _error_reviews() -> tuple[tuple[ReviewDocument, str], ...]:
|
||||||
|
invalid_pipeline = Pipeline(cast(tuple[Modifier, ...], (object(),)))
|
||||||
|
preflight_result = invalid_pipeline.transform("PREFLIGHT_SOURCE_SECRET")
|
||||||
|
preflight = build_review_document("PREFLIGHT_SOURCE_SECRET", preflight_result)
|
||||||
|
|
||||||
|
transform_result = Pipeline((_exploding_modifier("test.transform-error"),)).transform(
|
||||||
|
"TRANSFORM_SOURCE_SECRET"
|
||||||
|
)
|
||||||
|
transform = build_review_document("TRANSFORM_SOURCE_SECRET", transform_result)
|
||||||
|
|
||||||
|
review_error = _exploding_modifier("test.final-review-error", trigger="done")
|
||||||
|
producer = _replace_modifier(
|
||||||
|
"start",
|
||||||
|
"done",
|
||||||
|
modifier_id="test.final-producer",
|
||||||
|
reason="produce final review trigger",
|
||||||
|
)
|
||||||
|
final_result = Pipeline((review_error, producer)).transform("start")
|
||||||
|
final_review = build_review_document("start", final_result)
|
||||||
|
return (
|
||||||
|
(preflight, "run.preflight_failed"),
|
||||||
|
(transform, "run.transform_failed"),
|
||||||
|
(final_review, "run.final_review_failed"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_error_projection_uses_stable_codes_and_hides_diagnostics_in_summary() -> None:
|
||||||
|
for review, expected_code in _error_reviews():
|
||||||
|
summary = review_document_to_dict(review, detail="summary")
|
||||||
|
changes = review_document_to_dict(review, detail="changes")
|
||||||
|
|
||||||
|
summary_error = _object(_array(summary["errors"])[0])
|
||||||
|
changes_error = _object(_array(changes["errors"])[0])
|
||||||
|
assert summary_error["code"] == expected_code
|
||||||
|
assert summary_error["stage"] == review.errors[0].stage.value
|
||||||
|
assert "diagnostic_type" not in summary_error
|
||||||
|
assert "message" not in summary_error
|
||||||
|
assert "ERROR_MESSAGE_SECRET" not in render_json_report(review, detail="summary")
|
||||||
|
assert changes_error["diagnostic_type"] == review.errors[0].error_type
|
||||||
|
assert changes_error["message"] == review.errors[0].message
|
||||||
|
|
||||||
|
|
||||||
|
def test_projection_preserves_unicode_line_endings_and_exact_coordinates() -> None:
|
||||||
|
modifier = _replace_modifier(
|
||||||
|
"🙂Cafe\u0301",
|
||||||
|
"完成",
|
||||||
|
modifier_id="test.unicode",
|
||||||
|
reason="Unicode coordinate test",
|
||||||
|
)
|
||||||
|
input_markdown = "\ufeff首行\r\n🙂Cafe\u0301\r尾行\n"
|
||||||
|
result = Pipeline((modifier,)).transform(input_markdown)
|
||||||
|
review = build_review_document(input_markdown, result)
|
||||||
|
|
||||||
|
projection = review_document_to_dict(review, detail="full")
|
||||||
|
stage = _object(_array(projection["stages"])[0])
|
||||||
|
change = _object(_array(stage["changes"])[0])
|
||||||
|
|
||||||
|
assert _object(projection["input"])["code_point_length"] == len(input_markdown)
|
||||||
|
assert _object(projection["input"])["markdown"] == input_markdown
|
||||||
|
assert change["location"] == {"line": 2, "column": 1}
|
||||||
|
assert change["span"] == {
|
||||||
|
"start": input_markdown.index("🙂"),
|
||||||
|
"end": input_markdown.index("🙂") + len("🙂Cafe\u0301"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_json_report_is_deterministic_unicode_json_without_bom_or_final_newline() -> None:
|
||||||
|
review = _rich_success_review()
|
||||||
|
|
||||||
|
first = render_json_report(review, detail="full")
|
||||||
|
second = render_json_report(review, detail=ReviewDetail.FULL)
|
||||||
|
|
||||||
|
assert first == second
|
||||||
|
assert loads(first) == review_document_to_dict(review, detail="full")
|
||||||
|
assert not first.startswith("\ufeff")
|
||||||
|
assert not first.endswith("\n")
|
||||||
|
assert "中文_SOURCE_ONLY" in first
|
||||||
|
assert "\\u4e2d" not in first.lower()
|
||||||
|
assert "\\tTAIL" in first
|
||||||
|
assert "NaN" not in first
|
||||||
|
assert "Infinity" not in first
|
||||||
|
|
||||||
|
|
||||||
|
def test_projection_returns_fresh_containers_without_mutating_review() -> None:
|
||||||
|
review = _rich_success_review()
|
||||||
|
|
||||||
|
first = review_document_to_dict(review, detail="full")
|
||||||
|
second = review_document_to_dict(review, detail="full")
|
||||||
|
first["schema_name"] = "changed"
|
||||||
|
_array(first["modifiers"]).clear()
|
||||||
|
|
||||||
|
assert second["schema_name"] == "mdpolish.review"
|
||||||
|
assert len(_array(second["modifiers"])) == 1
|
||||||
|
assert review.modifiers[0].modifier_id == "test.machine-projection"
|
||||||
|
assert review_document_to_dict(review, detail="full") == second
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("detail", ["", "SUMMARY", "unknown", 1, True, None])
|
||||||
|
def test_projection_rejects_unknown_detail(detail: object) -> None:
|
||||||
|
review = _rich_success_review()
|
||||||
|
|
||||||
|
with pytest.raises(ReviewProjectionError, match="review projection failed at detail"):
|
||||||
|
review_document_to_dict(review, detail=detail) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def test_projection_rejects_wrong_review_and_nested_model_types() -> None:
|
||||||
|
with pytest.raises(ReviewProjectionError, match="ReviewDocument"):
|
||||||
|
review_document_to_dict(object()) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
review = _rich_success_review()
|
||||||
|
malformed = replace(review, modifiers=cast(tuple[ModifierInfo, ...], (object(),)))
|
||||||
|
with pytest.raises(ReviewProjectionError, match=r"modifiers\[0\]"):
|
||||||
|
review_document_to_dict(malformed)
|
||||||
|
|
||||||
|
|
||||||
|
def test_projection_rejects_unknown_enum_invalid_reference_and_large_integer() -> None:
|
||||||
|
review = _rich_success_review()
|
||||||
|
unknown_status = replace(review, status=cast(RunStatus, "future"))
|
||||||
|
bad_stage = replace(review.stages[0], modifier_position=1)
|
||||||
|
bad_reference = replace(review, stages=(bad_stage,))
|
||||||
|
review_change = review.stages[0].changes[0]
|
||||||
|
large_location = replace(
|
||||||
|
review_change,
|
||||||
|
location=ReviewLocation(line=_MAX_SAFE_JSON_INTEGER + 1, column=1),
|
||||||
|
)
|
||||||
|
large_stage = replace(review.stages[0], changes=(large_location,))
|
||||||
|
large_integer = replace(review, stages=(large_stage,))
|
||||||
|
|
||||||
|
for malformed in (unknown_status, bad_reference, large_integer):
|
||||||
|
with pytest.raises(ReviewProjectionError):
|
||||||
|
review_document_to_dict(malformed)
|
||||||
|
|
||||||
|
|
||||||
|
def test_projection_rejects_nonfinite_parameter_and_invalid_unicode_without_leaking_values() -> None:
|
||||||
|
review = _rich_success_review()
|
||||||
|
modifier = review.modifiers[0]
|
||||||
|
nonfinite_modifier = replace(modifier, parameters=(("NONFINITE_SECRET", float("nan")),))
|
||||||
|
nonfinite_stage = replace(review.stages[0], modifier=nonfinite_modifier)
|
||||||
|
nonfinite_review = replace(
|
||||||
|
review,
|
||||||
|
modifiers=(nonfinite_modifier,),
|
||||||
|
stages=(nonfinite_stage,),
|
||||||
|
)
|
||||||
|
invalid_unicode = replace(review, input_markdown="UNICODE_SECRET\ud800")
|
||||||
|
|
||||||
|
with pytest.raises(ReviewProjectionError) as nonfinite_error:
|
||||||
|
review_document_to_dict(nonfinite_review)
|
||||||
|
assert "NONFINITE_SECRET" not in str(nonfinite_error.value)
|
||||||
|
|
||||||
|
with pytest.raises(ReviewProjectionError) as unicode_error:
|
||||||
|
review_document_to_dict(invalid_unicode)
|
||||||
|
assert "UNICODE_SECRET" not in str(unicode_error.value)
|
||||||
|
|
||||||
|
|
||||||
|
def test_empty_document_projection_distinguishes_hidden_and_empty_markdown() -> None:
|
||||||
|
result = Pipeline(()).transform("")
|
||||||
|
review = build_review_document("", result)
|
||||||
|
|
||||||
|
summary = review_document_to_dict(review, detail="summary")
|
||||||
|
full = review_document_to_dict(review, detail="full")
|
||||||
|
|
||||||
|
assert "markdown" not in _object(summary["input"])
|
||||||
|
assert _object(full["input"])["markdown"] == ""
|
||||||
|
assert _object(full["current"])["markdown"] == ""
|
||||||
|
assert full["modifiers"] == []
|
||||||
|
assert full["stages"] == []
|
||||||
|
assert full["errors"] == []
|
||||||
|
assert full["residual_proposals"] == []
|
||||||
|
assert review.current_kind is ReviewCurrentKind.SUCCESS_OUTPUT
|
||||||
Reference in New Issue
Block a user