实现本地清洗实验与产物保存
This commit is contained in:
@@ -7,8 +7,9 @@
|
||||
profile 表达论文、政务文档、RAG、文档对比等不同需求。
|
||||
|
||||
仓库当前已从纯文档治理进入第一版核心实现阶段：已经提供可安装的 Python 内存处理包和测试，用于验证
|
||||
组件组合、精确修改和审计协议，并已有一个严格整行匹配的论文清洗组件。仓库仍不提供完整规则集、命令行工具、
|
||||
文件读写适配器或生产接口，因此目前还不是拿来即可完成整篇文档清洗的成品工具。
|
||||
组件组合、精确修改和审计协议，并已有一个严格整行匹配的论文清洗组件。仓库另有一个只供本地评审的实验脚本，
|
||||
可以保存指定论文的成功输出、审计和 diff。仓库仍不提供完整规则集、公共命令行工具、通用文件适配器或生产接口，
|
||||
因此目前还不是拿来即可完成整篇文档清洗的成品工具。
|
||||
|
||||
## 当前阶段
|
||||
|
||||
@@ -17,11 +18,13 @@ profile 表达论文、政务文档、RAG、文档对比等不同需求。
|
||||
- 提供可安装的 Python 3.11+ 内存处理包，运行时只依赖标准库；
|
||||
- 已实现不可变数据契约、组件基类、原子修改执行器、顺序流水线、审计记录和最终稳定性复查；
|
||||
- 当前唯一正式组件是 `paper.arxiv_submission_stamp`，只删除严格整行匹配的 arXiv 提交边栏戳；
|
||||
- 该组件已在 5 份论文 Markdown 上只读验证，只命中 sim 和 springer 各一处，合法参考文献保持不变；
|
||||
- 当前基础检查为 Ruff、mypy 和 87 项 pytest 测试，实际命令见本文“当前可用检查”。
|
||||
- 已有仓库内实验运行层，能严格读取显式清单、保存成功 Markdown、JSON 审计和 unified diff，并保持输入不变；
|
||||
- 该组件已对 5 份论文 Markdown 完成保存型实验，只修改 sim 和 springer 各一处，合法参考文献保持不变；
|
||||
- 当前基础检查为 Ruff、mypy 和 113 项 pytest 测试，实际命令见本文“当前可用检查”。
|
||||
|
||||
项目还没有完整清洗规则集、Markdown/HTML parser、profile 格式、文件读写、CLI、批处理或生产接口。当前组件
|
||||
只能证明第一条严格规则已经闭环，不能据此认为论文、GovDoc、表格或图片已经具备完整清洗能力。
|
||||
项目还没有完整清洗规则集、Markdown/HTML parser、profile 格式、通用文件输入接口、公共 CLI、通用批处理或生产接口。
|
||||
当前实验脚本只固定运行已批准的 5 份论文和一个组件；它只能证明第一条严格规则已经闭环，不能据此认为论文、GovDoc、
|
||||
表格或图片已经具备完整清洗能力。
|
||||
|
||||
## 服务对象与复用目标
|
||||
|
||||
@@ -43,6 +46,9 @@ GovDoc 目录、具体客户名称或某一转换器的固定输出路径。
|
||||
|
||||
两组数据都不是可提交的自动测试 fixture。`data/` 已被 Git 忽略，清洗实验不得覆盖这些输入。
|
||||
|
||||
本地清洗实验产物位于 `artifacts/<YYYY-MM-DD>/runs/<run_id>/`。产物可能包含完整原文，同样受 Git 忽略，
|
||||
默认保留 30 个日历日，不得提交或复制到外部系统。当前只批准为上述 5 份论文副本保存产物，未批准保存 GovDoc 输出。
|
||||
|
||||
## 面向复用的设计原则
|
||||
|
||||
- **通用核心**：只接收 Markdown；第一版只执行确定、可审计的精确修改，不读取 PDF、图片或转换器 JSON；
|
||||
@@ -66,18 +72,26 @@ mdpolish/
|
||||
│ ├── __init__.py # 第一版核心公共导出
|
||||
│ ├── component.py # 组件基类和元数据契约
|
||||
│ ├── edits.py # 文本编辑验证与原子应用
|
||||
│ ├── experiment.py # 本地实验输入预检与批量编排
|
||||
│ ├── models.py # 不可变数据模型和运行状态
|
||||
│ ├── pipeline.py # 顺序执行和最终稳定性复查
|
||||
│ ├── reporting.py # JSON 审计、行列位置和 unified diff
|
||||
│ ├── artifact_store.py # 私有产物目录和原子发布
|
||||
│ ├── py.typed # 类型信息声明
|
||||
│ └── components/
|
||||
│ ├── __init__.py
|
||||
│ └── arxiv_submission_stamp.py
|
||||
├── scripts/
|
||||
│ └── run_clindb_arxiv_experiment.py # 固定 5 份论文的仓库内实验入口
|
||||
├── tests/
|
||||
│ ├── test_arxiv_submission_stamp.py
|
||||
│ ├── test_artifact_store.py
|
||||
│ ├── test_component.py
|
||||
│ ├── test_edits.py
|
||||
│ ├── test_experiment.py
|
||||
│ ├── test_models.py
|
||||
│ └── test_pipeline.py
|
||||
│ ├── test_pipeline.py
|
||||
│ └── test_reporting.py
|
||||
├── data/ # 本地测试数据；Git 忽略；此处只展开常用入口
|
||||
│ └── md/
|
||||
│ ├── dmp.md
|
||||
@@ -85,6 +99,8 @@ mdpolish/
|
||||
│ ├── jama.md
|
||||
│ ├── sim.md
|
||||
│ └── springer.md
|
||||
├── artifacts/ # 本地敏感实验产物；Git 忽略
|
||||
│ └── <YYYY-MM-DD>/runs/<run_id>/
|
||||
└── research-wiki/
|
||||
├── README.md # Wiki 分类与维护规则
|
||||
├── design/ # 批准前的选择；批准后冻结
|
||||
@@ -104,9 +120,11 @@ mdpolish/
|
||||
4. 与任务直接相关的 `research-wiki/design/` 记录。
|
||||
|
||||
第一版核心的当前机制见 `research-wiki/explanation/first-executable-core.md`，首个真实组件见
|
||||
`research-wiki/explanation/arxiv-submission-stamp.md`。下一项候选是 ClinDB 范围中的 HTML 实体双重转义，
|
||||
`research-wiki/explanation/arxiv-submission-stamp.md`，本地实验产物机制见
|
||||
`research-wiki/explanation/local-experiment-artifacts.md`，实际运行步骤见
|
||||
`research-wiki/guides/run-local-clindb-arxiv-experiment.md`。下一项候选是 ClinDB 范围中的 HTML 实体双重转义，
|
||||
但必须先用新 design 确定只在哪些 HTML 范围替换、如何避开字面示例以及实体替换边界。解析器、CLI、文件适配器、
|
||||
profile 格式和独立检查能力仍需分别设计，不能从当前核心或单个组件存在推导为已经获批。
|
||||
profile 格式和独立检查能力仍需分别设计；当前本地实验适配器不能被推导成这些公共接口已经获批。
|
||||
|
||||
## 当前可用检查
|
||||
|
||||
@@ -117,7 +135,7 @@ python -m venv .venv
|
||||
|
||||
# 第一版核心的基础验收
|
||||
.venv/bin/ruff check .
|
||||
.venv/bin/mypy src tests
|
||||
.venv/bin/mypy src tests scripts/run_clindb_arxiv_experiment.py
|
||||
.venv/bin/pytest
|
||||
|
||||
# 两份 Agent 入口除标题外必须一致；无输出且退出码为 0 表示通过
|
||||
@@ -130,6 +148,6 @@ find research-wiki -maxdepth 2 -type f | sort
|
||||
git status --short
|
||||
```
|
||||
|
||||
上述安装和三项基础验收已于 2026-08-22 在 Python 3.13.11 环境实际运行：Ruff 通过，mypy 检查 12 个源码与
|
||||
测试文件无问题，pytest 共 87 项测试通过。`requires-python` 仍以 `pyproject.toml` 声明的 Python 3.11 及以上为准；
|
||||
上述安装和三项基础验收已于 2026-08-22 在 Python 3.13.11 环境实际运行：Ruff 通过，mypy 检查 19 个源码、
|
||||
测试和实验脚本文件无问题，pytest 共 113 项测试通过。`requires-python` 仍以 `pyproject.toml` 声明的 Python 3.11 及以上为准；
|
||||
本次结果不等于已经在每个受支持版本上完成兼容性验证。
|
||||
|
||||
@@ -0,0 +1,545 @@
|
||||
# 0005：本地清洗实验运行与产物保存契约
|
||||
|
||||
## 状态
|
||||
|
||||
已批准并冻结（2026-08-22）。
|
||||
|
||||
本设计扩展 `0002` 中已划定的输入输出适配层，并落实 `0003` 中“由未来适配层决定是否保存成功输出”的边界。
|
||||
它不修改 `DocumentSnapshot`、`ProposedChange`、`Change`、`TransformResult` 或组件契约，也不改变
|
||||
`0004` 当时只读验证已经完成的历史事实。
|
||||
|
||||
`0004` 没有授权在那一轮验证中保存真实清洗产物，不等于永久禁止未来的本地实验输出。本设计批准后，
|
||||
新的实验可以在本设计的位置、保留周期和隐私边界内保存产物。
|
||||
|
||||
## 1. 问题与可观察现象
|
||||
|
||||
当前内存核心已经能返回清洗后 Markdown 和逐项 `Change`，但仓库还没有文件读取、安全输出、JSON 审计或
|
||||
人可读 diff。当前只能通过临时 Python 代码在内存中查看结果，会带来四个实际问题：
|
||||
|
||||
1. 评审者无法直接打开清洗后 Markdown；
|
||||
2. 没有统一 diff，难以确认“只改了批准内容”；
|
||||
3. `Change` 只在 Python 对象中，进程结束后无法复核组件、理由、原文和改后文本；
|
||||
4. 批量处理时无法追溯实际输入、工具版本、组件顺序和每份文档的结束状态。
|
||||
|
||||
真实清洗流程必须能保存结果，但不能因此让组件读写文件，也不能把 `failed` 或 `unstable` 的部分文本冒充
|
||||
成功输出。保存的 diff、`before` / `after` 和清洗后 Markdown 都可能还原真实原文，因此也不能当成普通日志
|
||||
提交到 Git。
|
||||
|
||||
## 2. 目标与非目标
|
||||
|
||||
### 2.1 目标
|
||||
|
||||
- 建立只服务本地评审的最小实验运行层；
|
||||
- 显式读取调用方列出的 Markdown 文件，不递归猜测数据集；
|
||||
- 继续使用同一个 `Pipeline.transform()` 完成清洗，不在适配层实现第二套规则；
|
||||
- 对每份成功文档保存独立的清洗后 Markdown、机器可读审计和人可读 diff；
|
||||
- 保存整次运行清单，使输入范围、工具环境、组件顺序、哈希、状态和产物位置可追溯；
|
||||
- 从文件级别区分 `success`、`failed` 和 `unstable`，不为后两者生成正式清洗文档；
|
||||
- 不覆盖、改名或移动任何输入；
|
||||
- 将含真实文本的产物限定为 Git 忽略的本地敏感数据。
|
||||
|
||||
### 2.2 非目标
|
||||
|
||||
- 不实现 Inspector、Finding、人工建议或审核状态；
|
||||
- 不实现 Profile 对象、TOML/YAML/JSON 配置文件或组件自动发现；
|
||||
- 不提供安装后稳定的公共 CLI、服务接口、CI 集成或 Web 界面；
|
||||
- 不原地覆盖、备份或恢复输入文件；
|
||||
- 不把 `partial_markdown` 写成清洗产物；
|
||||
- 不建设数据库、远程对象存储、任务调度或长期审计系统；
|
||||
- 不引入 Markdown parser、AST、新清洗规则或多轮执行；
|
||||
- 不定义真实数据适合公开、共享或长期归档的条件；
|
||||
- 不处理 PDF、DOCX、图片、转换器 JSON 或文件间关联。
|
||||
|
||||
## 3. 当前基础和不改变的边界
|
||||
|
||||
当前 `TransformResult` 已经包含持久化所需的主要运行事实：
|
||||
|
||||
| 对象 | 已有信息 | 本设计的处理 |
|
||||
| --- | --- | --- |
|
||||
| `ComponentInfo` | 组件标识、版本、参数、适用边界 | 写入运行清单 |
|
||||
| `Change` | 候选引用、理由、范围、`before`、`after`、批次前后哈希 | 写入文档审计 |
|
||||
| `RunError` | 组件、阶段、错误类型和安全说明 | 写入文档审计 |
|
||||
| `ResidualProposal` | 最终复查的有效残留修改 | 只写入 `failed` / `unstable` 审计 |
|
||||
| `TransformResult` | 状态、输入/当前哈希、修改、错误、成功或部分文本 | 决定可以生成哪些产物 |
|
||||
|
||||
本设计只在核心外补充文件身份、运行身份、环境和序列化信息。以下边界保持不变：
|
||||
|
||||
- 组件仍只接收 `DocumentSnapshot` 并返回 `ProposedChange`；
|
||||
- 组件不知道文件路径、运行 ID、输出目录或 JSON 格式；
|
||||
- `Pipeline` 仍只处理内存 Markdown；
|
||||
- Python 字符下标仍是修改位置的唯一权威；
|
||||
- reporter 不重新判断业务规则，只表示现有运行结果；
|
||||
- 文件适配层不把读写成功冒充成清洗成功。
|
||||
|
||||
## 4. 方案比较
|
||||
|
||||
### 4.1 继续只返回内存结果
|
||||
|
||||
不增加新契约，但每次真实实验都需要临时代码，结果无法留存，人也无法方便查看完整 Markdown 和 diff。
|
||||
不采用。
|
||||
|
||||
### 4.2 只保存清洗后 Markdown
|
||||
|
||||
人可以打开结果，但无法证明是哪个组件改了什么，也无法区分输入不同、组件不同还是工具不同。
|
||||
输出文件与内存审计会脱节。不采用。
|
||||
|
||||
### 4.3 建设完整 CLI、Profile 和批处理平台
|
||||
|
||||
能够一次解决对外运行，但当前只有一个无参数组件，配置、退出码、安装命令和公共兼容都没有足够消费者与失败样本。
|
||||
这会过早固定外部接口。不采用。
|
||||
|
||||
### 4.4 本地实验运行层加结构化产物
|
||||
|
||||
调用方显式提供文档和 `Pipeline`，外层只负责严格读取、调用核心、表示结果和写入新目录。它不提供稳定公共 CLI，
|
||||
也不引入配置语言。每次运行同时保存成功文本、JSON 审计和 diff。
|
||||
|
||||
采用此方案。它能满足当前 5 份论文的人工复核，同时不迫使核心、组件和未来对外工具提前承担未经验证的接口。
|
||||
|
||||
## 5. 总体边界与依赖方向
|
||||
|
||||
```text
|
||||
调用方显式列出文档和顺序
|
||||
│
|
||||
▼
|
||||
本地实验运行层
|
||||
│ │
|
||||
│ ├── 读取 UTF-8 Markdown
|
||||
│ └── 记录文档身份与环境
|
||||
▼
|
||||
Pipeline.transform(markdown)
|
||||
│
|
||||
▼
|
||||
TransformResult
|
||||
│
|
||||
▼
|
||||
reporter / 产物写入
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
JSON cleaned.md unified diff
|
||||
```
|
||||
|
||||
依赖必须保持从外向内：
|
||||
|
||||
```text
|
||||
本地实验脚本
|
||||
│
|
||||
▼
|
||||
experiment.py
|
||||
├────────► pipeline.py ──────► 当前内存核心
|
||||
├────────► reporting.py ─────► models.py
|
||||
└────────► artifact_store.py
|
||||
```
|
||||
|
||||
- `reporting.py` 可以依赖数据模型，不读文件，不调用组件；
|
||||
- `artifact_store.py` 只负责产物路径、权限、写入校验和原子发布，不调用流水线，不解释修改语义；
|
||||
- `experiment.py` 可以依赖 `Pipeline`、reporter 和 artifact store，负责输入预检与文档级批量编排；
|
||||
- 业务组件和内存核心不反向导入这三个模块；
|
||||
- 日期目录、文件名或未来存储位置的变化应限制在 artifact store，不改变 reporter 或流水线；
|
||||
- 实验脚本可以显式组装 ClinDB 流水线，但通用模块不得硬编码 `data/md/`、DOI 或论文缩写。
|
||||
|
||||
## 6. 输入契约
|
||||
|
||||
### 6.1 显式文档清单
|
||||
|
||||
调用方必须按顺序提供非空文档清单。每项至少包含：
|
||||
|
||||
- `document_id`：当前运行内唯一的稳定标识；
|
||||
- `source_path`：实际读取路径；
|
||||
- `source_label`：写入清单的可展示来源，用于避免无条件泄露绝对路径。
|
||||
|
||||
`document_id` 只允许小写 ASCII 字母、数字、`.`、`_` 和 `-`，必须以字母或数字开头，不得包含路径分隔符或
|
||||
`..` 路径段。文档 ID 重复、源路径重复、源文件不存在或不是普通文件时，整次运行在产生最终产物目录前失败。
|
||||
|
||||
第一个实际实验的文档 ID 固定为 `dmp`、`ejhf`、`jama`、`sim` 和 `springer`，输入只是 `data/md/` 中对应的
|
||||
5 份本地副本。本设计不授权对仓库外 GovDoc 数据保存清洗产物。
|
||||
|
||||
### 6.2 编码和文本保真
|
||||
|
||||
- 输入以二进制读取，再使用严格 UTF-8 解码；
|
||||
- 无效 UTF-8 使整次运行在预检阶段失败，不使用替换字符或自动猜测编码；
|
||||
- 不剔除 UTF-8 BOM，不规范化 Unicode，不转换换行，不补文件末换行；
|
||||
- 清洗后 Markdown 使用 UTF-8 严格编码，保留内存输出的所有字符。
|
||||
|
||||
核心快照哈希仍以 Markdown 字符串重新编码后的 UTF-8 字节为权威。适配层另外记录原始文件字节的 SHA-256；
|
||||
对有效 UTF-8 输入，两者应当一致。如果不一致，视为适配层错误并停止运行。
|
||||
|
||||
### 6.3 流水线来源
|
||||
|
||||
实验运行层接收调用方已显式建立的 `Pipeline`。它不根据文件名、内容或安装环境选择组件，也不在不同文档之间改变
|
||||
组件顺序。当前没有 Profile 身份；运行清单直接记录每次结果中已验证的有序 `ComponentInfo`。
|
||||
|
||||
## 7. 运行状态与批量语义
|
||||
|
||||
### 7.1 文档级别
|
||||
|
||||
每份文档独立调用同一个 `Pipeline`。文档结果直接沿用核心状态：
|
||||
|
||||
- `success`：可以保存正式清洗 Markdown、文档审计和 diff；
|
||||
- `failed`：只保存文档审计和错误，不持久化 `partial_markdown`；
|
||||
- `unstable`：只保存文档审计和残留候选，不持久化 `partial_markdown`。
|
||||
|
||||
对 `failed` 和 `unstable`，之前已经发生的 `Change` 仍必须写入审计，但不得生成名为 `cleaned.md`、`output.md`
|
||||
或类似正式输出的文件。
|
||||
|
||||
### 7.2 批量级别
|
||||
|
||||
单份文档的核心失败或不稳定不中断其他已预检文档。运行层继续按调用方给定的顺序处理，并在清单中保留每份文档的
|
||||
状态。整体状态使用以下优先级：
|
||||
|
||||
```text
|
||||
任一文档 failed → 整体 failed
|
||||
否则任一文档 unstable → 整体 unstable
|
||||
否则 → 整体 success
|
||||
```
|
||||
|
||||
这是实验汇总状态，不改变核心 `RunStatus`。整体失败时，其他文档的成功产物可以保留，但 `manifest.json`
|
||||
必须明确该批次并非全部成功。
|
||||
|
||||
### 7.3 预检与致命错误
|
||||
|
||||
在调用任何组件前，运行层必须完成：
|
||||
|
||||
1. 验证运行日期、UTC 偏移、运行 ID 和所有文档 ID；
|
||||
2. 验证输出根目录不会落入任何输入文件路径；
|
||||
3. 确认最终运行目录不存在；
|
||||
4. 读取所有输入字节、严格解码并计算哈希；
|
||||
5. 验证所有文档清单字段。
|
||||
|
||||
任一预检失败都使整次运行立即失败，不调用 `Pipeline`，不生成最终运行目录。产物写入、JSON 序列化或最终目录
|
||||
发布失败同样是整次运行的致命错误，不得把不完整临时目录报告为已完成实验。
|
||||
|
||||
## 8. 产物目录与发布
|
||||
|
||||
### 8.1 固定位置
|
||||
|
||||
仓库内本地实验产物固定放在：
|
||||
|
||||
```text
|
||||
artifacts/<run_date>/runs/<run_id>/
|
||||
```
|
||||
|
||||
`artifacts/` 已由 `.gitignore` 忽略。本设计不授权从 Git 忽略中移除该目录，也不允许通过 `git add -f`
|
||||
强制提交任何产物。
|
||||
|
||||
`run_date` 使用 `YYYY-MM-DD`，表示实验开始时本机时区中的日历日期。本地实验脚本在启动时只计算一次日期和
|
||||
UTC 偏移，再把它们显式传给运行层；运行层不在处理不同文档时重新读取日期。`run_date` 必须能够按 ISO 8601
|
||||
日历日期严格解析，日期目录不接受其他格式。
|
||||
|
||||
`run_id` 由调用方显式提供，适用与 `document_id` 相同的安全字符规则。不使用当前时间、随机数或文件名暗中生成
|
||||
运行 ID。同一日期下的目标运行目录已存在时必须拒绝运行，不覆盖、合并或自动加后缀。同一个 `run_id` 可以在
|
||||
不同日期下再次使用，两次运行仍由完整日期路径区分。
|
||||
|
||||
### 8.2 目录结构
|
||||
|
||||
```text
|
||||
artifacts/
|
||||
└── <run_date>/
|
||||
└── runs/
|
||||
└── <run_id>/
|
||||
├── manifest.json
|
||||
└── documents/
|
||||
└── <document_id>/
|
||||
├── result.json
|
||||
├── cleaned.md # 仅 success
|
||||
└── changes.diff # 仅 success
|
||||
```
|
||||
|
||||
所有 `success` 文档都生成 `cleaned.md` 和 `changes.diff`。即使零修改，`cleaned.md` 仍完整保存成功输出，
|
||||
`changes.diff` 是长度为零的文件。这使每份成功文档的产物结构一致。
|
||||
|
||||
`failed` 和 `unstable` 文档目录只包含 `result.json`。不创建隐含部分结果的 Markdown 或 diff。
|
||||
|
||||
### 8.3 原子发布
|
||||
|
||||
artifact store 先在 `artifacts/<run_date>/runs/` 下创建当次运行专用的临时目录，完成全部文件写入、
|
||||
哈希校验和清单校验后，
|
||||
再在同一文件系统内将它重命名为最终 `<run_id>` 目录。
|
||||
|
||||
- 发布前的临时目录不是成功产物；
|
||||
- 任一写入或校验失败时尝试清理本次专用临时目录；
|
||||
- 不对已存在的最终目录使用替换语义；
|
||||
- 本设计只保证“不发布已知不完整的运行目录”，不承诺跨平台断电耐久性。
|
||||
|
||||
## 9. 持久化事实与 JSON 契约
|
||||
|
||||
### 9.1 事实权威
|
||||
|
||||
- `manifest.json` 是整次运行身份、环境、流水线、文档索引和汇总的权威；
|
||||
- 每份文档的 `result.json` 是该文档状态、哈希、实际修改、错误和残留候选的权威；
|
||||
- `cleaned.md` 是成功文本内容的权威；
|
||||
- `changes.diff` 是从输入和成功输出派生的人工评审视图，不是第二份修改记录。
|
||||
|
||||
清单中的计数和路径是从文档结果派生的索引，发布前必须校验与对应 `result.json` 和文件存在性一致。
|
||||
|
||||
### 9.2 JSON 通用表示
|
||||
|
||||
两类 JSON 都使用：
|
||||
|
||||
- UTF-8，不写 BOM；
|
||||
- 根对象的 `schema_version` 固定为整数 `1`；
|
||||
- 两空格缩进；
|
||||
- 不把非 ASCII 字符转成反斜杠加 `u` 的 Unicode 转义形式；
|
||||
- 文件末有一个 `\n`；
|
||||
- 数组顺序保留调用方文档顺序、组件顺序和核心修改顺序。
|
||||
|
||||
JSON 中的字段名和枚举值使用英文标识符。组件的中文修改理由和 `before` / `after` 保留原文。
|
||||
|
||||
### 9.3 `manifest.json`
|
||||
|
||||
运行清单至少包含：
|
||||
|
||||
```text
|
||||
schema_version
|
||||
run
|
||||
run_id
|
||||
run_date
|
||||
utc_offset
|
||||
status
|
||||
started_at_utc
|
||||
completed_at_utc
|
||||
retention_until
|
||||
tool
|
||||
name
|
||||
package_version
|
||||
python_version
|
||||
platform
|
||||
git_commit # 无法取得时为 null
|
||||
git_dirty # 无法取得时为 null
|
||||
pipeline
|
||||
components[] # 实际有序 ComponentInfo
|
||||
component_id
|
||||
version
|
||||
parameters
|
||||
applicability
|
||||
documents[]
|
||||
document_id
|
||||
source_label
|
||||
status
|
||||
input_sha256
|
||||
current_sha256
|
||||
change_count
|
||||
result_path
|
||||
cleaned_path # 非 success 为 null
|
||||
diff_path # 非 success 为 null
|
||||
summary
|
||||
document_count
|
||||
success_count
|
||||
failed_count
|
||||
unstable_count
|
||||
change_count
|
||||
```
|
||||
|
||||
`run_date` 必须等于产物路径中的日期。`utc_offset` 使用 `+HH:MM` 或 `-HH:MM`，说明计算该日期时的本机
|
||||
UTC 偏移。`started_at_utc`、`completed_at_utc` 和 `retention_until` 使用带 `Z` 的 UTC ISO 8601 字符串。
|
||||
时间、平台和 Git 信息属于外层运行证据，不进入组件参数，也不影响核心结果的确定性。清单不读取或保存环境变量、
|
||||
用户名、主机名或其他可能包含秘密的全局环境信息。
|
||||
|
||||
`pipeline.components` 必须来自本次文档结果中的已验证元数据。同一次运行中各文档的组件元数据不一致时，
|
||||
视为运行层错误，不发布最终目录。
|
||||
|
||||
### 9.4 `result.json`
|
||||
|
||||
文档审计至少包含：
|
||||
|
||||
```text
|
||||
schema_version
|
||||
document
|
||||
document_id
|
||||
source_label
|
||||
status
|
||||
input_sha256
|
||||
current_sha256
|
||||
changes[]
|
||||
component_id
|
||||
component_version
|
||||
component_position
|
||||
proposal_ref
|
||||
component_position
|
||||
snapshot_sha256
|
||||
proposal_index
|
||||
edit_index
|
||||
reason
|
||||
span
|
||||
start
|
||||
end
|
||||
location # 派生定位，不用于应用修改
|
||||
line # 1-based，相对 before_sha256 快照
|
||||
column # 1-based，相对 before_sha256 快照
|
||||
before
|
||||
after
|
||||
before_sha256
|
||||
after_sha256
|
||||
errors[]
|
||||
component_id
|
||||
component_version
|
||||
component_position
|
||||
stage
|
||||
error_type
|
||||
message
|
||||
residual_proposals[]
|
||||
component_id
|
||||
component_version
|
||||
component_position
|
||||
proposal_ref
|
||||
proposal
|
||||
snapshot_sha256
|
||||
reason
|
||||
edits[]
|
||||
snapshot_sha256
|
||||
span
|
||||
start
|
||||
end
|
||||
expected_text
|
||||
replacement
|
||||
output
|
||||
cleaned_path # 非 success 为 null
|
||||
diff_path # 非 success 为 null
|
||||
```
|
||||
|
||||
`location` 由 reporter 基于对应 `before_sha256` 快照派生，只便于人查看。`span.start` / `span.end`
|
||||
仍是唯一修改权威。当多个组件产生中间快照时，reporter 必须按组件批次从输入重放已记录修改，每次都校验前后哈希；
|
||||
无法重放时不得发布产物。
|
||||
|
||||
`result.json` 不内嵌完整 `output_markdown` 或 `partial_markdown`。成功全文只在 `cleaned.md` 中保存；
|
||||
失败和不稳定的部分全文不落盘。`changes`、`residual_proposals` 中的局部原文仍属于完整本地审计的一部分。
|
||||
|
||||
## 10. diff 契约
|
||||
|
||||
`changes.diff` 使用原始输入和最终成功输出生成 unified diff：
|
||||
|
||||
- 旧文件标签固定为 `a/<document_id>.md`；
|
||||
- 新文件标签固定为 `b/<document_id>.md`；
|
||||
- 不把绝对路径和时间戳写入 diff 头；
|
||||
- 上下文固定为 3 行；
|
||||
- diff 自身使用 `\n` 作为报告换行，不表示清洗后 Markdown 被转换为 `\n`；
|
||||
- 零修改的成功文档产生空 diff；
|
||||
- diff 只用于人工评审，不用于重放或应用修改。
|
||||
|
||||
统一 diff 只展示输入到最终输出的总变化。如果需要查看某个组件的原因和中间快照，以 `result.json`
|
||||
中的 `Change` 审计为准。
|
||||
|
||||
## 11. 隐私、权限和保留周期
|
||||
|
||||
### 11.1 数据级别
|
||||
|
||||
本设计生成的所有产物都按“本地敏感实验数据”处理，包括：
|
||||
|
||||
- `cleaned.md` 中的完整文档；
|
||||
- `changes.diff` 中的原文上下文；
|
||||
- `result.json` 中的 `before`、`after` 和残留候选；
|
||||
- `manifest.json` 中可能暴露数据集结构的文档名和来源标签。
|
||||
|
||||
这些产物：
|
||||
|
||||
- 只能保存在本机 `artifacts/` 下；
|
||||
- 不得提交、推送、发布、上传或复制到 Wiki；
|
||||
- 终端默认只输出运行 ID、状态、计数和产物目录，不输出 `before`、`after` 或 diff 片段；
|
||||
- 实验结束后不得自动拷贝到其他仓库或用户目录。
|
||||
|
||||
第一版创建运行目录时将目录权限设为只有当前用户可读、写和进入，普通产物文件只有当前用户可读写。
|
||||
如果平台不支持这些权限语义，实验必须明确报错，不静默降级为宽松权限。
|
||||
|
||||
### 11.2 保留周期
|
||||
|
||||
本地产物默认保留 30 个日历日。`manifest.json` 记录根据 `completed_at_utc` 计算的 `retention_until`。
|
||||
|
||||
第一版不实现自动删除，避免在没有人工确认时执行破坏性操作。到期产物只标记为待清理；删除前必须由用户确认
|
||||
具体运行目录。需要超过 30 天保留、跨机共享或长期归档时，必须先单独确认保存位置和数据边界。
|
||||
|
||||
## 12. 实现结构
|
||||
|
||||
批准后允许新增：
|
||||
|
||||
```text
|
||||
src/mdpolish/
|
||||
├── artifact_store.py # 日期路径、权限、写入校验和原子产物发布
|
||||
├── experiment.py # 显式文件输入、预检和批量运行
|
||||
└── reporting.py # JSON 表示、位置派生和 unified diff
|
||||
scripts/
|
||||
└── run_clindb_arxiv_experiment.py # 仓库内已知 5 份论文的本地运行脚本
|
||||
tests/
|
||||
├── test_artifact_store.py
|
||||
├── test_experiment.py
|
||||
└── test_reporting.py
|
||||
```
|
||||
|
||||
- 三个新模块不从顶层 `mdpolish.__init__` 导出，暂不承诺稳定公共 API；
|
||||
- 本地脚本显式创建只包含 `ArxivSubmissionStampComponent` 的 `Pipeline`；
|
||||
- 脚本启动时捕获一次本机日期、UTC 偏移和 UTC 开始时间，不在文档循环中重新计算日期；
|
||||
- 脚本输入是 `data/md/` 中的 5 份缩写文件，输出为 `artifacts/<run_date>/runs/<run_id>/`；
|
||||
- 脚本可以接收运行 ID，但不是通用文件清洗 CLI，不注册 `project.scripts`；
|
||||
- 精确命令和实际运行步骤在实现并验证后进入 `guides/`，README 只链接当前权威，不在多处复制。
|
||||
|
||||
第一版继续只使用 Python 标准库。不修改包安装命令，不增加运行依赖。
|
||||
|
||||
## 13. 测试和验收
|
||||
|
||||
### 13.1 合成测试
|
||||
|
||||
自动测试只使用 `tmp_path` 和小型虚构 Markdown，不读取或复制真实论文。至少覆盖：
|
||||
|
||||
- 空文档、中文、组合 Unicode、UTF-8 BOM、`\n`、`\r\n` 和无末尾换行；
|
||||
- 无效 UTF-8、缺失文件、非普通文件、重复文档 ID、重复源路径和非法运行 ID 在预检失败；
|
||||
- 非法日期、日期路径与 manifest 不一致、同日重复运行 ID 明确失败；
|
||||
- 目标运行目录已存在时拒绝，不覆盖其中任何文件；
|
||||
- `success` 生成三类文档产物，零修改时仍生成完整 `cleaned.md` 和空 diff；
|
||||
- `failed` / `unstable` 只生成 `result.json`，不保存部分全文；
|
||||
- 一份文档核心失败后继续处理其他文档，整体状态按第 7.2 节汇总；
|
||||
- `result.json` 完整表示候选引用、修改理由、范围、位置、`before`、`after` 和哈希；
|
||||
- 多组件修改可以重放中间快照并正确派生行列，哈希不一致时拒绝发布；
|
||||
- unified diff 使用逻辑文档名，不包含绝对路径或时间；
|
||||
- 输出 Markdown 重新读取后的哈希等于核心 `current_sha256`；
|
||||
- 运行前后输入文件的字节和哈希不变；
|
||||
- JSON 编码、schema 版本、顺序、文件末换行和清单计数符合契约；
|
||||
- 产物目录及文件权限不宽于第 11 节的边界；
|
||||
- 产物写入中途失败时不发布最终运行目录。
|
||||
|
||||
### 13.2 本地 5 份论文验收
|
||||
|
||||
实现并通过合成测试后，允许对 `data/md/` 中 5 份本地副本运行实验脚本，并在本地
|
||||
`artifacts/<run_date>/runs/<run_id>/` 保存完整产物。验收必须确认：
|
||||
|
||||
1. 清单列出 `dmp`、`ejhf`、`jama`、`sim` 和 `springer` 共 5 份输入；
|
||||
2. 流水线只有 `paper.arxiv_submission_stamp` `1.0.0`，参数为空；
|
||||
3. 5 份文档都为 `success`；
|
||||
4. `sim` 和 `springer` 各有 1 条删除，其余文档零修改，合计 2 条 `Change`；
|
||||
5. 两条修改的组件、理由、原文、空替换、位置和前后哈希都可从 `result.json` 查看；
|
||||
6. `sim` 和 `springer` 的 diff 只包含已批准的提交戳删除，Springer 合法参考文献保留；
|
||||
7. 所有 `cleaned.md` 与对应成功哈希一致；
|
||||
8. 5 份输入文件在运行前后逐字节不变；
|
||||
9. 产物受 Git 忽略，提交前 `git status` 不列出任何实验产物。
|
||||
|
||||
真实实验产物只用于本地人工评审，不进入合成测试期望值、Wiki、Git 提交或终端输出。
|
||||
|
||||
## 14. 风险与代价
|
||||
|
||||
- **JSON 过早成为契约：** 第一版只服务本地实验，不承诺向后兼容；字段语义变化时必须增加
|
||||
`schema_version`，不静默改变旧文件语义。
|
||||
- **完整审计会复制真实片段：** `before`、`after` 和 diff 有意保留原文，换取可人工复核；代价是所有产物都必须
|
||||
按敏感数据处理。
|
||||
- **批量中部分文档成功：** 这便于查看每份输入，但调用方必须检查整体和文档状态，不能因为目录里有部分
|
||||
`cleaned.md` 就宣称整批成功。
|
||||
- **行列派生需要重放修改：** 它能在不改核心模型的情况下便于人定位，但增加 reporter 的复杂度。重放过程必须
|
||||
完全验证哈希，失败就拒绝发布。
|
||||
- **时间和环境使 manifest 不再字节级确定：** 这些是复现实验必需的外层证据，不影响相同文本和流水线的核心结果
|
||||
确定性。
|
||||
- **严格权限会降低跨平台便携性：** 第一版是本机 Linux 实验能力，不借本设计声称 Windows 或共享盘已支持。
|
||||
- **固定 30 天只是本地实验保留约定：** 第一版不自动删除，所以仍需要用户定期确认和清理过期目录。
|
||||
|
||||
## 15. 批准后的实施边界
|
||||
|
||||
批准本设计后，只授权：
|
||||
|
||||
1. 新增第 12 节列出的模块、本地脚本和测试；
|
||||
2. 按第 6 至 11 节实现显式输入、核心调用、JSON、diff、原子发布和保留信息；
|
||||
3. 对 `data/md/` 的 5 份本地论文副本运行只含 arXiv 组件的实验；
|
||||
4. 将完整本地产物保存到 Git 忽略的 `artifacts/<run_date>/runs/<run_id>/`；
|
||||
5. 实现后根据真实行为更新 README、对应 `explanation/` 和经验证的 `guides/`。
|
||||
|
||||
批准不授权：
|
||||
|
||||
- 修改或覆盖任何输入文件；
|
||||
- 对 `/home/lihaoze/gov_test_data` 或其他仓库外数据保存产物；
|
||||
- 把实验产物加入 Git、Wiki、其他仓库、云存储或外部系统；
|
||||
- 实现原地覆盖、公共 CLI、Profile、Inspector、parser、多轮执行或其他清洗组件；
|
||||
- 提交、推送、创建 PR 或发布。
|
||||
@@ -54,7 +54,7 @@ arXiv:<新版数字编号和版本> [<ASCII 分类>] <日> <英文月份缩写>
|
||||
合成测试覆盖严格匹配、反向引用、首行/中间/末行、三种行尾、多个命中、围栏中仍删除、审计字段、确定性和
|
||||
第二次运行零修改。测试只使用短小的虚构字符串，不含真实论文片段。
|
||||
|
||||
2026-08-22 又对本地 5 份 ClinDB-ReviewBench Markdown 做了只读、纯内存复核：
|
||||
2026-08-22 先对本地 5 份 ClinDB-ReviewBench Markdown 做了只读、纯内存复核：
|
||||
|
||||
| 复核项 | 结果 |
|
||||
| --- | --- |
|
||||
@@ -64,13 +64,20 @@ arXiv:<新版数字编号和版本> [<ASCII 分类>] <日> <英文月份缩写>
|
||||
| 第二次运行 | 5 份合计 0 条修改 |
|
||||
| 源文件复读 | 5/5 与处理前内存内容一致，没有回写 |
|
||||
|
||||
本次没有保存清洗后 Markdown，没有把真实原文复制进测试、日志或仓库。安装、静态检查和完整测试命令仍只在根目录
|
||||
这次早期复核没有保存清洗后 Markdown，也没有把真实原文复制进测试、日志或仓库。它是 `0004` 当时验证边界的
|
||||
历史事实。
|
||||
|
||||
`0005` 批准本地产物机制后，同日又完成一次保存型实验：5 份文档全部为 `success`，仍然只修改 sim 和 springer
|
||||
各一处；输出哈希全部匹配，输入运行前后字节不变，Springer 两处合法引用仍保留。产物位于
|
||||
`artifacts/2026-08-22/runs/clindb-arxiv-stamp-artifacts-v1/`，只在本机保留并受 Git 忽略。具体产物结构和验证结果见
|
||||
[`local-experiment-artifacts.md`](local-experiment-artifacts.md)。安装、静态检查和完整测试命令仍只在根目录
|
||||
[`README.md`](../../README.md#当前可用检查) 维护。
|
||||
|
||||
## 6. 剩余边界
|
||||
|
||||
这个组件只证明第一条严格删除规则能够在公共核心上闭环，不表示论文已经清洗完成。HTML 实体、Word 批注、手稿
|
||||
行号、断词、表格和参考文献间距仍未实现；文件输出、profile 和批处理也不存在。
|
||||
行号、断词、表格和参考文献间距仍未实现。当前只有固定输入和固定组件的本地实验输出；通用文件接口、profile、
|
||||
公共 CLI 和通用批处理仍不存在。
|
||||
|
||||
如果出现新的提交戳格式，默认行为是保留。必须先补充真实证据、反向样例和 design，再决定是否放宽模式，不能为了
|
||||
提高命中数量直接修改正则表达式。
|
||||
|
||||
@@ -128,8 +128,10 @@
|
||||
同一候选修改中的多条记录共享候选引用，同一组件批次中的所有记录共享批次前后哈希。记录只描述已经发生的修改；
|
||||
验证失败或最终复查中没有执行的候选不会冒充实际改动。
|
||||
|
||||
这些内容当前只存在于内存返回值中。仓库没有 reporter、审计文件格式或日志持久化，调用方也不能默认把失败结果中
|
||||
的部分文本写回原文件。
|
||||
这些内容在核心中只存在于内存返回值中。核心外已经有一个获批的本地实验 reporter，可以校验修改链并把审计、
|
||||
成功 Markdown 和 diff 保存到私有产物目录；机制见
|
||||
[`local-experiment-artifacts.md`](local-experiment-artifacts.md)。这没有改变核心接口，也不允许把失败结果中的部分文本
|
||||
写成正式输出或写回原文件。
|
||||
|
||||
## 8. 当前验证和剩余边界
|
||||
|
||||
@@ -146,7 +148,8 @@
|
||||
- 除严格删除 arXiv 提交边栏戳外的其他论文、GovDoc 或 HTML 表格清洗组件；
|
||||
- 独立文档检查、人工建议或审核流程；
|
||||
- Markdown parser、AST 或共享业务中间表示；
|
||||
- 文件读写、CLI、批处理、项目 profile 格式和生产集成；
|
||||
- 审计结果的长期存储或脱敏输出协议。
|
||||
- 通用文件输入、公共 CLI、通用批处理、项目 profile 格式和生产集成；
|
||||
- 审计结果的长期存储、自动清理或脱敏输出协议。
|
||||
|
||||
这些边界中的任何一项要进入实现，都需要先用新的 design 明确语义、代价和验收方式。
|
||||
当前只有一个固定数据和组件组合的本地实验脚本，不构成上述公共能力。这些边界中的任何一项要进入实现，都需要先用
|
||||
新的 design 明确语义、代价和验收方式。
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
# 本地清洗实验如何保存 Markdown、审计和 diff
|
||||
|
||||
## 1. 它解决什么问题
|
||||
|
||||
内存流水线可以安全地产生 `TransformResult`，但进程结束后，评审者仍需要打开清洗后的完整 Markdown、查看总 diff，
|
||||
并追溯每条修改属于哪个组件、为什么修改、修改前后是什么。
|
||||
|
||||
当前本地实验层把这些结果保存到独立目录，同时继续保持三个边界：
|
||||
|
||||
- 组件和 `Pipeline` 仍然不读写文件；
|
||||
- 输入文件永远不被覆盖；
|
||||
- 只有 `success` 文档才产生正式的清洗后 Markdown。
|
||||
|
||||
已经实现的范围来自已批准的
|
||||
[`0005-local-experiment-runner-and-artifacts.md`](../design/0005-local-experiment-runner-and-artifacts.md)。
|
||||
精确字段、校验和函数签名以 `src/mdpolish/` 中的代码与测试为准。
|
||||
|
||||
## 2. 三层怎样解耦
|
||||
|
||||
```text
|
||||
experiment.py
|
||||
├── pipeline.py 只负责内存清洗
|
||||
├── reporting.py 只负责 JSON、行列和 unified diff
|
||||
└── artifact_store.py 只负责日期目录、权限和原子发布
|
||||
```
|
||||
|
||||
- `experiment.py` 严格读取调用方显式列出的 UTF-8 Markdown，逐份调用同一个 `Pipeline`；
|
||||
- `reporting.py` 重放并校验 `Change` 的快照链，再生成机器可读审计和人可读 diff；
|
||||
- `artifact_store.py` 不理解清洗规则，只把已经生成的字节写入私有临时目录，校验后一次性发布。
|
||||
|
||||
因此，新增组件不会改变文件层；调整目录布局不会影响清洗和报告；修改 JSON 或 diff 时也不需要碰流水线。
|
||||
ClinDB 的 5 份论文和 arXiv 组件组合只存在于仓库内实验脚本，通用模块没有硬编码论文名。
|
||||
|
||||
## 3. 输入怎样保持原样
|
||||
|
||||
实验层先完成整批预检：
|
||||
|
||||
1. 验证日期、运行 ID、文档 ID 和来源标签；
|
||||
2. 确认所有路径存在、是普通文件且没有重复；
|
||||
3. 以二进制读取全部输入；
|
||||
4. 使用严格 UTF-8 解码；
|
||||
5. 比较原始字节 SHA-256 与核心 Markdown SHA-256。
|
||||
|
||||
读取过程不剔除 BOM，不规范化 Unicode，不转换 `\n`、`\r\n` 或文件末尾换行。全部文档运行结束后，
|
||||
实验层再次读取每份输入并比较原始字节和哈希；任何变化都会阻止产物发布。
|
||||
|
||||
输入缺失、不是普通文件、不是有效 UTF-8 或清单冲突属于整批预检失败。这时不调用流水线，也不创建最终运行目录。
|
||||
|
||||
## 4. 状态怎样决定产物
|
||||
|
||||
每份文档独立运行，某一份发生核心错误不会阻止其他已经预检的文档继续产生结果。
|
||||
|
||||
| 文档状态 | `result.json` | `cleaned.md` | `changes.diff` |
|
||||
| --- | --- | --- | --- |
|
||||
| `success` | 有 | 有 | 有；零修改时为空文件 |
|
||||
| `failed` | 有 | 无 | 无 |
|
||||
| `unstable` | 有 | 无 | 无 |
|
||||
|
||||
`failed` 和 `unstable` 的审计仍保留已经实际发生的 `Change`、错误或残留候选，但不持久化
|
||||
`partial_markdown`，避免半成品看起来像正式结果。
|
||||
|
||||
整批状态按 `failed`、`unstable`、`success` 的优先级汇总。即使其他文档有成功产物，只要一份失败，
|
||||
`manifest.json` 就会把整批标为 `failed`。
|
||||
|
||||
## 5. 产物怎样组织
|
||||
|
||||
```text
|
||||
artifacts/
|
||||
└── <YYYY-MM-DD>/
|
||||
└── runs/
|
||||
└── <run_id>/
|
||||
├── manifest.json
|
||||
└── documents/
|
||||
└── <document_id>/
|
||||
├── result.json
|
||||
├── cleaned.md
|
||||
└── changes.diff
|
||||
```
|
||||
|
||||
日期取实验启动时本机时区中的日历日期。运行 ID 由调用方显式提供；同一日期下已经存在同名目录时拒绝覆盖。
|
||||
|
||||
`manifest.json` 是整次运行的索引，记录：
|
||||
|
||||
- 开始、完成和到期时间；
|
||||
- 本机日期及 UTC 偏移；
|
||||
- mdpolish、Python、平台和 Git 状态；
|
||||
- 实际组件顺序、版本和参数；
|
||||
- 每份输入的来源标签、前后哈希、状态、修改数量和产物相对路径；
|
||||
- 整批成功、失败、不稳定和修改数量。
|
||||
|
||||
每份 `result.json` 保存实际修改、错误与残留候选。每条修改包含组件、候选引用、理由、Python 字符范围、
|
||||
1-based 行列、`before`、`after` 和批次前后哈希。完整成功文本只存在于 `cleaned.md`。
|
||||
|
||||
`changes.diff` 是原始输入到最终成功输出的 unified diff，只用于人工查看。它不包含绝对路径或时间戳，
|
||||
也不是修改重放的权威；机器审计仍以 `result.json` 为准。
|
||||
|
||||
## 6. 为什么 reporter 要重放修改
|
||||
|
||||
第二个组件看到的是第一个组件修改后的快照，因此后续 `Change.span` 不一定对应最初输入。为了生成准确行列，
|
||||
reporter 从输入开始，按组件批次重放修改：
|
||||
|
||||
1. 当前文本哈希必须等于该批次 `before_sha256`；
|
||||
2. 每条范围内的原文必须等于 `before`；
|
||||
3. 同一批次按核心相同的从后向前顺序应用；
|
||||
4. 结果哈希必须等于 `after_sha256`；
|
||||
5. 全部批次完成后必须等于 `TransformResult.current_sha256`。
|
||||
|
||||
任何一步不一致都说明内存结果、reporter 或调用方式违反契约，整次运行不会发布最终目录。行列只是方便人查看的
|
||||
派生信息，修改权威仍是快照绑定的 Python 字符范围。
|
||||
|
||||
## 7. 文件怎样安全发布
|
||||
|
||||
artifact store 先在同一日期的 `runs/` 下建立本次专用临时目录。所有文件写入后都会重新读取校验，
|
||||
成功 Markdown 还要再次核对输出 SHA-256，manifest 的身份、状态、计数和路径也必须与各文档审计及实际文件一致。
|
||||
只有全部文件、清单和权限都通过，临时目录才会在 `runs/` 目录协作锁内重新检查目标，并原子重命名为最终运行 ID。
|
||||
|
||||
当前 Linux 本地实现使用：
|
||||
|
||||
- 目录权限 `0700`；
|
||||
- 文件权限 `0600`；
|
||||
- 已存在的目标目录拒绝覆盖；
|
||||
- 写入中途失败时不发布最终目录。
|
||||
|
||||
这保证不会发布已知不完整的结果，但不承诺跨平台断电耐久性或网络文件系统语义。
|
||||
|
||||
## 8. 隐私和保留边界
|
||||
|
||||
`cleaned.md`、diff 和 JSON 审计都可能包含真实原文，因此整个 `artifacts/` 都是本地敏感数据：
|
||||
|
||||
- 受 Git 忽略；
|
||||
- 不进入 Wiki、提交、推送或外部系统；
|
||||
- 终端只显示状态、计数和目录；
|
||||
- 默认保留 30 个日历日；
|
||||
- manifest 记录 `retention_until`；
|
||||
- 第一版不自动删除，到期后仍需用户确认具体目录再清理。
|
||||
|
||||
当前只批准对 `data/md/` 中 5 份论文副本保存产物。仓库外 GovDoc 和其他真实数据没有因此获得输出授权。
|
||||
|
||||
## 9. 已完成的真实验证
|
||||
|
||||
2026-08-22 使用 `paper.arxiv_submission_stamp` `1.0.0` 对 5 份本地论文副本完成一次保存型实验：
|
||||
|
||||
| 项目 | 结果 |
|
||||
| --- | --- |
|
||||
| 运行 ID | `clindb-arxiv-stamp-artifacts-v1` |
|
||||
| 输出位置 | `artifacts/2026-08-22/runs/clindb-arxiv-stamp-artifacts-v1/` |
|
||||
| 文档状态 | 5/5 `success` |
|
||||
| 实际修改 | `sim` 1 条、`springer` 1 条，其余 0 条 |
|
||||
| 修改位置 | `sim` 1:1、`springer` 18:1 |
|
||||
| 合法反向样例 | Springer 两处 `arXiv preprint arXiv:` 均保留 |
|
||||
| 输出校验 | 5/5 `cleaned.md` 与 `current_sha256` 一致 |
|
||||
| 输入只读 | 5/5 运行前后字节和哈希不变 |
|
||||
| 权限 | 全部运行目录 `0700`，产物文件 `0600` |
|
||||
|
||||
实际运行方法见
|
||||
[`run-local-clindb-arxiv-experiment.md`](../guides/run-local-clindb-arxiv-experiment.md)。
|
||||
@@ -0,0 +1,129 @@
|
||||
# 运行本地 ClinDB arXiv 清洗实验
|
||||
|
||||
## 1. 适用范围
|
||||
|
||||
本指南只运行仓库内已经批准的本地实验脚本：
|
||||
|
||||
- 输入：`data/md/` 中的 `dmp.md`、`ejhf.md`、`jama.md`、`sim.md` 和 `springer.md`；
|
||||
- 流水线：只包含 `paper.arxiv_submission_stamp` `1.0.0`；
|
||||
- 输出：`artifacts/<YYYY-MM-DD>/runs/<run_id>/`；
|
||||
- 输入只读，不覆盖原文件；
|
||||
- 不处理 `/home/lihaoze/gov_test_data`。
|
||||
|
||||
本指南于 2026-08-22 在 Python 3.13.11 环境实际验证。
|
||||
|
||||
## 2. 前置条件
|
||||
|
||||
在仓库根目录执行，并确认隔离环境和基础检查可用：
|
||||
|
||||
```bash
|
||||
.venv/bin/python --version
|
||||
.venv/bin/ruff check .
|
||||
.venv/bin/mypy src tests scripts/run_clindb_arxiv_experiment.py
|
||||
.venv/bin/pytest
|
||||
```
|
||||
|
||||
确认 5 份本地输入存在：
|
||||
|
||||
```bash
|
||||
find data/md -maxdepth 1 -type f -name '*.md' -printf '%f\n' | sort
|
||||
```
|
||||
|
||||
预期看到：
|
||||
|
||||
```text
|
||||
dmp.md
|
||||
ejhf.md
|
||||
jama.md
|
||||
sim.md
|
||||
springer.md
|
||||
```
|
||||
|
||||
## 3. 运行实验
|
||||
|
||||
为本次实验人工选择一个小写运行 ID。同一天已经使用过的 ID 不能覆盖；需要重跑时换一个新 ID。
|
||||
|
||||
```bash
|
||||
.venv/bin/python scripts/run_clindb_arxiv_experiment.py \
|
||||
--run-id clindb-arxiv-stamp-review
|
||||
```
|
||||
|
||||
成功时终端只显示运行 ID、状态、文档数、修改数和产物目录，例如：
|
||||
|
||||
```text
|
||||
run_id=clindb-arxiv-stamp-review
|
||||
status=success
|
||||
documents=5
|
||||
changes=2
|
||||
artifacts=/.../mdpolish/artifacts/<YYYY-MM-DD>/runs/clindb-arxiv-stamp-review
|
||||
```
|
||||
|
||||
终端不会打印论文原文或 diff。
|
||||
|
||||
## 4. 查看结果
|
||||
|
||||
进入终端输出的运行目录。目录结构为：
|
||||
|
||||
```text
|
||||
manifest.json
|
||||
documents/
|
||||
├── dmp/
|
||||
│ ├── result.json
|
||||
│ ├── cleaned.md
|
||||
│ └── changes.diff
|
||||
├── ejhf/
|
||||
├── jama/
|
||||
├── sim/
|
||||
└── springer/
|
||||
```
|
||||
|
||||
先看 `manifest.json` 的整批状态和汇总，再查看各文档：
|
||||
|
||||
- `cleaned.md`：成功清洗后的完整 Markdown；
|
||||
- `changes.diff`：输入到成功输出的人工对比；
|
||||
- `result.json`：组件、理由、位置、`before`、`after` 和哈希等机器审计。
|
||||
|
||||
`dmp`、`ejhf` 和 `jama` 当前应为零修改，diff 是空文件；`sim` 和 `springer` 当前各有一条删除。
|
||||
|
||||
## 5. 判断成功
|
||||
|
||||
本轮验收口径是：
|
||||
|
||||
- manifest 整体状态为 `success`；
|
||||
- 5 份文档全部为 `success`；
|
||||
- 合计 2 条修改；
|
||||
- `sim` 修改位置为 1:1；
|
||||
- `springer` 修改位置为 18:1；
|
||||
- Springer 两条合法 `arXiv preprint arXiv:` 参考文献保留；
|
||||
- 每份 `cleaned.md` 的 SHA-256 等于对应 `result.json.current_sha256`；
|
||||
- 输入文件运行前后不变。
|
||||
|
||||
只看到运行目录存在不等于成功，必须先检查 manifest 和文档状态。
|
||||
|
||||
## 6. 常见失败
|
||||
|
||||
### 运行目录已经存在
|
||||
|
||||
脚本拒绝覆盖同一日期下的同名运行目录。选择新的 `--run-id`，不要删除或覆盖旧目录来绕过检查。
|
||||
|
||||
### 输入缺失或不是 UTF-8
|
||||
|
||||
整批预检会失败，不运行任何组件，也不发布最终目录。先确认 `data/md/` 中 5 份文件存在且未被修改。
|
||||
|
||||
### 状态为 `failed` 或 `unstable`
|
||||
|
||||
对应文档只会生成 `result.json`，不会生成 `cleaned.md` 或 diff。查看错误或残留候选，不要把其他文档的部分成功
|
||||
当成整批成功。
|
||||
|
||||
### Snap 版本的 `jq` 报权限错误
|
||||
|
||||
产物目录权限是 `0700`。某些 Snap 沙箱工具不能进入私有目录，即使当前用户拥有权限。可直接用编辑器查看 JSON，
|
||||
或使用当前虚拟环境中的 Python 读取；不要为了兼容受限工具放宽产物权限。
|
||||
|
||||
## 7. 数据边界
|
||||
|
||||
产物包含完整论文和原文片段，只能保存在本机 Git 忽略的 `artifacts/`。不得执行 `git add -f`，不得复制到 Wiki、
|
||||
其他仓库、云存储或外部系统。
|
||||
|
||||
`manifest.json` 中的 `retention_until` 是默认 30 天到期时间。第一版不会自动删除；到期后如需清理，必须先确认
|
||||
具体运行目录。
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Run the approved arXiv component over the five local ClinDB paper copies."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from mdpolish.components import ArxivSubmissionStampComponent
|
||||
from mdpolish.experiment import InputDocument, collect_tool_metadata, run_experiment
|
||||
from mdpolish.models import RunStatus
|
||||
from mdpolish.pipeline import Pipeline
|
||||
|
||||
_DOCUMENT_IDS = ("dmp", "ejhf", "jama", "sim", "springer")
|
||||
|
||||
|
||||
def _parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--run-id", required=True, help="safe identifier within today's artifact directory")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = _parse_args()
|
||||
repository_root = Path(__file__).resolve().parents[1]
|
||||
documents = tuple(
|
||||
InputDocument(
|
||||
document_id=document_id,
|
||||
source_path=repository_root / "data" / "md" / f"{document_id}.md",
|
||||
source_label=f"data/md/{document_id}.md",
|
||||
)
|
||||
for document_id in _DOCUMENT_IDS
|
||||
)
|
||||
started_at = datetime.now().astimezone()
|
||||
try:
|
||||
result = run_experiment(
|
||||
pipeline=Pipeline([ArxivSubmissionStampComponent()]),
|
||||
documents=documents,
|
||||
run_id=args.run_id,
|
||||
artifacts_root=repository_root / "artifacts",
|
||||
started_at=started_at,
|
||||
tool=collect_tool_metadata(repository_root),
|
||||
)
|
||||
except Exception as error:
|
||||
print(f"experiment failed: {type(error).__name__}: {error}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
print(f"run_id={args.run_id}")
|
||||
print(f"status={result.status.value}")
|
||||
print(f"documents={result.document_count}")
|
||||
print(f"changes={result.change_count}")
|
||||
print(f"artifacts={result.run_directory}")
|
||||
return 0 if result.status is RunStatus.SUCCESS else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,290 @@
|
||||
"""Private local storage for complete experiment artifact directories."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import stat
|
||||
import tempfile
|
||||
from dataclasses import dataclass
|
||||
from datetime import date
|
||||
from hashlib import sha256
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
_SAFE_ID_PATTERN = re.compile(r"^[a-z0-9][a-z0-9._-]*$")
|
||||
_DIRECTORY_MODE = 0o700
|
||||
_FILE_MODE = 0o600
|
||||
|
||||
|
||||
class ArtifactStoreError(RuntimeError):
|
||||
"""A local artifact directory cannot be published safely."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class StoredDocument:
|
||||
"""Complete encoded artifacts for one document."""
|
||||
|
||||
document_id: str
|
||||
result_json: bytes
|
||||
cleaned_markdown: bytes | None
|
||||
diff: bytes | None
|
||||
output_sha256: str | None
|
||||
|
||||
|
||||
def _validate_identifier(value: str, field_name: str) -> None:
|
||||
if not isinstance(value, str) or _SAFE_ID_PATTERN.fullmatch(value) is None or ".." in value:
|
||||
raise ArtifactStoreError(f"{field_name} must be a safe lowercase identifier")
|
||||
|
||||
|
||||
def _validate_run_date(run_date: str) -> None:
|
||||
if not isinstance(run_date, str):
|
||||
raise ArtifactStoreError("run_date must use YYYY-MM-DD")
|
||||
try:
|
||||
parsed = date.fromisoformat(run_date)
|
||||
except ValueError as error:
|
||||
raise ArtifactStoreError("run_date must use YYYY-MM-DD") from error
|
||||
if parsed.isoformat() != run_date:
|
||||
raise ArtifactStoreError("run_date must use YYYY-MM-DD")
|
||||
|
||||
|
||||
def _require_private_directory(path: Path) -> None:
|
||||
if path.is_symlink():
|
||||
raise ArtifactStoreError(f"artifact directory cannot be a symlink: {path}")
|
||||
if not path.exists():
|
||||
path.mkdir(mode=_DIRECTORY_MODE, parents=False)
|
||||
path.chmod(_DIRECTORY_MODE)
|
||||
if not path.is_dir():
|
||||
raise ArtifactStoreError(f"artifact path is not a directory: {path}")
|
||||
mode = stat.S_IMODE(path.stat().st_mode)
|
||||
if mode != _DIRECTORY_MODE:
|
||||
raise ArtifactStoreError(f"artifact directory must have mode 0700: {path}")
|
||||
|
||||
|
||||
def _create_private_directory(path: Path) -> None:
|
||||
path.mkdir(mode=_DIRECTORY_MODE)
|
||||
path.chmod(_DIRECTORY_MODE)
|
||||
_require_private_directory(path)
|
||||
|
||||
|
||||
def _prepare_runs_directory(artifacts_root: Path, run_date: str) -> Path:
|
||||
if not artifacts_root.exists():
|
||||
artifacts_root.mkdir(mode=_DIRECTORY_MODE, parents=True)
|
||||
artifacts_root.chmod(_DIRECTORY_MODE)
|
||||
_require_private_directory(artifacts_root)
|
||||
|
||||
date_directory = artifacts_root / run_date
|
||||
_require_private_directory(date_directory)
|
||||
runs_directory = date_directory / "runs"
|
||||
_require_private_directory(runs_directory)
|
||||
return runs_directory
|
||||
|
||||
|
||||
def _write_private_file(path: Path, content: bytes) -> None:
|
||||
if not isinstance(content, bytes):
|
||||
raise ArtifactStoreError("artifact content must be bytes")
|
||||
with path.open("xb") as artifact_file:
|
||||
artifact_file.write(content)
|
||||
artifact_file.flush()
|
||||
os.fsync(artifact_file.fileno())
|
||||
path.chmod(_FILE_MODE)
|
||||
if stat.S_IMODE(path.stat().st_mode) != _FILE_MODE:
|
||||
raise ArtifactStoreError(f"artifact file must have mode 0600: {path}")
|
||||
if path.read_bytes() != content:
|
||||
raise ArtifactStoreError(f"artifact readback verification failed: {path}")
|
||||
|
||||
|
||||
def _validate_document(document: StoredDocument) -> None:
|
||||
_validate_identifier(document.document_id, "document_id")
|
||||
has_output = document.cleaned_markdown is not None
|
||||
if has_output != (document.diff is not None) or has_output != (document.output_sha256 is not None):
|
||||
raise ArtifactStoreError("cleaned Markdown, diff, and output hash must be present together")
|
||||
if document.cleaned_markdown is not None:
|
||||
digest = sha256(document.cleaned_markdown).hexdigest()
|
||||
if digest != document.output_sha256:
|
||||
raise ArtifactStoreError("cleaned Markdown does not match its output hash")
|
||||
|
||||
|
||||
def _json_object(content: bytes, label: str) -> dict[str, object]:
|
||||
if not isinstance(content, bytes) or not content.endswith(b"\n") or content.startswith(b"\xef\xbb\xbf"):
|
||||
raise ArtifactStoreError(f"{label} must be BOM-free UTF-8 JSON ending in a newline")
|
||||
try:
|
||||
decoded = json.loads(content)
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as error:
|
||||
raise ArtifactStoreError(f"{label} must contain valid UTF-8 JSON") from error
|
||||
if not isinstance(decoded, dict) or not all(isinstance(key, str) for key in decoded):
|
||||
raise ArtifactStoreError(f"{label} must contain a JSON object")
|
||||
return cast(dict[str, object], decoded)
|
||||
|
||||
|
||||
def _nested_object(payload: dict[str, object], field_name: str, label: str) -> dict[str, object]:
|
||||
value = payload.get(field_name)
|
||||
if not isinstance(value, dict) or not all(isinstance(key, str) for key in value):
|
||||
raise ArtifactStoreError(f"{label}.{field_name} must be a JSON object")
|
||||
return cast(dict[str, object], value)
|
||||
|
||||
|
||||
def _validate_bundle(
|
||||
*,
|
||||
run_date: str,
|
||||
run_id: str,
|
||||
manifest_json: bytes,
|
||||
documents: tuple[StoredDocument, ...],
|
||||
) -> None:
|
||||
manifest = _json_object(manifest_json, "manifest")
|
||||
if manifest.get("schema_version") != 1:
|
||||
raise ArtifactStoreError("manifest schema_version must be 1")
|
||||
run = _nested_object(manifest, "run", "manifest")
|
||||
if run.get("run_date") != run_date or run.get("run_id") != run_id:
|
||||
raise ArtifactStoreError("manifest run identity does not match the target path")
|
||||
|
||||
document_indexes = manifest.get("documents")
|
||||
if not isinstance(document_indexes, list) or len(document_indexes) != len(documents):
|
||||
raise ArtifactStoreError("manifest documents do not match the stored documents")
|
||||
|
||||
success_count = 0
|
||||
failed_count = 0
|
||||
unstable_count = 0
|
||||
change_count = 0
|
||||
for position, document in enumerate(documents):
|
||||
raw_index = document_indexes[position]
|
||||
if not isinstance(raw_index, dict) or not all(isinstance(key, str) for key in raw_index):
|
||||
raise ArtifactStoreError("manifest document indexes must be JSON objects")
|
||||
index = cast(dict[str, object], raw_index)
|
||||
report = _json_object(document.result_json, f"result for {document.document_id}")
|
||||
report_document = _nested_object(report, "document", f"result for {document.document_id}")
|
||||
report_output = _nested_object(report, "output", f"result for {document.document_id}")
|
||||
status = report.get("status")
|
||||
changes = report.get("changes")
|
||||
if (
|
||||
report.get("schema_version") != 1
|
||||
or report_document.get("document_id") != document.document_id
|
||||
or status not in {"success", "failed", "unstable"}
|
||||
or not isinstance(changes, list)
|
||||
):
|
||||
raise ArtifactStoreError("a document result does not match its stored identity")
|
||||
|
||||
has_output = document.cleaned_markdown is not None
|
||||
expected_cleaned_name = "cleaned.md" if has_output else None
|
||||
expected_diff_name = "changes.diff" if has_output else None
|
||||
base = f"documents/{document.document_id}"
|
||||
expected_cleaned_path = f"{base}/cleaned.md" if has_output else None
|
||||
expected_diff_path = f"{base}/changes.diff" if has_output else None
|
||||
if has_output != (status == "success"):
|
||||
raise ArtifactStoreError("document output presence does not match its result status")
|
||||
if report_output != {
|
||||
"cleaned_path": expected_cleaned_name,
|
||||
"diff_path": expected_diff_name,
|
||||
}:
|
||||
raise ArtifactStoreError("document result paths do not match its stored files")
|
||||
if document.output_sha256 != (report.get("current_sha256") if has_output else None):
|
||||
raise ArtifactStoreError("document output hash does not match its result")
|
||||
|
||||
expected_index = {
|
||||
"document_id": document.document_id,
|
||||
"source_label": report_document.get("source_label"),
|
||||
"status": status,
|
||||
"input_sha256": report.get("input_sha256"),
|
||||
"current_sha256": report.get("current_sha256"),
|
||||
"change_count": len(changes),
|
||||
"result_path": f"{base}/result.json",
|
||||
"cleaned_path": expected_cleaned_path,
|
||||
"diff_path": expected_diff_path,
|
||||
}
|
||||
if index != expected_index:
|
||||
raise ArtifactStoreError("a manifest document index does not match its result and files")
|
||||
|
||||
change_count += len(changes)
|
||||
if status == "success":
|
||||
success_count += 1
|
||||
elif status == "failed":
|
||||
failed_count += 1
|
||||
else:
|
||||
unstable_count += 1
|
||||
|
||||
overall_status = "failed" if failed_count else "unstable" if unstable_count else "success"
|
||||
expected_summary = {
|
||||
"document_count": len(documents),
|
||||
"success_count": success_count,
|
||||
"failed_count": failed_count,
|
||||
"unstable_count": unstable_count,
|
||||
"change_count": change_count,
|
||||
}
|
||||
if run.get("status") != overall_status or manifest.get("summary") != expected_summary:
|
||||
raise ArtifactStoreError("manifest status or summary does not match its documents")
|
||||
|
||||
|
||||
def _rename_no_replace(source: Path, target: Path) -> None:
|
||||
directory_fd = os.open(source.parent, os.O_RDONLY | os.O_DIRECTORY)
|
||||
try:
|
||||
fcntl.flock(directory_fd, fcntl.LOCK_EX)
|
||||
if target.exists() or target.is_symlink():
|
||||
raise ArtifactStoreError("the target run directory already exists")
|
||||
os.rename(source, target)
|
||||
finally:
|
||||
fcntl.flock(directory_fd, fcntl.LOCK_UN)
|
||||
os.close(directory_fd)
|
||||
|
||||
|
||||
def publish_run(
|
||||
*,
|
||||
artifacts_root: Path,
|
||||
run_date: str,
|
||||
run_id: str,
|
||||
manifest_json: bytes,
|
||||
documents: tuple[StoredDocument, ...],
|
||||
) -> Path:
|
||||
"""Publish one complete run directory without replacing an existing run."""
|
||||
if not isinstance(artifacts_root, Path):
|
||||
raise TypeError("artifacts_root must be a Path")
|
||||
_validate_run_date(run_date)
|
||||
_validate_identifier(run_id, "run_id")
|
||||
if not documents:
|
||||
raise ArtifactStoreError("an artifact run requires at least one document")
|
||||
|
||||
seen_ids: set[str] = set()
|
||||
for document in documents:
|
||||
if not isinstance(document, StoredDocument):
|
||||
raise ArtifactStoreError("documents must contain only StoredDocument values")
|
||||
_validate_document(document)
|
||||
if document.document_id in seen_ids:
|
||||
raise ArtifactStoreError("document_id values must be unique")
|
||||
seen_ids.add(document.document_id)
|
||||
_validate_bundle(
|
||||
run_date=run_date,
|
||||
run_id=run_id,
|
||||
manifest_json=manifest_json,
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
runs_directory = _prepare_runs_directory(artifacts_root, run_date)
|
||||
final_directory = runs_directory / run_id
|
||||
if final_directory.exists() or final_directory.is_symlink():
|
||||
raise ArtifactStoreError("the target run directory already exists")
|
||||
|
||||
temporary_directory = Path(tempfile.mkdtemp(prefix=f".{run_id}.", dir=runs_directory))
|
||||
temporary_directory.chmod(_DIRECTORY_MODE)
|
||||
try:
|
||||
documents_directory = temporary_directory / "documents"
|
||||
_create_private_directory(documents_directory)
|
||||
for document in documents:
|
||||
document_directory = documents_directory / document.document_id
|
||||
_create_private_directory(document_directory)
|
||||
_write_private_file(document_directory / "result.json", document.result_json)
|
||||
if document.cleaned_markdown is not None and document.diff is not None:
|
||||
_write_private_file(document_directory / "cleaned.md", document.cleaned_markdown)
|
||||
_write_private_file(document_directory / "changes.diff", document.diff)
|
||||
|
||||
_write_private_file(temporary_directory / "manifest.json", manifest_json)
|
||||
_rename_no_replace(temporary_directory, final_directory)
|
||||
except Exception:
|
||||
if temporary_directory.exists():
|
||||
shutil.rmtree(temporary_directory)
|
||||
raise
|
||||
|
||||
if stat.S_IMODE(final_directory.stat().st_mode) != _DIRECTORY_MODE:
|
||||
raise ArtifactStoreError("published run directory does not have mode 0700")
|
||||
return final_directory
|
||||
@@ -0,0 +1,380 @@
|
||||
"""Explicit local file experiments built around the in-memory pipeline."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import platform as platform_module
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from hashlib import sha256
|
||||
from importlib.metadata import PackageNotFoundError, version
|
||||
from pathlib import Path
|
||||
|
||||
from mdpolish.artifact_store import StoredDocument, publish_run
|
||||
from mdpolish.models import ComponentInfo, RunStatus, markdown_sha256
|
||||
from mdpolish.pipeline import Pipeline
|
||||
from mdpolish.reporting import (
|
||||
DocumentReport,
|
||||
JsonObject,
|
||||
JsonValue,
|
||||
build_document_report,
|
||||
component_info_json,
|
||||
encode_json,
|
||||
)
|
||||
|
||||
_SAFE_ID_PATTERN = re.compile(r"^[a-z0-9][a-z0-9._-]*$")
|
||||
_UTC_OFFSET_PATTERN = re.compile(r"^[+-][0-9]{2}:[0-9]{2}$")
|
||||
|
||||
|
||||
class ExperimentError(RuntimeError):
|
||||
"""A local experiment cannot be completed without violating its contract."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class InputDocument:
|
||||
"""One explicitly selected local Markdown input."""
|
||||
|
||||
document_id: str
|
||||
source_path: Path
|
||||
source_label: str
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ToolMetadata:
|
||||
"""Source-safe tool and environment facts for a run manifest."""
|
||||
|
||||
name: str
|
||||
package_version: str
|
||||
python_version: str
|
||||
platform: str
|
||||
git_commit: str | None
|
||||
git_dirty: bool | None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ExperimentResult:
|
||||
"""Non-sensitive terminal summary of one published local run."""
|
||||
|
||||
run_directory: Path
|
||||
status: RunStatus
|
||||
document_count: int
|
||||
success_count: int
|
||||
failed_count: int
|
||||
unstable_count: int
|
||||
change_count: int
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _PreparedDocument:
|
||||
input: InputDocument
|
||||
resolved_path: Path
|
||||
source_bytes: bytes
|
||||
markdown: str
|
||||
input_sha256: str
|
||||
|
||||
|
||||
def _safe_identifier(value: str, field_name: str) -> None:
|
||||
if not isinstance(value, str) or _SAFE_ID_PATTERN.fullmatch(value) is None or ".." in value:
|
||||
raise ExperimentError(f"{field_name} must be a safe lowercase identifier")
|
||||
|
||||
|
||||
def _run_git(repository_root: Path, *arguments: str) -> str | None:
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
("git", *arguments),
|
||||
cwd=repository_root,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return None
|
||||
if completed.returncode != 0:
|
||||
return None
|
||||
return completed.stdout.strip()
|
||||
|
||||
|
||||
def collect_tool_metadata(repository_root: Path) -> ToolMetadata:
|
||||
"""Collect only the reproducibility facts approved for the manifest."""
|
||||
if not isinstance(repository_root, Path):
|
||||
raise TypeError("repository_root must be a Path")
|
||||
try:
|
||||
package_version = version("mdpolish")
|
||||
except PackageNotFoundError:
|
||||
package_version = "uninstalled"
|
||||
|
||||
git_commit = _run_git(repository_root, "rev-parse", "HEAD")
|
||||
git_status = _run_git(repository_root, "status", "--porcelain")
|
||||
return ToolMetadata(
|
||||
name="mdpolish",
|
||||
package_version=package_version,
|
||||
python_version=platform_module.python_version(),
|
||||
platform=f"{sys.platform}-{platform_module.machine()}",
|
||||
git_commit=git_commit,
|
||||
git_dirty=None if git_status is None else bool(git_status),
|
||||
)
|
||||
|
||||
|
||||
def _utc_text(value: datetime) -> str:
|
||||
if value.tzinfo is None or value.utcoffset() is None:
|
||||
raise ExperimentError("experiment timestamps must be timezone-aware")
|
||||
utc_value = value.astimezone(UTC)
|
||||
return utc_value.isoformat(timespec="seconds").replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _utc_offset_text(value: datetime) -> str:
|
||||
offset = value.utcoffset()
|
||||
if offset is None:
|
||||
raise ExperimentError("experiment timestamps must be timezone-aware")
|
||||
total_minutes = int(offset.total_seconds() // 60)
|
||||
sign = "+" if total_minutes >= 0 else "-"
|
||||
absolute_minutes = abs(total_minutes)
|
||||
hours, minutes = divmod(absolute_minutes, 60)
|
||||
text = f"{sign}{hours:02d}:{minutes:02d}"
|
||||
if _UTC_OFFSET_PATTERN.fullmatch(text) is None or hours > 23 or minutes > 59:
|
||||
raise ExperimentError("the experiment UTC offset cannot be represented safely")
|
||||
return text
|
||||
|
||||
|
||||
def _prepare_documents(
|
||||
documents: tuple[InputDocument, ...],
|
||||
*,
|
||||
final_directory: Path,
|
||||
) -> tuple[_PreparedDocument, ...]:
|
||||
if not isinstance(documents, tuple) or not documents:
|
||||
raise ExperimentError("documents must be a non-empty tuple")
|
||||
|
||||
seen_ids: set[str] = set()
|
||||
seen_paths: set[Path] = set()
|
||||
prepared: list[_PreparedDocument] = []
|
||||
final_resolved = final_directory.resolve(strict=False)
|
||||
for document in documents:
|
||||
if not isinstance(document, InputDocument):
|
||||
raise ExperimentError("documents must contain only InputDocument values")
|
||||
_safe_identifier(document.document_id, "document_id")
|
||||
if document.document_id in seen_ids:
|
||||
raise ExperimentError("document_id values must be unique")
|
||||
seen_ids.add(document.document_id)
|
||||
if not isinstance(document.source_path, Path):
|
||||
raise ExperimentError("source_path must be a Path")
|
||||
if (
|
||||
not isinstance(document.source_label, str)
|
||||
or not document.source_label.strip()
|
||||
or "\0" in document.source_label
|
||||
):
|
||||
raise ExperimentError("source_label must be a non-empty display string")
|
||||
|
||||
try:
|
||||
resolved_path = document.source_path.resolve(strict=True)
|
||||
except OSError as error:
|
||||
raise ExperimentError(f"input file cannot be resolved: {document.source_label}") from error
|
||||
if not resolved_path.is_file():
|
||||
raise ExperimentError(f"input path is not a regular file: {document.source_label}")
|
||||
if resolved_path in seen_paths:
|
||||
raise ExperimentError("source_path values must be unique")
|
||||
seen_paths.add(resolved_path)
|
||||
if (
|
||||
resolved_path == final_resolved
|
||||
or final_resolved in resolved_path.parents
|
||||
or resolved_path in final_resolved.parents
|
||||
):
|
||||
raise ExperimentError("an input file and target run directory cannot overlap")
|
||||
|
||||
try:
|
||||
source_bytes = resolved_path.read_bytes()
|
||||
markdown = source_bytes.decode("utf-8", errors="strict")
|
||||
except (OSError, UnicodeDecodeError) as error:
|
||||
raise ExperimentError(f"input file must be readable strict UTF-8: {document.source_label}") from error
|
||||
input_sha256 = sha256(source_bytes).hexdigest()
|
||||
if markdown_sha256(markdown) != input_sha256:
|
||||
raise ExperimentError("decoded Markdown does not reproduce the input byte hash")
|
||||
prepared.append(
|
||||
_PreparedDocument(
|
||||
input=document,
|
||||
resolved_path=resolved_path,
|
||||
source_bytes=source_bytes,
|
||||
markdown=markdown,
|
||||
input_sha256=input_sha256,
|
||||
)
|
||||
)
|
||||
return tuple(prepared)
|
||||
|
||||
|
||||
def _overall_status(reports: tuple[DocumentReport, ...]) -> RunStatus:
|
||||
if any(report.status is RunStatus.FAILED for report in reports):
|
||||
return RunStatus.FAILED
|
||||
if any(report.status is RunStatus.UNSTABLE for report in reports):
|
||||
return RunStatus.UNSTABLE
|
||||
return RunStatus.SUCCESS
|
||||
|
||||
|
||||
def _document_index(report: DocumentReport) -> JsonObject:
|
||||
base = f"documents/{report.document_id}"
|
||||
return {
|
||||
"document_id": report.document_id,
|
||||
"source_label": report.source_label,
|
||||
"status": report.status.value,
|
||||
"input_sha256": report.input_sha256,
|
||||
"current_sha256": report.current_sha256,
|
||||
"change_count": report.change_count,
|
||||
"result_path": f"{base}/result.json",
|
||||
"cleaned_path": f"{base}/cleaned.md" if report.status is RunStatus.SUCCESS else None,
|
||||
"diff_path": f"{base}/changes.diff" if report.status is RunStatus.SUCCESS else None,
|
||||
}
|
||||
|
||||
|
||||
def _manifest_payload(
|
||||
*,
|
||||
run_id: str,
|
||||
run_date: str,
|
||||
utc_offset: str,
|
||||
started_at: datetime,
|
||||
completed_at: datetime,
|
||||
tool: ToolMetadata,
|
||||
components: tuple[ComponentInfo, ...],
|
||||
reports: tuple[DocumentReport, ...],
|
||||
) -> JsonObject:
|
||||
status = _overall_status(reports)
|
||||
component_values: list[JsonValue] = [component_info_json(component) for component in components]
|
||||
document_values: list[JsonValue] = [_document_index(report) for report in reports]
|
||||
success_count = sum(report.status is RunStatus.SUCCESS for report in reports)
|
||||
failed_count = sum(report.status is RunStatus.FAILED for report in reports)
|
||||
unstable_count = sum(report.status is RunStatus.UNSTABLE for report in reports)
|
||||
change_count = sum(report.change_count for report in reports)
|
||||
retention_until = completed_at.astimezone(UTC) + timedelta(days=30)
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"run": {
|
||||
"run_id": run_id,
|
||||
"run_date": run_date,
|
||||
"utc_offset": utc_offset,
|
||||
"status": status.value,
|
||||
"started_at_utc": _utc_text(started_at),
|
||||
"completed_at_utc": _utc_text(completed_at),
|
||||
"retention_until": _utc_text(retention_until),
|
||||
},
|
||||
"tool": {
|
||||
"name": tool.name,
|
||||
"package_version": tool.package_version,
|
||||
"python_version": tool.python_version,
|
||||
"platform": tool.platform,
|
||||
"git_commit": tool.git_commit,
|
||||
"git_dirty": tool.git_dirty,
|
||||
},
|
||||
"pipeline": {"components": component_values},
|
||||
"documents": document_values,
|
||||
"summary": {
|
||||
"document_count": len(reports),
|
||||
"success_count": success_count,
|
||||
"failed_count": failed_count,
|
||||
"unstable_count": unstable_count,
|
||||
"change_count": change_count,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def run_experiment(
|
||||
*,
|
||||
pipeline: Pipeline,
|
||||
documents: tuple[InputDocument, ...],
|
||||
run_id: str,
|
||||
artifacts_root: Path,
|
||||
started_at: datetime,
|
||||
tool: ToolMetadata,
|
||||
completed_at: datetime | None = None,
|
||||
) -> ExperimentResult:
|
||||
"""Run explicit inputs and publish one complete private artifact directory."""
|
||||
if not isinstance(pipeline, Pipeline):
|
||||
raise TypeError("pipeline must be a Pipeline")
|
||||
_safe_identifier(run_id, "run_id")
|
||||
if not isinstance(artifacts_root, Path):
|
||||
raise TypeError("artifacts_root must be a Path")
|
||||
if not isinstance(tool, ToolMetadata):
|
||||
raise TypeError("tool must be ToolMetadata")
|
||||
if started_at.tzinfo is None or started_at.utcoffset() is None:
|
||||
raise ExperimentError("started_at must be timezone-aware")
|
||||
|
||||
run_date = started_at.date().isoformat()
|
||||
utc_offset = _utc_offset_text(started_at)
|
||||
final_directory = artifacts_root / run_date / "runs" / run_id
|
||||
if final_directory.exists() or final_directory.is_symlink():
|
||||
raise ExperimentError("the target run directory already exists")
|
||||
prepared = _prepare_documents(documents, final_directory=final_directory)
|
||||
|
||||
reports: list[DocumentReport] = []
|
||||
expected_components: tuple[ComponentInfo, ...] | None = None
|
||||
for document in prepared:
|
||||
result = pipeline.transform(document.markdown)
|
||||
if expected_components is None:
|
||||
expected_components = result.components
|
||||
elif result.components != expected_components:
|
||||
raise ExperimentError("pipeline component metadata changed between documents")
|
||||
reports.append(
|
||||
build_document_report(
|
||||
document_id=document.input.document_id,
|
||||
source_label=document.input.source_label,
|
||||
input_markdown=document.markdown,
|
||||
result=result,
|
||||
)
|
||||
)
|
||||
|
||||
for document in prepared:
|
||||
try:
|
||||
current_bytes = document.resolved_path.read_bytes()
|
||||
except OSError as error:
|
||||
raise ExperimentError(
|
||||
f"input file cannot be verified after the run: {document.input.source_label}"
|
||||
) from error
|
||||
if current_bytes != document.source_bytes or sha256(current_bytes).hexdigest() != document.input_sha256:
|
||||
raise ExperimentError(f"input file changed during the run: {document.input.source_label}")
|
||||
|
||||
report_tuple = tuple(reports)
|
||||
if expected_components is None:
|
||||
raise ExperimentError("a non-empty experiment produced no component metadata")
|
||||
completion = datetime.now(UTC) if completed_at is None else completed_at
|
||||
if completion.tzinfo is None or completion.utcoffset() is None:
|
||||
raise ExperimentError("completed_at must be timezone-aware")
|
||||
if completion.astimezone(UTC) < started_at.astimezone(UTC):
|
||||
raise ExperimentError("completed_at cannot be earlier than started_at")
|
||||
|
||||
manifest = _manifest_payload(
|
||||
run_id=run_id,
|
||||
run_date=run_date,
|
||||
utc_offset=utc_offset,
|
||||
started_at=started_at,
|
||||
completed_at=completion,
|
||||
tool=tool,
|
||||
components=expected_components,
|
||||
reports=report_tuple,
|
||||
)
|
||||
stored_documents = tuple(
|
||||
StoredDocument(
|
||||
document_id=report.document_id,
|
||||
result_json=report.result_json,
|
||||
cleaned_markdown=report.cleaned_markdown,
|
||||
diff=report.diff,
|
||||
output_sha256=report.current_sha256 if report.status is RunStatus.SUCCESS else None,
|
||||
)
|
||||
for report in report_tuple
|
||||
)
|
||||
run_directory = publish_run(
|
||||
artifacts_root=artifacts_root,
|
||||
run_date=run_date,
|
||||
run_id=run_id,
|
||||
manifest_json=encode_json(manifest),
|
||||
documents=stored_documents,
|
||||
)
|
||||
|
||||
status = _overall_status(report_tuple)
|
||||
return ExperimentResult(
|
||||
run_directory=run_directory,
|
||||
status=status,
|
||||
document_count=len(report_tuple),
|
||||
success_count=sum(report.status is RunStatus.SUCCESS for report in report_tuple),
|
||||
failed_count=sum(report.status is RunStatus.FAILED for report in report_tuple),
|
||||
unstable_count=sum(report.status is RunStatus.UNSTABLE for report in report_tuple),
|
||||
change_count=sum(report.change_count for report in report_tuple),
|
||||
)
|
||||
@@ -0,0 +1,315 @@
|
||||
"""Pure JSON and diff representations for local experiment results."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from difflib import unified_diff
|
||||
from typing import TypeAlias
|
||||
|
||||
from mdpolish.models import (
|
||||
Change,
|
||||
ComponentInfo,
|
||||
ParameterValue,
|
||||
ProposedChange,
|
||||
ResidualProposal,
|
||||
RunError,
|
||||
RunStatus,
|
||||
TextEdit,
|
||||
TransformResult,
|
||||
markdown_sha256,
|
||||
)
|
||||
|
||||
JsonValue: TypeAlias = bool | int | float | str | list["JsonValue"] | dict[str, "JsonValue"] | None
|
||||
JsonObject: TypeAlias = dict[str, JsonValue]
|
||||
|
||||
|
||||
class ReportingError(ValueError):
|
||||
"""A core result cannot be represented without losing its audit contract."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class DocumentReport:
|
||||
"""Rendered artifacts and index facts for one document result."""
|
||||
|
||||
document_id: str
|
||||
source_label: str
|
||||
status: RunStatus
|
||||
input_sha256: str
|
||||
current_sha256: str
|
||||
change_count: int
|
||||
result_json: bytes
|
||||
cleaned_markdown: bytes | None
|
||||
diff: bytes | None
|
||||
|
||||
|
||||
def _parameter_value_json(value: ParameterValue) -> JsonValue:
|
||||
if isinstance(value, tuple):
|
||||
return [_parameter_value_json(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def component_info_json(component: ComponentInfo) -> JsonObject:
|
||||
"""Represent validated component metadata without changing tuple order."""
|
||||
parameters: list[JsonValue] = []
|
||||
for name, value in component.parameters:
|
||||
parameters.append([name, _parameter_value_json(value)])
|
||||
return {
|
||||
"component_id": component.component_id,
|
||||
"version": component.version,
|
||||
"parameters": parameters,
|
||||
"applicability": component.applicability,
|
||||
}
|
||||
|
||||
|
||||
def _proposal_reference_json(change: Change) -> JsonObject:
|
||||
reference = change.proposal_ref
|
||||
return {
|
||||
"component_position": reference.component_position,
|
||||
"snapshot_sha256": reference.snapshot_sha256,
|
||||
"proposal_index": reference.proposal_index,
|
||||
}
|
||||
|
||||
|
||||
def _text_edit_json(edit: TextEdit) -> JsonObject:
|
||||
return {
|
||||
"snapshot_sha256": edit.snapshot_sha256,
|
||||
"span": {"start": edit.span.start, "end": edit.span.end},
|
||||
"expected_text": edit.expected_text,
|
||||
"replacement": edit.replacement,
|
||||
}
|
||||
|
||||
|
||||
def _proposed_change_json(proposal: ProposedChange) -> JsonObject:
|
||||
edits: list[JsonValue] = [_text_edit_json(edit) for edit in proposal.edits]
|
||||
return {
|
||||
"snapshot_sha256": proposal.snapshot_sha256,
|
||||
"reason": proposal.reason,
|
||||
"edits": edits,
|
||||
}
|
||||
|
||||
|
||||
def _residual_proposal_json(residual: ResidualProposal) -> JsonObject:
|
||||
reference = residual.proposal_ref
|
||||
return {
|
||||
"component_id": residual.component_id,
|
||||
"component_version": residual.component_version,
|
||||
"component_position": residual.component_position,
|
||||
"proposal_ref": {
|
||||
"component_position": reference.component_position,
|
||||
"snapshot_sha256": reference.snapshot_sha256,
|
||||
"proposal_index": reference.proposal_index,
|
||||
},
|
||||
"proposal": _proposed_change_json(residual.proposal),
|
||||
}
|
||||
|
||||
|
||||
def _run_error_json(error: RunError) -> JsonObject:
|
||||
return {
|
||||
"component_id": error.component_id,
|
||||
"component_version": error.component_version,
|
||||
"component_position": error.component_position,
|
||||
"stage": error.stage.value,
|
||||
"error_type": error.error_type,
|
||||
"message": error.message,
|
||||
}
|
||||
|
||||
|
||||
def _line_column(markdown: str, offset: int) -> tuple[int, int]:
|
||||
if offset < 0 or offset > len(markdown):
|
||||
raise ReportingError("a change offset is outside its recorded snapshot")
|
||||
|
||||
line = 1
|
||||
column = 1
|
||||
position = 0
|
||||
while position < offset:
|
||||
character = markdown[position]
|
||||
if character == "\r":
|
||||
line += 1
|
||||
column = 1
|
||||
if position + 1 < offset and markdown[position + 1] == "\n":
|
||||
position += 2
|
||||
else:
|
||||
position += 1
|
||||
elif character == "\n":
|
||||
line += 1
|
||||
column = 1
|
||||
position += 1
|
||||
else:
|
||||
column += 1
|
||||
position += 1
|
||||
return line, column
|
||||
|
||||
|
||||
def _validate_change_identity(change: Change, components: tuple[ComponentInfo, ...]) -> None:
|
||||
if change.component_position < 0 or change.component_position >= len(components):
|
||||
raise ReportingError("a change has no matching component position")
|
||||
component = components[change.component_position]
|
||||
if change.component_id != component.component_id or change.component_version != component.version:
|
||||
raise ReportingError("a change identity does not match the recorded component")
|
||||
if change.proposal_ref.component_position != change.component_position:
|
||||
raise ReportingError("a change proposal reference has the wrong component position")
|
||||
if change.proposal_ref.snapshot_sha256 != change.before_sha256:
|
||||
raise ReportingError("a change proposal reference targets the wrong snapshot")
|
||||
|
||||
|
||||
def _replay_changes(input_markdown: str, result: TransformResult) -> tuple[tuple[tuple[int, int], ...], str]:
|
||||
if markdown_sha256(input_markdown) != result.input_sha256:
|
||||
raise ReportingError("input Markdown does not match the transform result")
|
||||
|
||||
current = input_markdown
|
||||
locations: list[tuple[int, int]] = []
|
||||
cursor = 0
|
||||
while cursor < len(result.changes):
|
||||
first = result.changes[cursor]
|
||||
batch_key = (first.component_position, first.before_sha256, first.after_sha256)
|
||||
batch: list[Change] = []
|
||||
while cursor < len(result.changes):
|
||||
candidate = result.changes[cursor]
|
||||
candidate_key = (candidate.component_position, candidate.before_sha256, candidate.after_sha256)
|
||||
if candidate_key != batch_key:
|
||||
break
|
||||
batch.append(candidate)
|
||||
cursor += 1
|
||||
|
||||
if markdown_sha256(current) != first.before_sha256:
|
||||
raise ReportingError("a change batch does not follow the recorded snapshot chain")
|
||||
|
||||
for change in batch:
|
||||
_validate_change_identity(change, result.components)
|
||||
if change.before_sha256 != first.before_sha256 or change.after_sha256 != first.after_sha256:
|
||||
raise ReportingError("a change batch contains inconsistent snapshot hashes")
|
||||
if change.span.end > len(current):
|
||||
raise ReportingError("a change span is outside its recorded snapshot")
|
||||
if len(change.before) != change.span.end - change.span.start:
|
||||
raise ReportingError("a change before value does not match its span length")
|
||||
if current[change.span.start : change.span.end] != change.before:
|
||||
raise ReportingError("a change before value does not match its recorded snapshot")
|
||||
locations.append(_line_column(current, change.span.start))
|
||||
|
||||
application_order = sorted(
|
||||
batch,
|
||||
key=lambda change: (
|
||||
change.span.start,
|
||||
change.span.end,
|
||||
change.proposal_ref.proposal_index,
|
||||
change.edit_index,
|
||||
),
|
||||
reverse=True,
|
||||
)
|
||||
for change in application_order:
|
||||
current = current[: change.span.start] + change.after + current[change.span.end :]
|
||||
if markdown_sha256(current) != first.after_sha256:
|
||||
raise ReportingError("replayed changes do not produce the recorded batch hash")
|
||||
|
||||
current_markdown = result.output_markdown if result.status is RunStatus.SUCCESS else result.partial_markdown
|
||||
if current_markdown is None:
|
||||
raise ReportingError("a transform result does not contain its status-specific Markdown")
|
||||
if current != current_markdown or markdown_sha256(current) != result.current_sha256:
|
||||
raise ReportingError("replayed changes do not produce the transform result's current snapshot")
|
||||
return tuple(locations), current
|
||||
|
||||
|
||||
def _change_json(change: Change, location: tuple[int, int]) -> JsonObject:
|
||||
line, column = location
|
||||
return {
|
||||
"component_id": change.component_id,
|
||||
"component_version": change.component_version,
|
||||
"component_position": change.component_position,
|
||||
"proposal_ref": _proposal_reference_json(change),
|
||||
"edit_index": change.edit_index,
|
||||
"reason": change.reason,
|
||||
"span": {"start": change.span.start, "end": change.span.end},
|
||||
"location": {"line": line, "column": column},
|
||||
"before": change.before,
|
||||
"after": change.after,
|
||||
"before_sha256": change.before_sha256,
|
||||
"after_sha256": change.after_sha256,
|
||||
}
|
||||
|
||||
|
||||
def encode_json(payload: JsonObject) -> bytes:
|
||||
"""Encode a schema object using the approved stable local representation."""
|
||||
return (json.dumps(payload, ensure_ascii=False, indent=2) + "\n").encode("utf-8")
|
||||
|
||||
|
||||
def build_unified_diff(document_id: str, before: str, after: str) -> bytes:
|
||||
"""Build the human review view using logical document names only."""
|
||||
if before == after:
|
||||
return b""
|
||||
lines = unified_diff(
|
||||
before.splitlines(keepends=True),
|
||||
after.splitlines(keepends=True),
|
||||
fromfile=f"a/{document_id}.md",
|
||||
tofile=f"b/{document_id}.md",
|
||||
n=3,
|
||||
lineterm="\n",
|
||||
)
|
||||
normalized: list[str] = []
|
||||
line_endings = ("\r\n", "\n", "\r", "\v", "\f", "\x1c", "\x1d", "\x1e", "\x85", "\u2028", "\u2029")
|
||||
for line in lines:
|
||||
for ending in line_endings:
|
||||
if line.endswith(ending):
|
||||
normalized.append(line[: -len(ending)] + "\n")
|
||||
break
|
||||
else:
|
||||
normalized.append(line + "\n\\ No newline at end of file\n")
|
||||
return "".join(normalized).encode("utf-8")
|
||||
|
||||
|
||||
def build_document_report(
|
||||
*,
|
||||
document_id: str,
|
||||
source_label: str,
|
||||
input_markdown: str,
|
||||
result: TransformResult,
|
||||
) -> DocumentReport:
|
||||
"""Render one core result after replaying and verifying every applied batch."""
|
||||
locations, current_markdown = _replay_changes(input_markdown, result)
|
||||
changes: list[JsonValue] = [
|
||||
_change_json(change, location) for change, location in zip(result.changes, locations, strict=True)
|
||||
]
|
||||
errors: list[JsonValue] = [_run_error_json(error) for error in result.errors]
|
||||
residual_proposals: list[JsonValue] = [
|
||||
_residual_proposal_json(residual) for residual in result.residual_proposals
|
||||
]
|
||||
|
||||
if result.status is RunStatus.SUCCESS:
|
||||
cleaned_path: JsonValue = "cleaned.md"
|
||||
diff_path: JsonValue = "changes.diff"
|
||||
cleaned_markdown = current_markdown.encode("utf-8")
|
||||
diff = build_unified_diff(document_id, input_markdown, current_markdown)
|
||||
else:
|
||||
cleaned_path = None
|
||||
diff_path = None
|
||||
cleaned_markdown = None
|
||||
diff = None
|
||||
|
||||
payload: JsonObject = {
|
||||
"schema_version": 1,
|
||||
"document": {
|
||||
"document_id": document_id,
|
||||
"source_label": source_label,
|
||||
},
|
||||
"status": result.status.value,
|
||||
"input_sha256": result.input_sha256,
|
||||
"current_sha256": result.current_sha256,
|
||||
"changes": changes,
|
||||
"errors": errors,
|
||||
"residual_proposals": residual_proposals,
|
||||
"output": {
|
||||
"cleaned_path": cleaned_path,
|
||||
"diff_path": diff_path,
|
||||
},
|
||||
}
|
||||
return DocumentReport(
|
||||
document_id=document_id,
|
||||
source_label=source_label,
|
||||
status=result.status,
|
||||
input_sha256=result.input_sha256,
|
||||
current_sha256=result.current_sha256,
|
||||
change_count=len(result.changes),
|
||||
result_json=encode_json(payload),
|
||||
cleaned_markdown=cleaned_markdown,
|
||||
diff=diff,
|
||||
)
|
||||
@@ -0,0 +1,268 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import stat
|
||||
from hashlib import sha256
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
import mdpolish.artifact_store as artifact_store
|
||||
from mdpolish.artifact_store import ArtifactStoreError, StoredDocument, publish_run
|
||||
|
||||
|
||||
def stored_document(document_id: str = "paper", output: bytes | None = b"cleaned\n") -> StoredDocument:
|
||||
status = "success" if output is not None else "failed"
|
||||
current_sha256 = sha256(output).hexdigest() if output is not None else "2" * 64
|
||||
payload = {
|
||||
"schema_version": 1,
|
||||
"document": {"document_id": document_id, "source_label": f"inputs/{document_id}.md"},
|
||||
"status": status,
|
||||
"input_sha256": "1" * 64,
|
||||
"current_sha256": current_sha256,
|
||||
"changes": [],
|
||||
"errors": [] if status == "success" else [{"error_type": "SyntheticError"}],
|
||||
"residual_proposals": [],
|
||||
"output": {
|
||||
"cleaned_path": "cleaned.md" if output is not None else None,
|
||||
"diff_path": "changes.diff" if output is not None else None,
|
||||
},
|
||||
}
|
||||
return StoredDocument(
|
||||
document_id=document_id,
|
||||
result_json=(json.dumps(payload, indent=2) + "\n").encode(),
|
||||
cleaned_markdown=output,
|
||||
diff=b"" if output is not None else None,
|
||||
output_sha256=sha256(output).hexdigest() if output is not None else None,
|
||||
)
|
||||
|
||||
|
||||
def manifest_json(
|
||||
run_date: str,
|
||||
run_id: str,
|
||||
documents: tuple[StoredDocument, ...],
|
||||
) -> bytes:
|
||||
indexes: list[dict[str, object]] = []
|
||||
statuses: list[str] = []
|
||||
change_count = 0
|
||||
for document in documents:
|
||||
report = json.loads(document.result_json)
|
||||
status = report["status"]
|
||||
statuses.append(status)
|
||||
change_count += len(report["changes"])
|
||||
base = f"documents/{document.document_id}"
|
||||
indexes.append(
|
||||
{
|
||||
"document_id": document.document_id,
|
||||
"source_label": report["document"]["source_label"],
|
||||
"status": status,
|
||||
"input_sha256": report["input_sha256"],
|
||||
"current_sha256": report["current_sha256"],
|
||||
"change_count": len(report["changes"]),
|
||||
"result_path": f"{base}/result.json",
|
||||
"cleaned_path": f"{base}/cleaned.md" if status == "success" else None,
|
||||
"diff_path": f"{base}/changes.diff" if status == "success" else None,
|
||||
}
|
||||
)
|
||||
failed_count = statuses.count("failed")
|
||||
unstable_count = statuses.count("unstable")
|
||||
overall_status = "failed" if failed_count else "unstable" if unstable_count else "success"
|
||||
payload = {
|
||||
"schema_version": 1,
|
||||
"run": {"run_id": run_id, "run_date": run_date, "status": overall_status},
|
||||
"documents": indexes,
|
||||
"summary": {
|
||||
"document_count": len(documents),
|
||||
"success_count": statuses.count("success"),
|
||||
"failed_count": failed_count,
|
||||
"unstable_count": unstable_count,
|
||||
"change_count": change_count,
|
||||
},
|
||||
}
|
||||
return (json.dumps(payload, indent=2) + "\n").encode()
|
||||
|
||||
|
||||
def test_publish_run_creates_private_date_layout_and_status_specific_files(tmp_path: Path) -> None:
|
||||
artifacts_root = tmp_path / "artifacts"
|
||||
documents = (stored_document("success"), stored_document("failed", None))
|
||||
|
||||
run_directory = publish_run(
|
||||
artifacts_root=artifacts_root,
|
||||
run_date="2026-08-22",
|
||||
run_id="example-run",
|
||||
manifest_json=manifest_json("2026-08-22", "example-run", documents),
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
assert run_directory == artifacts_root / "2026-08-22" / "runs" / "example-run"
|
||||
assert json.loads((run_directory / "manifest.json").read_bytes())["run"]["status"] == "failed"
|
||||
assert (run_directory / "documents/success/result.json").is_file()
|
||||
assert (run_directory / "documents/success/cleaned.md").read_bytes() == b"cleaned\n"
|
||||
assert (run_directory / "documents/success/changes.diff").read_bytes() == b""
|
||||
assert (run_directory / "documents/failed/result.json").is_file()
|
||||
assert not (run_directory / "documents/failed/cleaned.md").exists()
|
||||
assert not (run_directory / "documents/failed/changes.diff").exists()
|
||||
|
||||
for directory in (
|
||||
artifacts_root,
|
||||
artifacts_root / "2026-08-22",
|
||||
artifacts_root / "2026-08-22/runs",
|
||||
run_directory,
|
||||
run_directory / "documents",
|
||||
run_directory / "documents/success",
|
||||
):
|
||||
assert stat.S_IMODE(directory.stat().st_mode) == 0o700
|
||||
for artifact_file in run_directory.rglob("*"):
|
||||
if artifact_file.is_file():
|
||||
assert stat.S_IMODE(artifact_file.stat().st_mode) == 0o600
|
||||
|
||||
|
||||
def test_publish_run_rejects_existing_target_without_overwriting(tmp_path: Path) -> None:
|
||||
artifacts_root = tmp_path / "artifacts"
|
||||
documents = (stored_document(),)
|
||||
first_manifest = manifest_json("2026-08-22", "same-run", documents)
|
||||
run_directory = publish_run(
|
||||
artifacts_root=artifacts_root,
|
||||
run_date="2026-08-22",
|
||||
run_id="same-run",
|
||||
manifest_json=first_manifest,
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
with pytest.raises(ArtifactStoreError, match="already exists"):
|
||||
publish_run(
|
||||
artifacts_root=artifacts_root,
|
||||
run_date="2026-08-22",
|
||||
run_id="same-run",
|
||||
manifest_json=first_manifest,
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
assert (run_directory / "manifest.json").read_bytes() == first_manifest
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("run_date", "run_id"),
|
||||
[
|
||||
("2026-8-22", "valid"),
|
||||
("2026-02-30", "valid"),
|
||||
("2026-08-22", "Uppercase"),
|
||||
("2026-08-22", "../escape"),
|
||||
("2026-08-22", "two..dots"),
|
||||
],
|
||||
)
|
||||
def test_publish_run_rejects_unsafe_date_and_run_id(tmp_path: Path, run_date: str, run_id: str) -> None:
|
||||
documents = (stored_document(),)
|
||||
with pytest.raises(ArtifactStoreError):
|
||||
publish_run(
|
||||
artifacts_root=tmp_path / "artifacts",
|
||||
run_date=run_date,
|
||||
run_id=run_id,
|
||||
manifest_json=manifest_json(run_date, run_id, documents),
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
|
||||
def test_publish_run_rejects_inconsistent_or_duplicate_document_artifacts(tmp_path: Path) -> None:
|
||||
valid = stored_document()
|
||||
bad_hash = StoredDocument("paper", valid.result_json, b"output", b"", "0" * 64)
|
||||
with pytest.raises(ArtifactStoreError, match="output hash"):
|
||||
publish_run(
|
||||
artifacts_root=tmp_path / "artifacts-a",
|
||||
run_date="2026-08-22",
|
||||
run_id="bad-hash",
|
||||
manifest_json=manifest_json("2026-08-22", "bad-hash", (bad_hash,)),
|
||||
documents=(bad_hash,),
|
||||
)
|
||||
|
||||
duplicates = (stored_document(), stored_document())
|
||||
with pytest.raises(ArtifactStoreError, match="unique"):
|
||||
publish_run(
|
||||
artifacts_root=tmp_path / "artifacts-b",
|
||||
run_date="2026-08-22",
|
||||
run_id="duplicate",
|
||||
manifest_json=manifest_json("2026-08-22", "duplicate", duplicates),
|
||||
documents=duplicates,
|
||||
)
|
||||
|
||||
|
||||
def test_publish_run_rejects_manifest_path_or_document_mismatch(tmp_path: Path) -> None:
|
||||
documents = (stored_document(),)
|
||||
wrong_date = manifest_json("2026-08-21", "review", documents)
|
||||
with pytest.raises(ArtifactStoreError, match="identity"):
|
||||
publish_run(
|
||||
artifacts_root=tmp_path / "artifacts-a",
|
||||
run_date="2026-08-22",
|
||||
run_id="review",
|
||||
manifest_json=wrong_date,
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
payload = json.loads(manifest_json("2026-08-22", "review", documents))
|
||||
payload["documents"][0]["change_count"] = 99
|
||||
mismatched_index = (json.dumps(payload, indent=2) + "\n").encode()
|
||||
with pytest.raises(ArtifactStoreError, match="index"):
|
||||
publish_run(
|
||||
artifacts_root=tmp_path / "artifacts-b",
|
||||
run_date="2026-08-22",
|
||||
run_id="review",
|
||||
manifest_json=mismatched_index,
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
|
||||
def test_publish_race_does_not_replace_a_new_target(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
documents = (stored_document(),)
|
||||
original_rename = artifact_store._rename_no_replace
|
||||
|
||||
def create_competing_target(source: Path, target: Path) -> None:
|
||||
target.mkdir(mode=0o700)
|
||||
(target / "keep").write_bytes(b"existing")
|
||||
original_rename(source, target)
|
||||
|
||||
monkeypatch.setattr(artifact_store, "_rename_no_replace", create_competing_target)
|
||||
with pytest.raises(ArtifactStoreError, match="already exists"):
|
||||
publish_run(
|
||||
artifacts_root=tmp_path / "artifacts",
|
||||
run_date="2026-08-22",
|
||||
run_id="raced",
|
||||
manifest_json=manifest_json("2026-08-22", "raced", documents),
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
target = tmp_path / "artifacts/2026-08-22/runs/raced"
|
||||
assert (target / "keep").read_bytes() == b"existing"
|
||||
assert not any(path.name.startswith(".raced.") for path in target.parent.iterdir())
|
||||
|
||||
|
||||
def test_write_failure_cleans_temporary_directory_and_does_not_publish(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
original_write = artifact_store._write_private_file
|
||||
call_count = 0
|
||||
|
||||
def fail_second_write(path: Path, content: bytes) -> None:
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 2:
|
||||
raise OSError("synthetic write failure")
|
||||
original_write(path, content)
|
||||
|
||||
monkeypatch.setattr(artifact_store, "_write_private_file", fail_second_write)
|
||||
documents = (stored_document(),)
|
||||
|
||||
with pytest.raises(OSError, match="synthetic"):
|
||||
publish_run(
|
||||
artifacts_root=tmp_path / "artifacts",
|
||||
run_date="2026-08-22",
|
||||
run_id="broken",
|
||||
manifest_json=manifest_json("2026-08-22", "broken", documents),
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
runs_directory = tmp_path / "artifacts/2026-08-22/runs"
|
||||
assert list(runs_directory.iterdir()) == []
|
||||
@@ -0,0 +1,335 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Mapping
|
||||
from datetime import UTC, datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from mdpolish import Component, DocumentSnapshot, Pipeline, ProposedChange, RunStatus, TextEdit, TextSpan
|
||||
from mdpolish.experiment import ExperimentError, InputDocument, ToolMetadata, run_experiment
|
||||
|
||||
|
||||
class ConditionalComponent(Component):
|
||||
@property
|
||||
def component_id(self) -> str:
|
||||
return "test.conditional"
|
||||
|
||||
@property
|
||||
def version(self) -> str:
|
||||
return "1.0.0"
|
||||
|
||||
@property
|
||||
def parameters(self) -> Mapping[str, object]:
|
||||
return {"needle": "old", "replacement": "new"}
|
||||
|
||||
@property
|
||||
def applicability(self) -> str:
|
||||
return "替换测试标记 old，遇到 boom 时模拟组件失败，排除其他内容。"
|
||||
|
||||
def _propose_changes(self, snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]:
|
||||
if snapshot.markdown == "boom":
|
||||
raise RuntimeError("private source")
|
||||
position = snapshot.markdown.find("old")
|
||||
if position < 0:
|
||||
return ()
|
||||
edit = TextEdit(snapshot.sha256, TextSpan(position, position + 3), "old", "new")
|
||||
return (ProposedChange(snapshot.sha256, "replace old test marker", (edit,)),)
|
||||
|
||||
|
||||
class NonIdempotentComponent(Component):
|
||||
@property
|
||||
def component_id(self) -> str:
|
||||
return "test.non-idempotent"
|
||||
|
||||
@property
|
||||
def version(self) -> str:
|
||||
return "1.0.0"
|
||||
|
||||
@property
|
||||
def parameters(self) -> Mapping[str, object]:
|
||||
return {}
|
||||
|
||||
@property
|
||||
def applicability(self) -> str:
|
||||
return "在测试文本末尾反复插入标记，只用于验证 unstable 产物边界。"
|
||||
|
||||
def _propose_changes(self, snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]:
|
||||
position = len(snapshot.markdown)
|
||||
edit = TextEdit(snapshot.sha256, TextSpan(position, position), "", "!")
|
||||
return (ProposedChange(snapshot.sha256, "append synthetic marker", (edit,)),)
|
||||
|
||||
|
||||
def tool_metadata() -> ToolMetadata:
|
||||
return ToolMetadata(
|
||||
name="mdpolish",
|
||||
package_version="0.1.0",
|
||||
python_version="3.13.11",
|
||||
platform="linux-x86_64",
|
||||
git_commit="a" * 40,
|
||||
git_dirty=False,
|
||||
)
|
||||
|
||||
|
||||
def input_document(path: Path, document_id: str) -> InputDocument:
|
||||
return InputDocument(document_id, path, f"inputs/{path.name}")
|
||||
|
||||
|
||||
def test_run_experiment_publishes_manifest_reports_diff_and_preserves_inputs(tmp_path: Path) -> None:
|
||||
first_path = tmp_path / "first.md"
|
||||
second_path = tmp_path / "second.md"
|
||||
first_path.write_bytes(b"before old\r\nafter\r\n")
|
||||
second_path.write_text("unchanged\n", encoding="utf-8")
|
||||
original_first = first_path.read_bytes()
|
||||
original_second = second_path.read_bytes()
|
||||
local_timezone = timezone(timedelta(hours=8))
|
||||
started_at = datetime(2026, 8, 22, 10, 30, tzinfo=local_timezone)
|
||||
completed_at = datetime(2026, 8, 22, 2, 31, tzinfo=UTC)
|
||||
|
||||
result = run_experiment(
|
||||
pipeline=Pipeline([ConditionalComponent()]),
|
||||
documents=(input_document(first_path, "first"), input_document(second_path, "second")),
|
||||
run_id="local-review",
|
||||
artifacts_root=tmp_path / "artifacts",
|
||||
started_at=started_at,
|
||||
completed_at=completed_at,
|
||||
tool=tool_metadata(),
|
||||
)
|
||||
|
||||
assert result.status is RunStatus.SUCCESS
|
||||
assert result.run_directory == tmp_path / "artifacts/2026-08-22/runs/local-review"
|
||||
assert result.document_count == 2
|
||||
assert result.success_count == 2
|
||||
assert result.change_count == 1
|
||||
assert first_path.read_bytes() == original_first
|
||||
assert second_path.read_bytes() == original_second
|
||||
assert (result.run_directory / "documents/first/cleaned.md").read_bytes() == b"before new\r\nafter\r\n"
|
||||
assert (result.run_directory / "documents/second/cleaned.md").read_bytes() == original_second
|
||||
assert (result.run_directory / "documents/second/changes.diff").read_bytes() == b""
|
||||
|
||||
manifest = json.loads((result.run_directory / "manifest.json").read_bytes())
|
||||
assert manifest["run"]["run_date"] == "2026-08-22"
|
||||
assert manifest["run"]["utc_offset"] == "+08:00"
|
||||
assert manifest["run"]["started_at_utc"] == "2026-08-22T02:30:00Z"
|
||||
assert manifest["run"]["completed_at_utc"] == "2026-08-22T02:31:00Z"
|
||||
assert manifest["run"]["retention_until"] == "2026-09-21T02:31:00Z"
|
||||
assert manifest["pipeline"]["components"][0]["component_id"] == "test.conditional"
|
||||
assert manifest["pipeline"]["components"][0]["parameters"] == [
|
||||
["needle", "old"],
|
||||
["replacement", "new"],
|
||||
]
|
||||
assert manifest["summary"] == {
|
||||
"document_count": 2,
|
||||
"success_count": 2,
|
||||
"failed_count": 0,
|
||||
"unstable_count": 0,
|
||||
"change_count": 1,
|
||||
}
|
||||
first_report = json.loads((result.run_directory / "documents/first/result.json").read_bytes())
|
||||
assert first_report["changes"][0]["location"] == {"line": 1, "column": 8}
|
||||
assert first_report["changes"][0]["before"] == "old"
|
||||
assert b"--- a/first.md\n+++ b/first.md\n" in (
|
||||
result.run_directory / "documents/first/changes.diff"
|
||||
).read_bytes()
|
||||
|
||||
|
||||
def test_document_failure_is_isolated_and_never_writes_partial_markdown(tmp_path: Path) -> None:
|
||||
good_path = tmp_path / "good.md"
|
||||
bad_path = tmp_path / "bad.md"
|
||||
good_path.write_text("old", encoding="utf-8")
|
||||
bad_path.write_text("boom", encoding="utf-8")
|
||||
|
||||
result = run_experiment(
|
||||
pipeline=Pipeline([ConditionalComponent()]),
|
||||
documents=(input_document(good_path, "good"), input_document(bad_path, "bad")),
|
||||
run_id="mixed",
|
||||
artifacts_root=tmp_path / "artifacts",
|
||||
started_at=datetime(2026, 8, 22, 10, tzinfo=UTC),
|
||||
completed_at=datetime(2026, 8, 22, 10, 1, tzinfo=UTC),
|
||||
tool=tool_metadata(),
|
||||
)
|
||||
|
||||
assert result.status is RunStatus.FAILED
|
||||
assert result.success_count == 1
|
||||
assert result.failed_count == 1
|
||||
assert (result.run_directory / "documents/good/cleaned.md").read_text() == "new"
|
||||
assert (result.run_directory / "documents/bad/result.json").is_file()
|
||||
assert not (result.run_directory / "documents/bad/cleaned.md").exists()
|
||||
assert not (result.run_directory / "documents/bad/changes.diff").exists()
|
||||
bad_report = json.loads((result.run_directory / "documents/bad/result.json").read_bytes())
|
||||
assert bad_report["status"] == "failed"
|
||||
assert "partial_markdown" not in bad_report
|
||||
|
||||
|
||||
def test_preflight_rejects_invalid_utf8_without_running_or_creating_artifacts(tmp_path: Path) -> None:
|
||||
invalid_path = tmp_path / "invalid.md"
|
||||
invalid_path.write_bytes(b"\xff")
|
||||
artifacts_root = tmp_path / "artifacts"
|
||||
|
||||
with pytest.raises(ExperimentError, match="strict UTF-8"):
|
||||
run_experiment(
|
||||
pipeline=Pipeline([ConditionalComponent()]),
|
||||
documents=(input_document(invalid_path, "invalid"),),
|
||||
run_id="invalid-input",
|
||||
artifacts_root=artifacts_root,
|
||||
started_at=datetime(2026, 8, 22, tzinfo=UTC),
|
||||
tool=tool_metadata(),
|
||||
)
|
||||
|
||||
assert not artifacts_root.exists()
|
||||
|
||||
|
||||
def test_preflight_rejects_invalid_document_manifests_before_creating_artifacts(tmp_path: Path) -> None:
|
||||
source_path = tmp_path / "paper.md"
|
||||
source_path.write_text("content", encoding="utf-8")
|
||||
directory_path = tmp_path / "directory"
|
||||
directory_path.mkdir()
|
||||
missing_path = tmp_path / "missing.md"
|
||||
cases = (
|
||||
(input_document(source_path, "duplicate"), input_document(source_path, "duplicate")),
|
||||
(input_document(source_path, "first"), input_document(source_path, "second")),
|
||||
(input_document(missing_path, "missing"),),
|
||||
(input_document(directory_path, "directory"),),
|
||||
(input_document(source_path, "two..dots"),),
|
||||
)
|
||||
|
||||
for index, documents in enumerate(cases):
|
||||
artifacts_root = tmp_path / f"artifacts-{index}"
|
||||
with pytest.raises(ExperimentError):
|
||||
run_experiment(
|
||||
pipeline=Pipeline([]),
|
||||
documents=documents,
|
||||
run_id="preflight",
|
||||
artifacts_root=artifacts_root,
|
||||
started_at=datetime(2026, 8, 22, tzinfo=UTC),
|
||||
tool=tool_metadata(),
|
||||
)
|
||||
assert not artifacts_root.exists()
|
||||
|
||||
|
||||
def test_preflight_rejects_an_output_path_nested_under_an_input_file(tmp_path: Path) -> None:
|
||||
source_path = tmp_path / "paper.md"
|
||||
source_path.write_text("content", encoding="utf-8")
|
||||
|
||||
with pytest.raises(ExperimentError, match="cannot overlap"):
|
||||
run_experiment(
|
||||
pipeline=Pipeline([]),
|
||||
documents=(input_document(source_path, "paper"),),
|
||||
run_id="overlap",
|
||||
artifacts_root=source_path,
|
||||
started_at=datetime(2026, 8, 22, tzinfo=UTC),
|
||||
tool=tool_metadata(),
|
||||
)
|
||||
|
||||
assert source_path.read_text(encoding="utf-8") == "content"
|
||||
|
||||
|
||||
def test_utf8_bom_crlf_and_missing_final_newline_are_preserved_exactly(tmp_path: Path) -> None:
|
||||
source_path = tmp_path / "bom.md"
|
||||
empty_path = tmp_path / "empty.md"
|
||||
source_bytes = b"\xef\xbb\xbfhead\r\nlast"
|
||||
source_path.write_bytes(source_bytes)
|
||||
empty_path.write_bytes(b"")
|
||||
|
||||
result = run_experiment(
|
||||
pipeline=Pipeline([]),
|
||||
documents=(input_document(source_path, "bom"), input_document(empty_path, "empty")),
|
||||
run_id="byte-preservation",
|
||||
artifacts_root=tmp_path / "artifacts",
|
||||
started_at=datetime(2026, 8, 22, tzinfo=UTC),
|
||||
completed_at=datetime(2026, 8, 22, 0, 1, tzinfo=UTC),
|
||||
tool=tool_metadata(),
|
||||
)
|
||||
|
||||
assert source_path.read_bytes() == source_bytes
|
||||
assert (result.run_directory / "documents/bom/cleaned.md").read_bytes() == source_bytes
|
||||
assert (result.run_directory / "documents/bom/changes.diff").read_bytes() == b""
|
||||
assert empty_path.read_bytes() == b""
|
||||
assert (result.run_directory / "documents/empty/cleaned.md").read_bytes() == b""
|
||||
assert (result.run_directory / "documents/empty/changes.diff").read_bytes() == b""
|
||||
|
||||
|
||||
def test_unstable_document_only_publishes_a_result_json(tmp_path: Path) -> None:
|
||||
source_path = tmp_path / "paper.md"
|
||||
source_path.write_text("content", encoding="utf-8")
|
||||
|
||||
result = run_experiment(
|
||||
pipeline=Pipeline([NonIdempotentComponent()]),
|
||||
documents=(input_document(source_path, "paper"),),
|
||||
run_id="unstable",
|
||||
artifacts_root=tmp_path / "artifacts",
|
||||
started_at=datetime(2026, 8, 22, tzinfo=UTC),
|
||||
completed_at=datetime(2026, 8, 22, 0, 1, tzinfo=UTC),
|
||||
tool=tool_metadata(),
|
||||
)
|
||||
|
||||
assert result.status is RunStatus.UNSTABLE
|
||||
assert result.unstable_count == 1
|
||||
assert (result.run_directory / "documents/paper/result.json").is_file()
|
||||
assert not (result.run_directory / "documents/paper/cleaned.md").exists()
|
||||
assert not (result.run_directory / "documents/paper/changes.diff").exists()
|
||||
manifest = json.loads((result.run_directory / "manifest.json").read_bytes())
|
||||
assert manifest["run"]["status"] == "unstable"
|
||||
|
||||
|
||||
def test_existing_same_date_run_is_rejected_without_overwriting(tmp_path: Path) -> None:
|
||||
source_path = tmp_path / "paper.md"
|
||||
source_path.write_text("old", encoding="utf-8")
|
||||
artifacts_root = tmp_path / "artifacts"
|
||||
pipeline = Pipeline([ConditionalComponent()])
|
||||
documents = (input_document(source_path, "paper"),)
|
||||
started_at = datetime(2026, 8, 22, tzinfo=UTC)
|
||||
completed_at = datetime(2026, 8, 22, 0, 1, tzinfo=UTC)
|
||||
tool = tool_metadata()
|
||||
first = run_experiment(
|
||||
pipeline=pipeline,
|
||||
documents=documents,
|
||||
run_id="same",
|
||||
artifacts_root=artifacts_root,
|
||||
started_at=started_at,
|
||||
completed_at=completed_at,
|
||||
tool=tool,
|
||||
)
|
||||
original_manifest = (first.run_directory / "manifest.json").read_bytes()
|
||||
|
||||
with pytest.raises(ExperimentError, match="already exists"):
|
||||
run_experiment(
|
||||
pipeline=pipeline,
|
||||
documents=documents,
|
||||
run_id="same",
|
||||
artifacts_root=artifacts_root,
|
||||
started_at=started_at,
|
||||
completed_at=completed_at,
|
||||
tool=tool,
|
||||
)
|
||||
|
||||
assert (first.run_directory / "manifest.json").read_bytes() == original_manifest
|
||||
|
||||
|
||||
def test_same_run_id_on_another_local_date_uses_a_separate_directory(tmp_path: Path) -> None:
|
||||
source_path = tmp_path / "paper.md"
|
||||
source_path.write_text("unchanged", encoding="utf-8")
|
||||
artifacts_root = tmp_path / "artifacts"
|
||||
|
||||
first = run_experiment(
|
||||
pipeline=Pipeline([]),
|
||||
documents=(input_document(source_path, "paper"),),
|
||||
run_id="daily",
|
||||
artifacts_root=artifacts_root,
|
||||
started_at=datetime(2026, 8, 22, 23, tzinfo=UTC),
|
||||
completed_at=datetime(2026, 8, 22, 23, 1, tzinfo=UTC),
|
||||
tool=tool_metadata(),
|
||||
)
|
||||
second = run_experiment(
|
||||
pipeline=Pipeline([]),
|
||||
documents=(input_document(source_path, "paper"),),
|
||||
run_id="daily",
|
||||
artifacts_root=artifacts_root,
|
||||
started_at=datetime(2026, 8, 23, 0, tzinfo=UTC),
|
||||
completed_at=datetime(2026, 8, 23, 0, 1, tzinfo=UTC),
|
||||
tool=tool_metadata(),
|
||||
)
|
||||
|
||||
assert first.run_directory.parent.parent.name == "2026-08-22"
|
||||
assert second.run_directory.parent.parent.name == "2026-08-23"
|
||||
@@ -0,0 +1,177 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import replace
|
||||
|
||||
import pytest
|
||||
|
||||
from mdpolish import Component, DocumentSnapshot, Pipeline, ProposedChange, RunStatus, TextEdit, TextSpan
|
||||
from mdpolish.reporting import ReportingError, build_document_report, build_unified_diff
|
||||
|
||||
|
||||
class ReplaceComponent(Component):
|
||||
def __init__(self, needle: str, replacement: str, component_id: str) -> None:
|
||||
self.needle = needle
|
||||
self.replacement = replacement
|
||||
self._component_id = component_id
|
||||
|
||||
@property
|
||||
def component_id(self) -> str:
|
||||
return self._component_id
|
||||
|
||||
@property
|
||||
def version(self) -> str:
|
||||
return "1.0.0"
|
||||
|
||||
@property
|
||||
def parameters(self) -> Mapping[str, object]:
|
||||
return {"needle": self.needle, "replacement": self.replacement}
|
||||
|
||||
@property
|
||||
def applicability(self) -> str:
|
||||
return "处理精确测试标记，要求完整匹配，排除其他文本。"
|
||||
|
||||
def _propose_changes(self, snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]:
|
||||
position = snapshot.markdown.find(self.needle)
|
||||
if position < 0:
|
||||
return ()
|
||||
edit = TextEdit(
|
||||
snapshot.sha256,
|
||||
TextSpan(position, position + len(self.needle)),
|
||||
self.needle,
|
||||
self.replacement,
|
||||
)
|
||||
return (ProposedChange(snapshot.sha256, f"replace {self.needle}", (edit,)),)
|
||||
|
||||
|
||||
class ExplodingComponent(ReplaceComponent):
|
||||
def __init__(self) -> None:
|
||||
super().__init__("unused", "unused-replacement", "test.exploding")
|
||||
|
||||
def _propose_changes(self, snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]:
|
||||
raise RuntimeError(f"private: {snapshot.markdown}")
|
||||
|
||||
|
||||
def test_success_report_replays_multiple_component_snapshots_and_derives_locations() -> None:
|
||||
markdown = "x\r\ntarget\n"
|
||||
pipeline = Pipeline(
|
||||
[
|
||||
ReplaceComponent("x", "XX", "test.expand"),
|
||||
ReplaceComponent("target", "done", "test.target"),
|
||||
]
|
||||
)
|
||||
result = pipeline.transform(markdown)
|
||||
|
||||
report = build_document_report(
|
||||
document_id="paper",
|
||||
source_label="data/md/paper.md",
|
||||
input_markdown=markdown,
|
||||
result=result,
|
||||
)
|
||||
payload = json.loads(report.result_json)
|
||||
|
||||
assert result.status is RunStatus.SUCCESS
|
||||
assert report.cleaned_markdown == b"XX\r\ndone\n"
|
||||
assert payload["changes"][0]["location"] == {"line": 1, "column": 1}
|
||||
assert payload["changes"][1]["location"] == {"line": 2, "column": 1}
|
||||
assert payload["changes"][1]["before"] == "target"
|
||||
assert payload["changes"][1]["after"] == "done"
|
||||
assert payload["changes"][1]["before_sha256"] == result.changes[1].before_sha256
|
||||
assert payload["changes"][1]["after_sha256"] == result.changes[1].after_sha256
|
||||
assert report.diff is not None
|
||||
assert b"--- a/paper.md\n+++ b/paper.md\n" in report.diff
|
||||
assert b"data/md/paper.md" not in report.diff
|
||||
|
||||
|
||||
def test_zero_change_success_still_has_cleaned_markdown_and_empty_diff() -> None:
|
||||
markdown = "中文\nCafe\u0301\n"
|
||||
result = Pipeline([]).transform(markdown)
|
||||
|
||||
report = build_document_report(
|
||||
document_id="unchanged",
|
||||
source_label="论文.md",
|
||||
input_markdown=markdown,
|
||||
result=result,
|
||||
)
|
||||
|
||||
assert report.cleaned_markdown == markdown.encode()
|
||||
assert report.diff == b""
|
||||
assert report.change_count == 0
|
||||
assert report.result_json.endswith(b"\n")
|
||||
assert "论文".encode() in report.result_json
|
||||
assert b"\\u4e2d" not in report.result_json
|
||||
|
||||
|
||||
def test_diff_preserves_final_newline_changes_and_uses_report_newlines() -> None:
|
||||
diff = build_unified_diff("paper", "same\r\n", "same")
|
||||
|
||||
assert b"--- a/paper.md\n+++ b/paper.md\n" in diff
|
||||
assert b"-same\n+same\n\\ No newline at end of file\n" in diff
|
||||
assert b"\r" not in diff
|
||||
|
||||
|
||||
def test_failed_report_keeps_change_audit_but_does_not_render_partial_markdown() -> None:
|
||||
pipeline = Pipeline(
|
||||
[
|
||||
ReplaceComponent("a", "b", "test.first"),
|
||||
ExplodingComponent(),
|
||||
]
|
||||
)
|
||||
result = pipeline.transform("a")
|
||||
|
||||
report = build_document_report(
|
||||
document_id="failed",
|
||||
source_label="failed.md",
|
||||
input_markdown="a",
|
||||
result=result,
|
||||
)
|
||||
payload = json.loads(report.result_json)
|
||||
|
||||
assert result.status is RunStatus.FAILED
|
||||
assert report.cleaned_markdown is None
|
||||
assert report.diff is None
|
||||
assert payload["output"] == {"cleaned_path": None, "diff_path": None}
|
||||
assert len(payload["changes"]) == 1
|
||||
assert payload["errors"][0]["error_type"] == "RuntimeError"
|
||||
assert "private" not in payload["errors"][0]["message"]
|
||||
assert "partial_markdown" not in payload
|
||||
|
||||
|
||||
def test_unstable_report_serializes_residual_proposals_without_partial_markdown() -> None:
|
||||
pipeline = Pipeline(
|
||||
[
|
||||
ReplaceComponent("bad", "good", "test.to-good"),
|
||||
ReplaceComponent("good", "bad", "test.to-bad"),
|
||||
]
|
||||
)
|
||||
result = pipeline.transform("bad")
|
||||
|
||||
report = build_document_report(
|
||||
document_id="unstable",
|
||||
source_label="unstable.md",
|
||||
input_markdown="bad",
|
||||
result=result,
|
||||
)
|
||||
payload = json.loads(report.result_json)
|
||||
|
||||
assert result.status is RunStatus.UNSTABLE
|
||||
assert report.cleaned_markdown is None
|
||||
assert report.diff is None
|
||||
assert payload["residual_proposals"][0]["proposal"]["reason"] == "replace bad"
|
||||
assert payload["residual_proposals"][0]["proposal"]["edits"][0]["expected_text"] == "bad"
|
||||
assert "partial_markdown" not in payload
|
||||
|
||||
|
||||
def test_report_rejects_a_tampered_change_hash_chain() -> None:
|
||||
result = Pipeline([ReplaceComponent("a", "b", "test.replace")]).transform("a")
|
||||
tampered_change = replace(result.changes[0], after_sha256="0" * 64)
|
||||
tampered_result = replace(result, changes=(tampered_change,))
|
||||
|
||||
with pytest.raises(ReportingError, match="batch hash"):
|
||||
build_document_report(
|
||||
document_id="tampered",
|
||||
source_label="tampered.md",
|
||||
input_markdown="a",
|
||||
result=tampered_result,
|
||||
)
|
||||
Reference in New Issue
Block a user