diff --git a/README.md b/README.md index e337ecf..9cce41f 100644 --- a/README.md +++ b/README.md @@ -7,8 +7,9 @@ profile 表达论文、政务文档、RAG、文档对比等不同需求。 仓库当前已从纯文档治理进入第一版核心实现阶段：已经提供可安装的 Python 内存处理包和测试，用于验证 -组件组合、精确修改和审计协议，并已有一个严格整行匹配的论文清洗组件。仓库仍不提供完整规则集、命令行工具、 -文件读写适配器或生产接口，因此目前还不是拿来即可完成整篇文档清洗的成品工具。 +组件组合、精确修改和审计协议，并已有一个严格整行匹配的论文清洗组件。仓库另有一个只供本地评审的实验脚本， +可以保存指定论文的成功输出、审计和 diff。仓库仍不提供完整规则集、公共命令行工具、通用文件适配器或生产接口， +因此目前还不是拿来即可完成整篇文档清洗的成品工具。 ## 当前阶段 @@ -17,11 +18,13 @@ profile 表达论文、政务文档、RAG、文档对比等不同需求。 - 提供可安装的 Python 3.11+ 内存处理包，运行时只依赖标准库； - 已实现不可变数据契约、组件基类、原子修改执行器、顺序流水线、审计记录和最终稳定性复查； - 当前唯一正式组件是 `paper.arxiv_submission_stamp`，只删除严格整行匹配的 arXiv 提交边栏戳； -- 该组件已在 5 份论文 Markdown 上只读验证，只命中 sim 和 springer 各一处，合法参考文献保持不变； -- 当前基础检查为 Ruff、mypy 和 87 项 pytest 测试，实际命令见本文“当前可用检查”。 +- 已有仓库内实验运行层，能严格读取显式清单、保存成功 Markdown、JSON 审计和 unified diff，并保持输入不变； +- 该组件已对 5 份论文 Markdown 完成保存型实验，只修改 sim 和 springer 各一处，合法参考文献保持不变； +- 当前基础检查为 Ruff、mypy 和 113 项 pytest 测试，实际命令见本文“当前可用检查”。 -项目还没有完整清洗规则集、Markdown/HTML parser、profile 格式、文件读写、CLI、批处理或生产接口。当前组件 -只能证明第一条严格规则已经闭环，不能据此认为论文、GovDoc、表格或图片已经具备完整清洗能力。 +项目还没有完整清洗规则集、Markdown/HTML parser、profile 格式、通用文件输入接口、公共 CLI、通用批处理或生产接口。 +当前实验脚本只固定运行已批准的 5 份论文和一个组件；它只能证明第一条严格规则已经闭环，不能据此认为论文、GovDoc、 +表格或图片已经具备完整清洗能力。 ## 服务对象与复用目标 @@ -43,6 +46,9 @@ GovDoc 目录、具体客户名称或某一转换器的固定输出路径。 两组数据都不是可提交的自动测试 fixture。`data/` 已被 Git 忽略，清洗实验不得覆盖这些输入。 +本地清洗实验产物位于 `artifacts//runs//`。产物可能包含完整原文，同样受 Git 忽略， +默认保留 30 个日历日，不得提交或复制到外部系统。当前只批准为上述 5 份论文副本保存产物，未批准保存 GovDoc 输出。 + ## 面向复用的设计原则 - **通用核心**：只接收 Markdown；第一版只执行确定、可审计的精确修改，不读取 PDF、图片或转换器 JSON； @@ -66,18 +72,26 @@ mdpolish/ │ ├── __init__.py # 第一版核心公共导出 │ ├── component.py # 组件基类和元数据契约 │ ├── edits.py # 文本编辑验证与原子应用 +│ ├── experiment.py # 本地实验输入预检与批量编排 │ ├── models.py # 不可变数据模型和运行状态 │ ├── pipeline.py # 顺序执行和最终稳定性复查 +│ ├── reporting.py # JSON 审计、行列位置和 unified diff +│ ├── artifact_store.py # 私有产物目录和原子发布 │ ├── py.typed # 类型信息声明 │ └── components/ │ ├── __init__.py │ └── arxiv_submission_stamp.py +├── scripts/ +│ └── run_clindb_arxiv_experiment.py # 固定 5 份论文的仓库内实验入口 ├── tests/ │ ├── test_arxiv_submission_stamp.py +│ ├── test_artifact_store.py │ ├── test_component.py │ ├── test_edits.py +│ ├── test_experiment.py │ ├── test_models.py -│ └── test_pipeline.py +│ ├── test_pipeline.py +│ └── test_reporting.py ├── data/ # 本地测试数据；Git 忽略；此处只展开常用入口 │ └── md/ │ ├── dmp.md @@ -85,6 +99,8 @@ mdpolish/ │ ├── jama.md │ ├── sim.md │ └── springer.md +├── artifacts/ # 本地敏感实验产物；Git 忽略 +│ └── /runs// └── research-wiki/ ├── README.md # Wiki 分类与维护规则 ├── design/ # 批准前的选择；批准后冻结 @@ -104,9 +120,11 @@ mdpolish/ 4. 与任务直接相关的 `research-wiki/design/` 记录。 第一版核心的当前机制见 `research-wiki/explanation/first-executable-core.md`，首个真实组件见 -`research-wiki/explanation/arxiv-submission-stamp.md`。下一项候选是 ClinDB 范围中的 HTML 实体双重转义， +`research-wiki/explanation/arxiv-submission-stamp.md`，本地实验产物机制见 +`research-wiki/explanation/local-experiment-artifacts.md`，实际运行步骤见 +`research-wiki/guides/run-local-clindb-arxiv-experiment.md`。下一项候选是 ClinDB 范围中的 HTML 实体双重转义， 但必须先用新 design 确定只在哪些 HTML 范围替换、如何避开字面示例以及实体替换边界。解析器、CLI、文件适配器、 -profile 格式和独立检查能力仍需分别设计，不能从当前核心或单个组件存在推导为已经获批。 +profile 格式和独立检查能力仍需分别设计；当前本地实验适配器不能被推导成这些公共接口已经获批。 ## 当前可用检查 @@ -117,7 +135,7 @@ python -m venv .venv # 第一版核心的基础验收 .venv/bin/ruff check . -.venv/bin/mypy src tests +.venv/bin/mypy src tests scripts/run_clindb_arxiv_experiment.py .venv/bin/pytest # 两份 Agent 入口除标题外必须一致；无输出且退出码为 0 表示通过 @@ -130,6 +148,6 @@ find research-wiki -maxdepth 2 -type f | sort git status --short ``` -上述安装和三项基础验收已于 2026-08-22 在 Python 3.13.11 环境实际运行：Ruff 通过，mypy 检查 12 个源码与 -测试文件无问题，pytest 共 87 项测试通过。`requires-python` 仍以 `pyproject.toml` 声明的 Python 3.11 及以上为准； +上述安装和三项基础验收已于 2026-08-22 在 Python 3.13.11 环境实际运行：Ruff 通过，mypy 检查 19 个源码、 +测试和实验脚本文件无问题，pytest 共 113 项测试通过。`requires-python` 仍以 `pyproject.toml` 声明的 Python 3.11 及以上为准； 本次结果不等于已经在每个受支持版本上完成兼容性验证。 diff --git a/research-wiki/design/0005-local-experiment-runner-and-artifacts.md b/research-wiki/design/0005-local-experiment-runner-and-artifacts.md new file mode 100644 index 0000000..153aff9 --- /dev/null +++ b/research-wiki/design/0005-local-experiment-runner-and-artifacts.md @@ -0,0 +1,545 @@ +# 0005：本地清洗实验运行与产物保存契约 + +## 状态 + +已批准并冻结（2026-08-22）。 + +本设计扩展 `0002` 中已划定的输入输出适配层，并落实 `0003` 中“由未来适配层决定是否保存成功输出”的边界。 +它不修改 `DocumentSnapshot`、`ProposedChange`、`Change`、`TransformResult` 或组件契约，也不改变 +`0004` 当时只读验证已经完成的历史事实。 + +`0004` 没有授权在那一轮验证中保存真实清洗产物，不等于永久禁止未来的本地实验输出。本设计批准后， +新的实验可以在本设计的位置、保留周期和隐私边界内保存产物。 + +## 1. 问题与可观察现象 + +当前内存核心已经能返回清洗后 Markdown 和逐项 `Change`，但仓库还没有文件读取、安全输出、JSON 审计或 +人可读 diff。当前只能通过临时 Python 代码在内存中查看结果，会带来四个实际问题： + +1. 评审者无法直接打开清洗后 Markdown； +2. 没有统一 diff，难以确认“只改了批准内容”； +3. `Change` 只在 Python 对象中，进程结束后无法复核组件、理由、原文和改后文本； +4. 批量处理时无法追溯实际输入、工具版本、组件顺序和每份文档的结束状态。 + +真实清洗流程必须能保存结果，但不能因此让组件读写文件，也不能把 `failed` 或 `unstable` 的部分文本冒充 +成功输出。保存的 diff、`before` / `after` 和清洗后 Markdown 都可能还原真实原文，因此也不能当成普通日志 +提交到 Git。 + +## 2. 目标与非目标 + +### 2.1 目标 + +- 建立只服务本地评审的最小实验运行层； +- 显式读取调用方列出的 Markdown 文件，不递归猜测数据集； +- 继续使用同一个 `Pipeline.transform()` 完成清洗，不在适配层实现第二套规则； +- 对每份成功文档保存独立的清洗后 Markdown、机器可读审计和人可读 diff； +- 保存整次运行清单，使输入范围、工具环境、组件顺序、哈希、状态和产物位置可追溯； +- 从文件级别区分 `success`、`failed` 和 `unstable`，不为后两者生成正式清洗文档； +- 不覆盖、改名或移动任何输入； +- 将含真实文本的产物限定为 Git 忽略的本地敏感数据。 + +### 2.2 非目标 + +- 不实现 Inspector、Finding、人工建议或审核状态； +- 不实现 Profile 对象、TOML/YAML/JSON 配置文件或组件自动发现； +- 不提供安装后稳定的公共 CLI、服务接口、CI 集成或 Web 界面； +- 不原地覆盖、备份或恢复输入文件； +- 不把 `partial_markdown` 写成清洗产物； +- 不建设数据库、远程对象存储、任务调度或长期审计系统； +- 不引入 Markdown parser、AST、新清洗规则或多轮执行； +- 不定义真实数据适合公开、共享或长期归档的条件； +- 不处理 PDF、DOCX、图片、转换器 JSON 或文件间关联。 + +## 3. 当前基础和不改变的边界 + +当前 `TransformResult` 已经包含持久化所需的主要运行事实： + +| 对象 | 已有信息 | 本设计的处理 | +| --- | --- | --- | +| `ComponentInfo` | 组件标识、版本、参数、适用边界 | 写入运行清单 | +| `Change` | 候选引用、理由、范围、`before`、`after`、批次前后哈希 | 写入文档审计 | +| `RunError` | 组件、阶段、错误类型和安全说明 | 写入文档审计 | +| `ResidualProposal` | 最终复查的有效残留修改 | 只写入 `failed` / `unstable` 审计 | +| `TransformResult` | 状态、输入/当前哈希、修改、错误、成功或部分文本 | 决定可以生成哪些产物 | + +本设计只在核心外补充文件身份、运行身份、环境和序列化信息。以下边界保持不变： + +- 组件仍只接收 `DocumentSnapshot` 并返回 `ProposedChange`； +- 组件不知道文件路径、运行 ID、输出目录或 JSON 格式； +- `Pipeline` 仍只处理内存 Markdown； +- Python 字符下标仍是修改位置的唯一权威； +- reporter 不重新判断业务规则，只表示现有运行结果； +- 文件适配层不把读写成功冒充成清洗成功。 + +## 4. 方案比较 + +### 4.1 继续只返回内存结果 + +不增加新契约，但每次真实实验都需要临时代码，结果无法留存，人也无法方便查看完整 Markdown 和 diff。 +不采用。 + +### 4.2 只保存清洗后 Markdown + +人可以打开结果，但无法证明是哪个组件改了什么，也无法区分输入不同、组件不同还是工具不同。 +输出文件与内存审计会脱节。不采用。 + +### 4.3 建设完整 CLI、Profile 和批处理平台 + +能够一次解决对外运行，但当前只有一个无参数组件，配置、退出码、安装命令和公共兼容都没有足够消费者与失败样本。 +这会过早固定外部接口。不采用。 + +### 4.4 本地实验运行层加结构化产物 + +调用方显式提供文档和 `Pipeline`，外层只负责严格读取、调用核心、表示结果和写入新目录。它不提供稳定公共 CLI， +也不引入配置语言。每次运行同时保存成功文本、JSON 审计和 diff。 + +采用此方案。它能满足当前 5 份论文的人工复核，同时不迫使核心、组件和未来对外工具提前承担未经验证的接口。 + +## 5. 总体边界与依赖方向 + +```text +调用方显式列出文档和顺序 + │ + ▼ + 本地实验运行层 + │ │ + │ ├── 读取 UTF-8 Markdown + │ └── 记录文档身份与环境 + ▼ + Pipeline.transform(markdown) + │ + ▼ + TransformResult + │ + ▼ + reporter / 产物写入 + │ │ │ + ▼ ▼ ▼ + JSON cleaned.md unified diff +``` + +依赖必须保持从外向内： + +```text +本地实验脚本 + │ + ▼ +experiment.py + ├────────► pipeline.py ──────► 当前内存核心 + ├────────► reporting.py ─────► models.py + └────────► artifact_store.py +``` + +- `reporting.py` 可以依赖数据模型，不读文件，不调用组件； +- `artifact_store.py` 只负责产物路径、权限、写入校验和原子发布，不调用流水线，不解释修改语义； +- `experiment.py` 可以依赖 `Pipeline`、reporter 和 artifact store，负责输入预检与文档级批量编排； +- 业务组件和内存核心不反向导入这三个模块； +- 日期目录、文件名或未来存储位置的变化应限制在 artifact store，不改变 reporter 或流水线； +- 实验脚本可以显式组装 ClinDB 流水线，但通用模块不得硬编码 `data/md/`、DOI 或论文缩写。 + +## 6. 输入契约 + +### 6.1 显式文档清单 + +调用方必须按顺序提供非空文档清单。每项至少包含： + +- `document_id`：当前运行内唯一的稳定标识； +- `source_path`：实际读取路径； +- `source_label`：写入清单的可展示来源，用于避免无条件泄露绝对路径。 + +`document_id` 只允许小写 ASCII 字母、数字、`.`、`_` 和 `-`，必须以字母或数字开头，不得包含路径分隔符或 +`..` 路径段。文档 ID 重复、源路径重复、源文件不存在或不是普通文件时，整次运行在产生最终产物目录前失败。 + +第一个实际实验的文档 ID 固定为 `dmp`、`ejhf`、`jama`、`sim` 和 `springer`，输入只是 `data/md/` 中对应的 +5 份本地副本。本设计不授权对仓库外 GovDoc 数据保存清洗产物。 + +### 6.2 编码和文本保真 + +- 输入以二进制读取，再使用严格 UTF-8 解码； +- 无效 UTF-8 使整次运行在预检阶段失败，不使用替换字符或自动猜测编码； +- 不剔除 UTF-8 BOM，不规范化 Unicode，不转换换行，不补文件末换行； +- 清洗后 Markdown 使用 UTF-8 严格编码，保留内存输出的所有字符。 + +核心快照哈希仍以 Markdown 字符串重新编码后的 UTF-8 字节为权威。适配层另外记录原始文件字节的 SHA-256； +对有效 UTF-8 输入，两者应当一致。如果不一致，视为适配层错误并停止运行。 + +### 6.3 流水线来源 + +实验运行层接收调用方已显式建立的 `Pipeline`。它不根据文件名、内容或安装环境选择组件，也不在不同文档之间改变 +组件顺序。当前没有 Profile 身份；运行清单直接记录每次结果中已验证的有序 `ComponentInfo`。 + +## 7. 运行状态与批量语义 + +### 7.1 文档级别 + +每份文档独立调用同一个 `Pipeline`。文档结果直接沿用核心状态： + +- `success`：可以保存正式清洗 Markdown、文档审计和 diff； +- `failed`：只保存文档审计和错误，不持久化 `partial_markdown`； +- `unstable`：只保存文档审计和残留候选，不持久化 `partial_markdown`。 + +对 `failed` 和 `unstable`，之前已经发生的 `Change` 仍必须写入审计，但不得生成名为 `cleaned.md`、`output.md` +或类似正式输出的文件。 + +### 7.2 批量级别 + +单份文档的核心失败或不稳定不中断其他已预检文档。运行层继续按调用方给定的顺序处理，并在清单中保留每份文档的 +状态。整体状态使用以下优先级： + +```text +任一文档 failed → 整体 failed +否则任一文档 unstable → 整体 unstable +否则 → 整体 success +``` + +这是实验汇总状态，不改变核心 `RunStatus`。整体失败时，其他文档的成功产物可以保留，但 `manifest.json` +必须明确该批次并非全部成功。 + +### 7.3 预检与致命错误 + +在调用任何组件前，运行层必须完成： + +1. 验证运行日期、UTC 偏移、运行 ID 和所有文档 ID； +2. 验证输出根目录不会落入任何输入文件路径； +3. 确认最终运行目录不存在； +4. 读取所有输入字节、严格解码并计算哈希； +5. 验证所有文档清单字段。 + +任一预检失败都使整次运行立即失败，不调用 `Pipeline`，不生成最终运行目录。产物写入、JSON 序列化或最终目录 +发布失败同样是整次运行的致命错误，不得把不完整临时目录报告为已完成实验。 + +## 8. 产物目录与发布 + +### 8.1 固定位置 + +仓库内本地实验产物固定放在： + +```text +artifacts//runs// +``` + +`artifacts/` 已由 `.gitignore` 忽略。本设计不授权从 Git 忽略中移除该目录，也不允许通过 `git add -f` +强制提交任何产物。 + +`run_date` 使用 `YYYY-MM-DD`，表示实验开始时本机时区中的日历日期。本地实验脚本在启动时只计算一次日期和 +UTC 偏移，再把它们显式传给运行层；运行层不在处理不同文档时重新读取日期。`run_date` 必须能够按 ISO 8601 +日历日期严格解析，日期目录不接受其他格式。 + +`run_id` 由调用方显式提供，适用与 `document_id` 相同的安全字符规则。不使用当前时间、随机数或文件名暗中生成 +运行 ID。同一日期下的目标运行目录已存在时必须拒绝运行，不覆盖、合并或自动加后缀。同一个 `run_id` 可以在 +不同日期下再次使用，两次运行仍由完整日期路径区分。 + +### 8.2 目录结构 + +```text +artifacts/ +└── / + └── runs/ + └── / + ├── manifest.json + └── documents/ + └── / + ├── result.json + ├── cleaned.md # 仅 success + └── changes.diff # 仅 success +``` + +所有 `success` 文档都生成 `cleaned.md` 和 `changes.diff`。即使零修改，`cleaned.md` 仍完整保存成功输出， +`changes.diff` 是长度为零的文件。这使每份成功文档的产物结构一致。 + +`failed` 和 `unstable` 文档目录只包含 `result.json`。不创建隐含部分结果的 Markdown 或 diff。 + +### 8.3 原子发布 + +artifact store 先在 `artifacts//runs/` 下创建当次运行专用的临时目录，完成全部文件写入、 +哈希校验和清单校验后， +再在同一文件系统内将它重命名为最终 `` 目录。 + +- 发布前的临时目录不是成功产物； +- 任一写入或校验失败时尝试清理本次专用临时目录； +- 不对已存在的最终目录使用替换语义； +- 本设计只保证“不发布已知不完整的运行目录”，不承诺跨平台断电耐久性。 + +## 9. 持久化事实与 JSON 契约 + +### 9.1 事实权威 + +- `manifest.json` 是整次运行身份、环境、流水线、文档索引和汇总的权威； +- 每份文档的 `result.json` 是该文档状态、哈希、实际修改、错误和残留候选的权威； +- `cleaned.md` 是成功文本内容的权威； +- `changes.diff` 是从输入和成功输出派生的人工评审视图，不是第二份修改记录。 + +清单中的计数和路径是从文档结果派生的索引，发布前必须校验与对应 `result.json` 和文件存在性一致。 + +### 9.2 JSON 通用表示 + +两类 JSON 都使用： + +- UTF-8，不写 BOM； +- 根对象的 `schema_version` 固定为整数 `1`； +- 两空格缩进； +- 不把非 ASCII 字符转成反斜杠加 `u` 的 Unicode 转义形式； +- 文件末有一个 `\n`； +- 数组顺序保留调用方文档顺序、组件顺序和核心修改顺序。 + +JSON 中的字段名和枚举值使用英文标识符。组件的中文修改理由和 `before` / `after` 保留原文。 + +### 9.3 `manifest.json` + +运行清单至少包含： + +```text +schema_version +run + run_id + run_date + utc_offset + status + started_at_utc + completed_at_utc + retention_until +tool + name + package_version + python_version + platform + git_commit # 无法取得时为 null + git_dirty # 无法取得时为 null +pipeline + components[] # 实际有序 ComponentInfo + component_id + version + parameters + applicability +documents[] + document_id + source_label + status + input_sha256 + current_sha256 + change_count + result_path + cleaned_path # 非 success 为 null + diff_path # 非 success 为 null +summary + document_count + success_count + failed_count + unstable_count + change_count +``` + +`run_date` 必须等于产物路径中的日期。`utc_offset` 使用 `+HH:MM` 或 `-HH:MM`，说明计算该日期时的本机 +UTC 偏移。`started_at_utc`、`completed_at_utc` 和 `retention_until` 使用带 `Z` 的 UTC ISO 8601 字符串。 +时间、平台和 Git 信息属于外层运行证据，不进入组件参数，也不影响核心结果的确定性。清单不读取或保存环境变量、 +用户名、主机名或其他可能包含秘密的全局环境信息。 + +`pipeline.components` 必须来自本次文档结果中的已验证元数据。同一次运行中各文档的组件元数据不一致时， +视为运行层错误，不发布最终目录。 + +### 9.4 `result.json` + +文档审计至少包含： + +```text +schema_version +document + document_id + source_label +status +input_sha256 +current_sha256 +changes[] + component_id + component_version + component_position + proposal_ref + component_position + snapshot_sha256 + proposal_index + edit_index + reason + span + start + end + location # 派生定位，不用于应用修改 + line # 1-based，相对 before_sha256 快照 + column # 1-based，相对 before_sha256 快照 + before + after + before_sha256 + after_sha256 +errors[] + component_id + component_version + component_position + stage + error_type + message +residual_proposals[] + component_id + component_version + component_position + proposal_ref + proposal + snapshot_sha256 + reason + edits[] + snapshot_sha256 + span + start + end + expected_text + replacement +output + cleaned_path # 非 success 为 null + diff_path # 非 success 为 null +``` + +`location` 由 reporter 基于对应 `before_sha256` 快照派生，只便于人查看。`span.start` / `span.end` +仍是唯一修改权威。当多个组件产生中间快照时，reporter 必须按组件批次从输入重放已记录修改，每次都校验前后哈希； +无法重放时不得发布产物。 + +`result.json` 不内嵌完整 `output_markdown` 或 `partial_markdown`。成功全文只在 `cleaned.md` 中保存； +失败和不稳定的部分全文不落盘。`changes`、`residual_proposals` 中的局部原文仍属于完整本地审计的一部分。 + +## 10. diff 契约 + +`changes.diff` 使用原始输入和最终成功输出生成 unified diff： + +- 旧文件标签固定为 `a/.md`； +- 新文件标签固定为 `b/.md`； +- 不把绝对路径和时间戳写入 diff 头； +- 上下文固定为 3 行； +- diff 自身使用 `\n` 作为报告换行，不表示清洗后 Markdown 被转换为 `\n`； +- 零修改的成功文档产生空 diff； +- diff 只用于人工评审，不用于重放或应用修改。 + +统一 diff 只展示输入到最终输出的总变化。如果需要查看某个组件的原因和中间快照，以 `result.json` +中的 `Change` 审计为准。 + +## 11. 隐私、权限和保留周期 + +### 11.1 数据级别 + +本设计生成的所有产物都按“本地敏感实验数据”处理，包括： + +- `cleaned.md` 中的完整文档； +- `changes.diff` 中的原文上下文； +- `result.json` 中的 `before`、`after` 和残留候选； +- `manifest.json` 中可能暴露数据集结构的文档名和来源标签。 + +这些产物： + +- 只能保存在本机 `artifacts/` 下； +- 不得提交、推送、发布、上传或复制到 Wiki； +- 终端默认只输出运行 ID、状态、计数和产物目录，不输出 `before`、`after` 或 diff 片段； +- 实验结束后不得自动拷贝到其他仓库或用户目录。 + +第一版创建运行目录时将目录权限设为只有当前用户可读、写和进入，普通产物文件只有当前用户可读写。 +如果平台不支持这些权限语义，实验必须明确报错，不静默降级为宽松权限。 + +### 11.2 保留周期 + +本地产物默认保留 30 个日历日。`manifest.json` 记录根据 `completed_at_utc` 计算的 `retention_until`。 + +第一版不实现自动删除，避免在没有人工确认时执行破坏性操作。到期产物只标记为待清理；删除前必须由用户确认 +具体运行目录。需要超过 30 天保留、跨机共享或长期归档时，必须先单独确认保存位置和数据边界。 + +## 12. 实现结构 + +批准后允许新增： + +```text +src/mdpolish/ +├── artifact_store.py # 日期路径、权限、写入校验和原子产物发布 +├── experiment.py # 显式文件输入、预检和批量运行 +└── reporting.py # JSON 表示、位置派生和 unified diff +scripts/ +└── run_clindb_arxiv_experiment.py # 仓库内已知 5 份论文的本地运行脚本 +tests/ +├── test_artifact_store.py +├── test_experiment.py +└── test_reporting.py +``` + +- 三个新模块不从顶层 `mdpolish.__init__` 导出，暂不承诺稳定公共 API； +- 本地脚本显式创建只包含 `ArxivSubmissionStampComponent` 的 `Pipeline`； +- 脚本启动时捕获一次本机日期、UTC 偏移和 UTC 开始时间，不在文档循环中重新计算日期； +- 脚本输入是 `data/md/` 中的 5 份缩写文件，输出为 `artifacts//runs//`； +- 脚本可以接收运行 ID，但不是通用文件清洗 CLI，不注册 `project.scripts`； +- 精确命令和实际运行步骤在实现并验证后进入 `guides/`，README 只链接当前权威，不在多处复制。 + +第一版继续只使用 Python 标准库。不修改包安装命令，不增加运行依赖。 + +## 13. 测试和验收 + +### 13.1 合成测试 + +自动测试只使用 `tmp_path` 和小型虚构 Markdown，不读取或复制真实论文。至少覆盖： + +- 空文档、中文、组合 Unicode、UTF-8 BOM、`\n`、`\r\n` 和无末尾换行； +- 无效 UTF-8、缺失文件、非普通文件、重复文档 ID、重复源路径和非法运行 ID 在预检失败； +- 非法日期、日期路径与 manifest 不一致、同日重复运行 ID 明确失败； +- 目标运行目录已存在时拒绝，不覆盖其中任何文件； +- `success` 生成三类文档产物，零修改时仍生成完整 `cleaned.md` 和空 diff； +- `failed` / `unstable` 只生成 `result.json`，不保存部分全文； +- 一份文档核心失败后继续处理其他文档，整体状态按第 7.2 节汇总； +- `result.json` 完整表示候选引用、修改理由、范围、位置、`before`、`after` 和哈希； +- 多组件修改可以重放中间快照并正确派生行列，哈希不一致时拒绝发布； +- unified diff 使用逻辑文档名，不包含绝对路径或时间； +- 输出 Markdown 重新读取后的哈希等于核心 `current_sha256`； +- 运行前后输入文件的字节和哈希不变； +- JSON 编码、schema 版本、顺序、文件末换行和清单计数符合契约； +- 产物目录及文件权限不宽于第 11 节的边界； +- 产物写入中途失败时不发布最终运行目录。 + +### 13.2 本地 5 份论文验收 + +实现并通过合成测试后，允许对 `data/md/` 中 5 份本地副本运行实验脚本，并在本地 +`artifacts//runs//` 保存完整产物。验收必须确认： + +1. 清单列出 `dmp`、`ejhf`、`jama`、`sim` 和 `springer` 共 5 份输入； +2. 流水线只有 `paper.arxiv_submission_stamp` `1.0.0`，参数为空； +3. 5 份文档都为 `success`； +4. `sim` 和 `springer` 各有 1 条删除，其余文档零修改，合计 2 条 `Change`； +5. 两条修改的组件、理由、原文、空替换、位置和前后哈希都可从 `result.json` 查看； +6. `sim` 和 `springer` 的 diff 只包含已批准的提交戳删除，Springer 合法参考文献保留； +7. 所有 `cleaned.md` 与对应成功哈希一致； +8. 5 份输入文件在运行前后逐字节不变； +9. 产物受 Git 忽略，提交前 `git status` 不列出任何实验产物。 + +真实实验产物只用于本地人工评审，不进入合成测试期望值、Wiki、Git 提交或终端输出。 + +## 14. 风险与代价 + +- **JSON 过早成为契约：** 第一版只服务本地实验，不承诺向后兼容；字段语义变化时必须增加 + `schema_version`，不静默改变旧文件语义。 +- **完整审计会复制真实片段：** `before`、`after` 和 diff 有意保留原文，换取可人工复核；代价是所有产物都必须 + 按敏感数据处理。 +- **批量中部分文档成功：** 这便于查看每份输入，但调用方必须检查整体和文档状态，不能因为目录里有部分 + `cleaned.md` 就宣称整批成功。 +- **行列派生需要重放修改：** 它能在不改核心模型的情况下便于人定位，但增加 reporter 的复杂度。重放过程必须 + 完全验证哈希，失败就拒绝发布。 +- **时间和环境使 manifest 不再字节级确定：** 这些是复现实验必需的外层证据，不影响相同文本和流水线的核心结果 + 确定性。 +- **严格权限会降低跨平台便携性：** 第一版是本机 Linux 实验能力，不借本设计声称 Windows 或共享盘已支持。 +- **固定 30 天只是本地实验保留约定：** 第一版不自动删除，所以仍需要用户定期确认和清理过期目录。 + +## 15. 批准后的实施边界 + +批准本设计后，只授权： + +1. 新增第 12 节列出的模块、本地脚本和测试； +2. 按第 6 至 11 节实现显式输入、核心调用、JSON、diff、原子发布和保留信息； +3. 对 `data/md/` 的 5 份本地论文副本运行只含 arXiv 组件的实验； +4. 将完整本地产物保存到 Git 忽略的 `artifacts//runs//`； +5. 实现后根据真实行为更新 README、对应 `explanation/` 和经验证的 `guides/`。 + +批准不授权： + +- 修改或覆盖任何输入文件； +- 对 `/home/lihaoze/gov_test_data` 或其他仓库外数据保存产物； +- 把实验产物加入 Git、Wiki、其他仓库、云存储或外部系统； +- 实现原地覆盖、公共 CLI、Profile、Inspector、parser、多轮执行或其他清洗组件； +- 提交、推送、创建 PR 或发布。 diff --git a/research-wiki/explanation/arxiv-submission-stamp.md b/research-wiki/explanation/arxiv-submission-stamp.md index 2435797..5d5703f 100644 --- a/research-wiki/explanation/arxiv-submission-stamp.md +++ b/research-wiki/explanation/arxiv-submission-stamp.md @@ -54,7 +54,7 @@ arXiv:<新版数字编号和版本> [] <日> <英文月份缩写> 合成测试覆盖严格匹配、反向引用、首行/中间/末行、三种行尾、多个命中、围栏中仍删除、审计字段、确定性和 第二次运行零修改。测试只使用短小的虚构字符串，不含真实论文片段。 -2026-08-22 又对本地 5 份 ClinDB-ReviewBench Markdown 做了只读、纯内存复核： +2026-08-22 先对本地 5 份 ClinDB-ReviewBench Markdown 做了只读、纯内存复核： | 复核项 | 结果 | | --- | --- | @@ -64,13 +64,20 @@ arXiv:<新版数字编号和版本> [] <日> <英文月份缩写> | 第二次运行 | 5 份合计 0 条修改 | | 源文件复读 | 5/5 与处理前内存内容一致，没有回写 | -本次没有保存清洗后 Markdown，没有把真实原文复制进测试、日志或仓库。安装、静态检查和完整测试命令仍只在根目录 +这次早期复核没有保存清洗后 Markdown，也没有把真实原文复制进测试、日志或仓库。它是 `0004` 当时验证边界的 +历史事实。 + +`0005` 批准本地产物机制后，同日又完成一次保存型实验：5 份文档全部为 `success`，仍然只修改 sim 和 springer +各一处；输出哈希全部匹配，输入运行前后字节不变，Springer 两处合法引用仍保留。产物位于 +`artifacts/2026-08-22/runs/clindb-arxiv-stamp-artifacts-v1/`，只在本机保留并受 Git 忽略。具体产物结构和验证结果见 +[`local-experiment-artifacts.md`](local-experiment-artifacts.md)。安装、静态检查和完整测试命令仍只在根目录 [`README.md`](../../README.md#当前可用检查) 维护。 ## 6. 剩余边界 这个组件只证明第一条严格删除规则能够在公共核心上闭环，不表示论文已经清洗完成。HTML 实体、Word 批注、手稿 -行号、断词、表格和参考文献间距仍未实现；文件输出、profile 和批处理也不存在。 +行号、断词、表格和参考文献间距仍未实现。当前只有固定输入和固定组件的本地实验输出；通用文件接口、profile、 +公共 CLI 和通用批处理仍不存在。 如果出现新的提交戳格式，默认行为是保留。必须先补充真实证据、反向样例和 design，再决定是否放宽模式，不能为了 提高命中数量直接修改正则表达式。 diff --git a/research-wiki/explanation/first-executable-core.md b/research-wiki/explanation/first-executable-core.md index ae06d5f..e95ca21 100644 --- a/research-wiki/explanation/first-executable-core.md +++ b/research-wiki/explanation/first-executable-core.md @@ -128,8 +128,10 @@ 同一候选修改中的多条记录共享候选引用，同一组件批次中的所有记录共享批次前后哈希。记录只描述已经发生的修改； 验证失败或最终复查中没有执行的候选不会冒充实际改动。 -这些内容当前只存在于内存返回值中。仓库没有 reporter、审计文件格式或日志持久化，调用方也不能默认把失败结果中 -的部分文本写回原文件。 +这些内容在核心中只存在于内存返回值中。核心外已经有一个获批的本地实验 reporter，可以校验修改链并把审计、 +成功 Markdown 和 diff 保存到私有产物目录；机制见 +[`local-experiment-artifacts.md`](local-experiment-artifacts.md)。这没有改变核心接口，也不允许把失败结果中的部分文本 +写成正式输出或写回原文件。 ## 8. 当前验证和剩余边界 @@ -146,7 +148,8 @@ - 除严格删除 arXiv 提交边栏戳外的其他论文、GovDoc 或 HTML 表格清洗组件； - 独立文档检查、人工建议或审核流程； - Markdown parser、AST 或共享业务中间表示； -- 文件读写、CLI、批处理、项目 profile 格式和生产集成； -- 审计结果的长期存储或脱敏输出协议。 +- 通用文件输入、公共 CLI、通用批处理、项目 profile 格式和生产集成； +- 审计结果的长期存储、自动清理或脱敏输出协议。 -这些边界中的任何一项要进入实现，都需要先用新的 design 明确语义、代价和验收方式。 +当前只有一个固定数据和组件组合的本地实验脚本，不构成上述公共能力。这些边界中的任何一项要进入实现，都需要先用 +新的 design 明确语义、代价和验收方式。 diff --git a/research-wiki/explanation/local-experiment-artifacts.md b/research-wiki/explanation/local-experiment-artifacts.md new file mode 100644 index 0000000..1236d1e --- /dev/null +++ b/research-wiki/explanation/local-experiment-artifacts.md @@ -0,0 +1,156 @@ +# 本地清洗实验如何保存 Markdown、审计和 diff + +## 1. 它解决什么问题 + +内存流水线可以安全地产生 `TransformResult`，但进程结束后，评审者仍需要打开清洗后的完整 Markdown、查看总 diff， +并追溯每条修改属于哪个组件、为什么修改、修改前后是什么。 + +当前本地实验层把这些结果保存到独立目录，同时继续保持三个边界： + +- 组件和 `Pipeline` 仍然不读写文件； +- 输入文件永远不被覆盖； +- 只有 `success` 文档才产生正式的清洗后 Markdown。 + +已经实现的范围来自已批准的 +[`0005-local-experiment-runner-and-artifacts.md`](../design/0005-local-experiment-runner-and-artifacts.md)。 +精确字段、校验和函数签名以 `src/mdpolish/` 中的代码与测试为准。 + +## 2. 三层怎样解耦 + +```text +experiment.py +├── pipeline.py 只负责内存清洗 +├── reporting.py 只负责 JSON、行列和 unified diff +└── artifact_store.py 只负责日期目录、权限和原子发布 +``` + +- `experiment.py` 严格读取调用方显式列出的 UTF-8 Markdown，逐份调用同一个 `Pipeline`； +- `reporting.py` 重放并校验 `Change` 的快照链，再生成机器可读审计和人可读 diff； +- `artifact_store.py` 不理解清洗规则，只把已经生成的字节写入私有临时目录，校验后一次性发布。 + +因此，新增组件不会改变文件层；调整目录布局不会影响清洗和报告；修改 JSON 或 diff 时也不需要碰流水线。 +ClinDB 的 5 份论文和 arXiv 组件组合只存在于仓库内实验脚本，通用模块没有硬编码论文名。 + +## 3. 输入怎样保持原样 + +实验层先完成整批预检： + +1. 验证日期、运行 ID、文档 ID 和来源标签； +2. 确认所有路径存在、是普通文件且没有重复； +3. 以二进制读取全部输入； +4. 使用严格 UTF-8 解码； +5. 比较原始字节 SHA-256 与核心 Markdown SHA-256。 + +读取过程不剔除 BOM，不规范化 Unicode，不转换 `\n`、`\r\n` 或文件末尾换行。全部文档运行结束后， +实验层再次读取每份输入并比较原始字节和哈希；任何变化都会阻止产物发布。 + +输入缺失、不是普通文件、不是有效 UTF-8 或清单冲突属于整批预检失败。这时不调用流水线，也不创建最终运行目录。 + +## 4. 状态怎样决定产物 + +每份文档独立运行，某一份发生核心错误不会阻止其他已经预检的文档继续产生结果。 + +| 文档状态 | `result.json` | `cleaned.md` | `changes.diff` | +| --- | --- | --- | --- | +| `success` | 有 | 有 | 有；零修改时为空文件 | +| `failed` | 有 | 无 | 无 | +| `unstable` | 有 | 无 | 无 | + +`failed` 和 `unstable` 的审计仍保留已经实际发生的 `Change`、错误或残留候选，但不持久化 +`partial_markdown`，避免半成品看起来像正式结果。 + +整批状态按 `failed`、`unstable`、`success` 的优先级汇总。即使其他文档有成功产物，只要一份失败， +`manifest.json` 就会把整批标为 `failed`。 + +## 5. 产物怎样组织 + +```text +artifacts/ +└── / + └── runs/ + └── / + ├── manifest.json + └── documents/ + └── / + ├── result.json + ├── cleaned.md + └── changes.diff +``` + +日期取实验启动时本机时区中的日历日期。运行 ID 由调用方显式提供；同一日期下已经存在同名目录时拒绝覆盖。 + +`manifest.json` 是整次运行的索引，记录： + +- 开始、完成和到期时间； +- 本机日期及 UTC 偏移； +- mdpolish、Python、平台和 Git 状态； +- 实际组件顺序、版本和参数； +- 每份输入的来源标签、前后哈希、状态、修改数量和产物相对路径； +- 整批成功、失败、不稳定和修改数量。 + +每份 `result.json` 保存实际修改、错误与残留候选。每条修改包含组件、候选引用、理由、Python 字符范围、 +1-based 行列、`before`、`after` 和批次前后哈希。完整成功文本只存在于 `cleaned.md`。 + +`changes.diff` 是原始输入到最终成功输出的 unified diff，只用于人工查看。它不包含绝对路径或时间戳， +也不是修改重放的权威；机器审计仍以 `result.json` 为准。 + +## 6. 为什么 reporter 要重放修改 + +第二个组件看到的是第一个组件修改后的快照，因此后续 `Change.span` 不一定对应最初输入。为了生成准确行列， +reporter 从输入开始，按组件批次重放修改： + +1. 当前文本哈希必须等于该批次 `before_sha256`； +2. 每条范围内的原文必须等于 `before`； +3. 同一批次按核心相同的从后向前顺序应用； +4. 结果哈希必须等于 `after_sha256`； +5. 全部批次完成后必须等于 `TransformResult.current_sha256`。 + +任何一步不一致都说明内存结果、reporter 或调用方式违反契约，整次运行不会发布最终目录。行列只是方便人查看的 +派生信息，修改权威仍是快照绑定的 Python 字符范围。 + +## 7. 文件怎样安全发布 + +artifact store 先在同一日期的 `runs/` 下建立本次专用临时目录。所有文件写入后都会重新读取校验， +成功 Markdown 还要再次核对输出 SHA-256，manifest 的身份、状态、计数和路径也必须与各文档审计及实际文件一致。 +只有全部文件、清单和权限都通过，临时目录才会在 `runs/` 目录协作锁内重新检查目标，并原子重命名为最终运行 ID。 + +当前 Linux 本地实现使用： + +- 目录权限 `0700`； +- 文件权限 `0600`； +- 已存在的目标目录拒绝覆盖； +- 写入中途失败时不发布最终目录。 + +这保证不会发布已知不完整的结果，但不承诺跨平台断电耐久性或网络文件系统语义。 + +## 8. 隐私和保留边界 + +`cleaned.md`、diff 和 JSON 审计都可能包含真实原文，因此整个 `artifacts/` 都是本地敏感数据： + +- 受 Git 忽略； +- 不进入 Wiki、提交、推送或外部系统； +- 终端只显示状态、计数和目录； +- 默认保留 30 个日历日； +- manifest 记录 `retention_until`； +- 第一版不自动删除，到期后仍需用户确认具体目录再清理。 + +当前只批准对 `data/md/` 中 5 份论文副本保存产物。仓库外 GovDoc 和其他真实数据没有因此获得输出授权。 + +## 9. 已完成的真实验证 + +2026-08-22 使用 `paper.arxiv_submission_stamp` `1.0.0` 对 5 份本地论文副本完成一次保存型实验： + +| 项目 | 结果 | +| --- | --- | +| 运行 ID | `clindb-arxiv-stamp-artifacts-v1` | +| 输出位置 | `artifacts/2026-08-22/runs/clindb-arxiv-stamp-artifacts-v1/` | +| 文档状态 | 5/5 `success` | +| 实际修改 | `sim` 1 条、`springer` 1 条，其余 0 条 | +| 修改位置 | `sim` 1:1、`springer` 18:1 | +| 合法反向样例 | Springer 两处 `arXiv preprint arXiv:` 均保留 | +| 输出校验 | 5/5 `cleaned.md` 与 `current_sha256` 一致 | +| 输入只读 | 5/5 运行前后字节和哈希不变 | +| 权限 | 全部运行目录 `0700`，产物文件 `0600` | + +实际运行方法见 +[`run-local-clindb-arxiv-experiment.md`](../guides/run-local-clindb-arxiv-experiment.md)。 diff --git a/research-wiki/guides/run-local-clindb-arxiv-experiment.md b/research-wiki/guides/run-local-clindb-arxiv-experiment.md new file mode 100644 index 0000000..dc443d6 --- /dev/null +++ b/research-wiki/guides/run-local-clindb-arxiv-experiment.md @@ -0,0 +1,129 @@ +# 运行本地 ClinDB arXiv 清洗实验 + +## 1. 适用范围 + +本指南只运行仓库内已经批准的本地实验脚本： + +- 输入：`data/md/` 中的 `dmp.md`、`ejhf.md`、`jama.md`、`sim.md` 和 `springer.md`； +- 流水线：只包含 `paper.arxiv_submission_stamp` `1.0.0`； +- 输出：`artifacts//runs//`； +- 输入只读，不覆盖原文件； +- 不处理 `/home/lihaoze/gov_test_data`。 + +本指南于 2026-08-22 在 Python 3.13.11 环境实际验证。 + +## 2. 前置条件 + +在仓库根目录执行，并确认隔离环境和基础检查可用： + +```bash +.venv/bin/python --version +.venv/bin/ruff check . +.venv/bin/mypy src tests scripts/run_clindb_arxiv_experiment.py +.venv/bin/pytest +``` + +确认 5 份本地输入存在： + +```bash +find data/md -maxdepth 1 -type f -name '*.md' -printf '%f\n' | sort +``` + +预期看到： + +```text +dmp.md +ejhf.md +jama.md +sim.md +springer.md +``` + +## 3. 运行实验 + +为本次实验人工选择一个小写运行 ID。同一天已经使用过的 ID 不能覆盖；需要重跑时换一个新 ID。 + +```bash +.venv/bin/python scripts/run_clindb_arxiv_experiment.py \ + --run-id clindb-arxiv-stamp-review +``` + +成功时终端只显示运行 ID、状态、文档数、修改数和产物目录，例如： + +```text +run_id=clindb-arxiv-stamp-review +status=success +documents=5 +changes=2 +artifacts=/.../mdpolish/artifacts//runs/clindb-arxiv-stamp-review +``` + +终端不会打印论文原文或 diff。 + +## 4. 查看结果 + +进入终端输出的运行目录。目录结构为： + +```text +manifest.json +documents/ +├── dmp/ +│ ├── result.json +│ ├── cleaned.md +│ └── changes.diff +├── ejhf/ +├── jama/ +├── sim/ +└── springer/ +``` + +先看 `manifest.json` 的整批状态和汇总，再查看各文档： + +- `cleaned.md`：成功清洗后的完整 Markdown； +- `changes.diff`：输入到成功输出的人工对比； +- `result.json`：组件、理由、位置、`before`、`after` 和哈希等机器审计。 + +`dmp`、`ejhf` 和 `jama` 当前应为零修改，diff 是空文件；`sim` 和 `springer` 当前各有一条删除。 + +## 5. 判断成功 + +本轮验收口径是： + +- manifest 整体状态为 `success`； +- 5 份文档全部为 `success`； +- 合计 2 条修改； +- `sim` 修改位置为 1:1； +- `springer` 修改位置为 18:1； +- Springer 两条合法 `arXiv preprint arXiv:` 参考文献保留； +- 每份 `cleaned.md` 的 SHA-256 等于对应 `result.json.current_sha256`； +- 输入文件运行前后不变。 + +只看到运行目录存在不等于成功，必须先检查 manifest 和文档状态。 + +## 6. 常见失败 + +### 运行目录已经存在 + +脚本拒绝覆盖同一日期下的同名运行目录。选择新的 `--run-id`，不要删除或覆盖旧目录来绕过检查。 + +### 输入缺失或不是 UTF-8 + +整批预检会失败，不运行任何组件，也不发布最终目录。先确认 `data/md/` 中 5 份文件存在且未被修改。 + +### 状态为 `failed` 或 `unstable` + +对应文档只会生成 `result.json`，不会生成 `cleaned.md` 或 diff。查看错误或残留候选，不要把其他文档的部分成功 +当成整批成功。 + +### Snap 版本的 `jq` 报权限错误 + +产物目录权限是 `0700`。某些 Snap 沙箱工具不能进入私有目录，即使当前用户拥有权限。可直接用编辑器查看 JSON， +或使用当前虚拟环境中的 Python 读取；不要为了兼容受限工具放宽产物权限。 + +## 7. 数据边界 + +产物包含完整论文和原文片段，只能保存在本机 Git 忽略的 `artifacts/`。不得执行 `git add -f`，不得复制到 Wiki、 +其他仓库、云存储或外部系统。 + +`manifest.json` 中的 `retention_until` 是默认 30 天到期时间。第一版不会自动删除；到期后如需清理，必须先确认 +具体运行目录。 diff --git a/scripts/run_clindb_arxiv_experiment.py b/scripts/run_clindb_arxiv_experiment.py new file mode 100644 index 0000000..7e00626 --- /dev/null +++ b/scripts/run_clindb_arxiv_experiment.py @@ -0,0 +1,58 @@ +"""Run the approved arXiv component over the five local ClinDB paper copies.""" + +from __future__ import annotations + +import argparse +import sys +from datetime import datetime +from pathlib import Path + +from mdpolish.components import ArxivSubmissionStampComponent +from mdpolish.experiment import InputDocument, collect_tool_metadata, run_experiment +from mdpolish.models import RunStatus +from mdpolish.pipeline import Pipeline + +_DOCUMENT_IDS = ("dmp", "ejhf", "jama", "sim", "springer") + + +def _parse_args() -> argparse.Namespace: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--run-id", required=True, help="safe identifier within today's artifact directory") + return parser.parse_args() + + +def main() -> int: + args = _parse_args() + repository_root = Path(__file__).resolve().parents[1] + documents = tuple( + InputDocument( + document_id=document_id, + source_path=repository_root / "data" / "md" / f"{document_id}.md", + source_label=f"data/md/{document_id}.md", + ) + for document_id in _DOCUMENT_IDS + ) + started_at = datetime.now().astimezone() + try: + result = run_experiment( + pipeline=Pipeline([ArxivSubmissionStampComponent()]), + documents=documents, + run_id=args.run_id, + artifacts_root=repository_root / "artifacts", + started_at=started_at, + tool=collect_tool_metadata(repository_root), + ) + except Exception as error: + print(f"experiment failed: {type(error).__name__}: {error}", file=sys.stderr) + return 1 + + print(f"run_id={args.run_id}") + print(f"status={result.status.value}") + print(f"documents={result.document_count}") + print(f"changes={result.change_count}") + print(f"artifacts={result.run_directory}") + return 0 if result.status is RunStatus.SUCCESS else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/mdpolish/artifact_store.py b/src/mdpolish/artifact_store.py new file mode 100644 index 0000000..70fe8e4 --- /dev/null +++ b/src/mdpolish/artifact_store.py @@ -0,0 +1,290 @@ +"""Private local storage for complete experiment artifact directories.""" + +from __future__ import annotations + +import fcntl +import json +import os +import re +import shutil +import stat +import tempfile +from dataclasses import dataclass +from datetime import date +from hashlib import sha256 +from pathlib import Path +from typing import cast + +_SAFE_ID_PATTERN = re.compile(r"^[a-z0-9][a-z0-9._-]*$") +_DIRECTORY_MODE = 0o700 +_FILE_MODE = 0o600 + + +class ArtifactStoreError(RuntimeError): + """A local artifact directory cannot be published safely.""" + + +@dataclass(frozen=True, slots=True) +class StoredDocument: + """Complete encoded artifacts for one document.""" + + document_id: str + result_json: bytes + cleaned_markdown: bytes | None + diff: bytes | None + output_sha256: str | None + + +def _validate_identifier(value: str, field_name: str) -> None: + if not isinstance(value, str) or _SAFE_ID_PATTERN.fullmatch(value) is None or ".." in value: + raise ArtifactStoreError(f"{field_name} must be a safe lowercase identifier") + + +def _validate_run_date(run_date: str) -> None: + if not isinstance(run_date, str): + raise ArtifactStoreError("run_date must use YYYY-MM-DD") + try: + parsed = date.fromisoformat(run_date) + except ValueError as error: + raise ArtifactStoreError("run_date must use YYYY-MM-DD") from error + if parsed.isoformat() != run_date: + raise ArtifactStoreError("run_date must use YYYY-MM-DD") + + +def _require_private_directory(path: Path) -> None: + if path.is_symlink(): + raise ArtifactStoreError(f"artifact directory cannot be a symlink: {path}") + if not path.exists(): + path.mkdir(mode=_DIRECTORY_MODE, parents=False) + path.chmod(_DIRECTORY_MODE) + if not path.is_dir(): + raise ArtifactStoreError(f"artifact path is not a directory: {path}") + mode = stat.S_IMODE(path.stat().st_mode) + if mode != _DIRECTORY_MODE: + raise ArtifactStoreError(f"artifact directory must have mode 0700: {path}") + + +def _create_private_directory(path: Path) -> None: + path.mkdir(mode=_DIRECTORY_MODE) + path.chmod(_DIRECTORY_MODE) + _require_private_directory(path) + + +def _prepare_runs_directory(artifacts_root: Path, run_date: str) -> Path: + if not artifacts_root.exists(): + artifacts_root.mkdir(mode=_DIRECTORY_MODE, parents=True) + artifacts_root.chmod(_DIRECTORY_MODE) + _require_private_directory(artifacts_root) + + date_directory = artifacts_root / run_date + _require_private_directory(date_directory) + runs_directory = date_directory / "runs" + _require_private_directory(runs_directory) + return runs_directory + + +def _write_private_file(path: Path, content: bytes) -> None: + if not isinstance(content, bytes): + raise ArtifactStoreError("artifact content must be bytes") + with path.open("xb") as artifact_file: + artifact_file.write(content) + artifact_file.flush() + os.fsync(artifact_file.fileno()) + path.chmod(_FILE_MODE) + if stat.S_IMODE(path.stat().st_mode) != _FILE_MODE: + raise ArtifactStoreError(f"artifact file must have mode 0600: {path}") + if path.read_bytes() != content: + raise ArtifactStoreError(f"artifact readback verification failed: {path}") + + +def _validate_document(document: StoredDocument) -> None: + _validate_identifier(document.document_id, "document_id") + has_output = document.cleaned_markdown is not None + if has_output != (document.diff is not None) or has_output != (document.output_sha256 is not None): + raise ArtifactStoreError("cleaned Markdown, diff, and output hash must be present together") + if document.cleaned_markdown is not None: + digest = sha256(document.cleaned_markdown).hexdigest() + if digest != document.output_sha256: + raise ArtifactStoreError("cleaned Markdown does not match its output hash") + + +def _json_object(content: bytes, label: str) -> dict[str, object]: + if not isinstance(content, bytes) or not content.endswith(b"\n") or content.startswith(b"\xef\xbb\xbf"): + raise ArtifactStoreError(f"{label} must be BOM-free UTF-8 JSON ending in a newline") + try: + decoded = json.loads(content) + except (UnicodeDecodeError, json.JSONDecodeError) as error: + raise ArtifactStoreError(f"{label} must contain valid UTF-8 JSON") from error + if not isinstance(decoded, dict) or not all(isinstance(key, str) for key in decoded): + raise ArtifactStoreError(f"{label} must contain a JSON object") + return cast(dict[str, object], decoded) + + +def _nested_object(payload: dict[str, object], field_name: str, label: str) -> dict[str, object]: + value = payload.get(field_name) + if not isinstance(value, dict) or not all(isinstance(key, str) for key in value): + raise ArtifactStoreError(f"{label}.{field_name} must be a JSON object") + return cast(dict[str, object], value) + + +def _validate_bundle( + *, + run_date: str, + run_id: str, + manifest_json: bytes, + documents: tuple[StoredDocument, ...], +) -> None: + manifest = _json_object(manifest_json, "manifest") + if manifest.get("schema_version") != 1: + raise ArtifactStoreError("manifest schema_version must be 1") + run = _nested_object(manifest, "run", "manifest") + if run.get("run_date") != run_date or run.get("run_id") != run_id: + raise ArtifactStoreError("manifest run identity does not match the target path") + + document_indexes = manifest.get("documents") + if not isinstance(document_indexes, list) or len(document_indexes) != len(documents): + raise ArtifactStoreError("manifest documents do not match the stored documents") + + success_count = 0 + failed_count = 0 + unstable_count = 0 + change_count = 0 + for position, document in enumerate(documents): + raw_index = document_indexes[position] + if not isinstance(raw_index, dict) or not all(isinstance(key, str) for key in raw_index): + raise ArtifactStoreError("manifest document indexes must be JSON objects") + index = cast(dict[str, object], raw_index) + report = _json_object(document.result_json, f"result for {document.document_id}") + report_document = _nested_object(report, "document", f"result for {document.document_id}") + report_output = _nested_object(report, "output", f"result for {document.document_id}") + status = report.get("status") + changes = report.get("changes") + if ( + report.get("schema_version") != 1 + or report_document.get("document_id") != document.document_id + or status not in {"success", "failed", "unstable"} + or not isinstance(changes, list) + ): + raise ArtifactStoreError("a document result does not match its stored identity") + + has_output = document.cleaned_markdown is not None + expected_cleaned_name = "cleaned.md" if has_output else None + expected_diff_name = "changes.diff" if has_output else None + base = f"documents/{document.document_id}" + expected_cleaned_path = f"{base}/cleaned.md" if has_output else None + expected_diff_path = f"{base}/changes.diff" if has_output else None + if has_output != (status == "success"): + raise ArtifactStoreError("document output presence does not match its result status") + if report_output != { + "cleaned_path": expected_cleaned_name, + "diff_path": expected_diff_name, + }: + raise ArtifactStoreError("document result paths do not match its stored files") + if document.output_sha256 != (report.get("current_sha256") if has_output else None): + raise ArtifactStoreError("document output hash does not match its result") + + expected_index = { + "document_id": document.document_id, + "source_label": report_document.get("source_label"), + "status": status, + "input_sha256": report.get("input_sha256"), + "current_sha256": report.get("current_sha256"), + "change_count": len(changes), + "result_path": f"{base}/result.json", + "cleaned_path": expected_cleaned_path, + "diff_path": expected_diff_path, + } + if index != expected_index: + raise ArtifactStoreError("a manifest document index does not match its result and files") + + change_count += len(changes) + if status == "success": + success_count += 1 + elif status == "failed": + failed_count += 1 + else: + unstable_count += 1 + + overall_status = "failed" if failed_count else "unstable" if unstable_count else "success" + expected_summary = { + "document_count": len(documents), + "success_count": success_count, + "failed_count": failed_count, + "unstable_count": unstable_count, + "change_count": change_count, + } + if run.get("status") != overall_status or manifest.get("summary") != expected_summary: + raise ArtifactStoreError("manifest status or summary does not match its documents") + + +def _rename_no_replace(source: Path, target: Path) -> None: + directory_fd = os.open(source.parent, os.O_RDONLY | os.O_DIRECTORY) + try: + fcntl.flock(directory_fd, fcntl.LOCK_EX) + if target.exists() or target.is_symlink(): + raise ArtifactStoreError("the target run directory already exists") + os.rename(source, target) + finally: + fcntl.flock(directory_fd, fcntl.LOCK_UN) + os.close(directory_fd) + + +def publish_run( + *, + artifacts_root: Path, + run_date: str, + run_id: str, + manifest_json: bytes, + documents: tuple[StoredDocument, ...], +) -> Path: + """Publish one complete run directory without replacing an existing run.""" + if not isinstance(artifacts_root, Path): + raise TypeError("artifacts_root must be a Path") + _validate_run_date(run_date) + _validate_identifier(run_id, "run_id") + if not documents: + raise ArtifactStoreError("an artifact run requires at least one document") + + seen_ids: set[str] = set() + for document in documents: + if not isinstance(document, StoredDocument): + raise ArtifactStoreError("documents must contain only StoredDocument values") + _validate_document(document) + if document.document_id in seen_ids: + raise ArtifactStoreError("document_id values must be unique") + seen_ids.add(document.document_id) + _validate_bundle( + run_date=run_date, + run_id=run_id, + manifest_json=manifest_json, + documents=documents, + ) + + runs_directory = _prepare_runs_directory(artifacts_root, run_date) + final_directory = runs_directory / run_id + if final_directory.exists() or final_directory.is_symlink(): + raise ArtifactStoreError("the target run directory already exists") + + temporary_directory = Path(tempfile.mkdtemp(prefix=f".{run_id}.", dir=runs_directory)) + temporary_directory.chmod(_DIRECTORY_MODE) + try: + documents_directory = temporary_directory / "documents" + _create_private_directory(documents_directory) + for document in documents: + document_directory = documents_directory / document.document_id + _create_private_directory(document_directory) + _write_private_file(document_directory / "result.json", document.result_json) + if document.cleaned_markdown is not None and document.diff is not None: + _write_private_file(document_directory / "cleaned.md", document.cleaned_markdown) + _write_private_file(document_directory / "changes.diff", document.diff) + + _write_private_file(temporary_directory / "manifest.json", manifest_json) + _rename_no_replace(temporary_directory, final_directory) + except Exception: + if temporary_directory.exists(): + shutil.rmtree(temporary_directory) + raise + + if stat.S_IMODE(final_directory.stat().st_mode) != _DIRECTORY_MODE: + raise ArtifactStoreError("published run directory does not have mode 0700") + return final_directory diff --git a/src/mdpolish/experiment.py b/src/mdpolish/experiment.py new file mode 100644 index 0000000..e2f6b5e --- /dev/null +++ b/src/mdpolish/experiment.py @@ -0,0 +1,380 @@ +"""Explicit local file experiments built around the in-memory pipeline.""" + +from __future__ import annotations + +import platform as platform_module +import re +import subprocess +import sys +from dataclasses import dataclass +from datetime import UTC, datetime, timedelta +from hashlib import sha256 +from importlib.metadata import PackageNotFoundError, version +from pathlib import Path + +from mdpolish.artifact_store import StoredDocument, publish_run +from mdpolish.models import ComponentInfo, RunStatus, markdown_sha256 +from mdpolish.pipeline import Pipeline +from mdpolish.reporting import ( + DocumentReport, + JsonObject, + JsonValue, + build_document_report, + component_info_json, + encode_json, +) + +_SAFE_ID_PATTERN = re.compile(r"^[a-z0-9][a-z0-9._-]*$") +_UTC_OFFSET_PATTERN = re.compile(r"^[+-][0-9]{2}:[0-9]{2}$") + + +class ExperimentError(RuntimeError): + """A local experiment cannot be completed without violating its contract.""" + + +@dataclass(frozen=True, slots=True) +class InputDocument: + """One explicitly selected local Markdown input.""" + + document_id: str + source_path: Path + source_label: str + + +@dataclass(frozen=True, slots=True) +class ToolMetadata: + """Source-safe tool and environment facts for a run manifest.""" + + name: str + package_version: str + python_version: str + platform: str + git_commit: str | None + git_dirty: bool | None + + +@dataclass(frozen=True, slots=True) +class ExperimentResult: + """Non-sensitive terminal summary of one published local run.""" + + run_directory: Path + status: RunStatus + document_count: int + success_count: int + failed_count: int + unstable_count: int + change_count: int + + +@dataclass(frozen=True, slots=True) +class _PreparedDocument: + input: InputDocument + resolved_path: Path + source_bytes: bytes + markdown: str + input_sha256: str + + +def _safe_identifier(value: str, field_name: str) -> None: + if not isinstance(value, str) or _SAFE_ID_PATTERN.fullmatch(value) is None or ".." in value: + raise ExperimentError(f"{field_name} must be a safe lowercase identifier") + + +def _run_git(repository_root: Path, *arguments: str) -> str | None: + try: + completed = subprocess.run( + ("git", *arguments), + cwd=repository_root, + check=False, + capture_output=True, + text=True, + timeout=10, + ) + except (OSError, subprocess.SubprocessError): + return None + if completed.returncode != 0: + return None + return completed.stdout.strip() + + +def collect_tool_metadata(repository_root: Path) -> ToolMetadata: + """Collect only the reproducibility facts approved for the manifest.""" + if not isinstance(repository_root, Path): + raise TypeError("repository_root must be a Path") + try: + package_version = version("mdpolish") + except PackageNotFoundError: + package_version = "uninstalled" + + git_commit = _run_git(repository_root, "rev-parse", "HEAD") + git_status = _run_git(repository_root, "status", "--porcelain") + return ToolMetadata( + name="mdpolish", + package_version=package_version, + python_version=platform_module.python_version(), + platform=f"{sys.platform}-{platform_module.machine()}", + git_commit=git_commit, + git_dirty=None if git_status is None else bool(git_status), + ) + + +def _utc_text(value: datetime) -> str: + if value.tzinfo is None or value.utcoffset() is None: + raise ExperimentError("experiment timestamps must be timezone-aware") + utc_value = value.astimezone(UTC) + return utc_value.isoformat(timespec="seconds").replace("+00:00", "Z") + + +def _utc_offset_text(value: datetime) -> str: + offset = value.utcoffset() + if offset is None: + raise ExperimentError("experiment timestamps must be timezone-aware") + total_minutes = int(offset.total_seconds() // 60) + sign = "+" if total_minutes >= 0 else "-" + absolute_minutes = abs(total_minutes) + hours, minutes = divmod(absolute_minutes, 60) + text = f"{sign}{hours:02d}:{minutes:02d}" + if _UTC_OFFSET_PATTERN.fullmatch(text) is None or hours > 23 or minutes > 59: + raise ExperimentError("the experiment UTC offset cannot be represented safely") + return text + + +def _prepare_documents( + documents: tuple[InputDocument, ...], + *, + final_directory: Path, +) -> tuple[_PreparedDocument, ...]: + if not isinstance(documents, tuple) or not documents: + raise ExperimentError("documents must be a non-empty tuple") + + seen_ids: set[str] = set() + seen_paths: set[Path] = set() + prepared: list[_PreparedDocument] = [] + final_resolved = final_directory.resolve(strict=False) + for document in documents: + if not isinstance(document, InputDocument): + raise ExperimentError("documents must contain only InputDocument values") + _safe_identifier(document.document_id, "document_id") + if document.document_id in seen_ids: + raise ExperimentError("document_id values must be unique") + seen_ids.add(document.document_id) + if not isinstance(document.source_path, Path): + raise ExperimentError("source_path must be a Path") + if ( + not isinstance(document.source_label, str) + or not document.source_label.strip() + or "\0" in document.source_label + ): + raise ExperimentError("source_label must be a non-empty display string") + + try: + resolved_path = document.source_path.resolve(strict=True) + except OSError as error: + raise ExperimentError(f"input file cannot be resolved: {document.source_label}") from error + if not resolved_path.is_file(): + raise ExperimentError(f"input path is not a regular file: {document.source_label}") + if resolved_path in seen_paths: + raise ExperimentError("source_path values must be unique") + seen_paths.add(resolved_path) + if ( + resolved_path == final_resolved + or final_resolved in resolved_path.parents + or resolved_path in final_resolved.parents + ): + raise ExperimentError("an input file and target run directory cannot overlap") + + try: + source_bytes = resolved_path.read_bytes() + markdown = source_bytes.decode("utf-8", errors="strict") + except (OSError, UnicodeDecodeError) as error: + raise ExperimentError(f"input file must be readable strict UTF-8: {document.source_label}") from error + input_sha256 = sha256(source_bytes).hexdigest() + if markdown_sha256(markdown) != input_sha256: + raise ExperimentError("decoded Markdown does not reproduce the input byte hash") + prepared.append( + _PreparedDocument( + input=document, + resolved_path=resolved_path, + source_bytes=source_bytes, + markdown=markdown, + input_sha256=input_sha256, + ) + ) + return tuple(prepared) + + +def _overall_status(reports: tuple[DocumentReport, ...]) -> RunStatus: + if any(report.status is RunStatus.FAILED for report in reports): + return RunStatus.FAILED + if any(report.status is RunStatus.UNSTABLE for report in reports): + return RunStatus.UNSTABLE + return RunStatus.SUCCESS + + +def _document_index(report: DocumentReport) -> JsonObject: + base = f"documents/{report.document_id}" + return { + "document_id": report.document_id, + "source_label": report.source_label, + "status": report.status.value, + "input_sha256": report.input_sha256, + "current_sha256": report.current_sha256, + "change_count": report.change_count, + "result_path": f"{base}/result.json", + "cleaned_path": f"{base}/cleaned.md" if report.status is RunStatus.SUCCESS else None, + "diff_path": f"{base}/changes.diff" if report.status is RunStatus.SUCCESS else None, + } + + +def _manifest_payload( + *, + run_id: str, + run_date: str, + utc_offset: str, + started_at: datetime, + completed_at: datetime, + tool: ToolMetadata, + components: tuple[ComponentInfo, ...], + reports: tuple[DocumentReport, ...], +) -> JsonObject: + status = _overall_status(reports) + component_values: list[JsonValue] = [component_info_json(component) for component in components] + document_values: list[JsonValue] = [_document_index(report) for report in reports] + success_count = sum(report.status is RunStatus.SUCCESS for report in reports) + failed_count = sum(report.status is RunStatus.FAILED for report in reports) + unstable_count = sum(report.status is RunStatus.UNSTABLE for report in reports) + change_count = sum(report.change_count for report in reports) + retention_until = completed_at.astimezone(UTC) + timedelta(days=30) + return { + "schema_version": 1, + "run": { + "run_id": run_id, + "run_date": run_date, + "utc_offset": utc_offset, + "status": status.value, + "started_at_utc": _utc_text(started_at), + "completed_at_utc": _utc_text(completed_at), + "retention_until": _utc_text(retention_until), + }, + "tool": { + "name": tool.name, + "package_version": tool.package_version, + "python_version": tool.python_version, + "platform": tool.platform, + "git_commit": tool.git_commit, + "git_dirty": tool.git_dirty, + }, + "pipeline": {"components": component_values}, + "documents": document_values, + "summary": { + "document_count": len(reports), + "success_count": success_count, + "failed_count": failed_count, + "unstable_count": unstable_count, + "change_count": change_count, + }, + } + + +def run_experiment( + *, + pipeline: Pipeline, + documents: tuple[InputDocument, ...], + run_id: str, + artifacts_root: Path, + started_at: datetime, + tool: ToolMetadata, + completed_at: datetime | None = None, +) -> ExperimentResult: + """Run explicit inputs and publish one complete private artifact directory.""" + if not isinstance(pipeline, Pipeline): + raise TypeError("pipeline must be a Pipeline") + _safe_identifier(run_id, "run_id") + if not isinstance(artifacts_root, Path): + raise TypeError("artifacts_root must be a Path") + if not isinstance(tool, ToolMetadata): + raise TypeError("tool must be ToolMetadata") + if started_at.tzinfo is None or started_at.utcoffset() is None: + raise ExperimentError("started_at must be timezone-aware") + + run_date = started_at.date().isoformat() + utc_offset = _utc_offset_text(started_at) + final_directory = artifacts_root / run_date / "runs" / run_id + if final_directory.exists() or final_directory.is_symlink(): + raise ExperimentError("the target run directory already exists") + prepared = _prepare_documents(documents, final_directory=final_directory) + + reports: list[DocumentReport] = [] + expected_components: tuple[ComponentInfo, ...] | None = None + for document in prepared: + result = pipeline.transform(document.markdown) + if expected_components is None: + expected_components = result.components + elif result.components != expected_components: + raise ExperimentError("pipeline component metadata changed between documents") + reports.append( + build_document_report( + document_id=document.input.document_id, + source_label=document.input.source_label, + input_markdown=document.markdown, + result=result, + ) + ) + + for document in prepared: + try: + current_bytes = document.resolved_path.read_bytes() + except OSError as error: + raise ExperimentError( + f"input file cannot be verified after the run: {document.input.source_label}" + ) from error + if current_bytes != document.source_bytes or sha256(current_bytes).hexdigest() != document.input_sha256: + raise ExperimentError(f"input file changed during the run: {document.input.source_label}") + + report_tuple = tuple(reports) + if expected_components is None: + raise ExperimentError("a non-empty experiment produced no component metadata") + completion = datetime.now(UTC) if completed_at is None else completed_at + if completion.tzinfo is None or completion.utcoffset() is None: + raise ExperimentError("completed_at must be timezone-aware") + if completion.astimezone(UTC) < started_at.astimezone(UTC): + raise ExperimentError("completed_at cannot be earlier than started_at") + + manifest = _manifest_payload( + run_id=run_id, + run_date=run_date, + utc_offset=utc_offset, + started_at=started_at, + completed_at=completion, + tool=tool, + components=expected_components, + reports=report_tuple, + ) + stored_documents = tuple( + StoredDocument( + document_id=report.document_id, + result_json=report.result_json, + cleaned_markdown=report.cleaned_markdown, + diff=report.diff, + output_sha256=report.current_sha256 if report.status is RunStatus.SUCCESS else None, + ) + for report in report_tuple + ) + run_directory = publish_run( + artifacts_root=artifacts_root, + run_date=run_date, + run_id=run_id, + manifest_json=encode_json(manifest), + documents=stored_documents, + ) + + status = _overall_status(report_tuple) + return ExperimentResult( + run_directory=run_directory, + status=status, + document_count=len(report_tuple), + success_count=sum(report.status is RunStatus.SUCCESS for report in report_tuple), + failed_count=sum(report.status is RunStatus.FAILED for report in report_tuple), + unstable_count=sum(report.status is RunStatus.UNSTABLE for report in report_tuple), + change_count=sum(report.change_count for report in report_tuple), + ) diff --git a/src/mdpolish/reporting.py b/src/mdpolish/reporting.py new file mode 100644 index 0000000..d5d0b6e --- /dev/null +++ b/src/mdpolish/reporting.py @@ -0,0 +1,315 @@ +"""Pure JSON and diff representations for local experiment results.""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from difflib import unified_diff +from typing import TypeAlias + +from mdpolish.models import ( + Change, + ComponentInfo, + ParameterValue, + ProposedChange, + ResidualProposal, + RunError, + RunStatus, + TextEdit, + TransformResult, + markdown_sha256, +) + +JsonValue: TypeAlias = bool | int | float | str | list["JsonValue"] | dict[str, "JsonValue"] | None +JsonObject: TypeAlias = dict[str, JsonValue] + + +class ReportingError(ValueError): + """A core result cannot be represented without losing its audit contract.""" + + +@dataclass(frozen=True, slots=True) +class DocumentReport: + """Rendered artifacts and index facts for one document result.""" + + document_id: str + source_label: str + status: RunStatus + input_sha256: str + current_sha256: str + change_count: int + result_json: bytes + cleaned_markdown: bytes | None + diff: bytes | None + + +def _parameter_value_json(value: ParameterValue) -> JsonValue: + if isinstance(value, tuple): + return [_parameter_value_json(item) for item in value] + return value + + +def component_info_json(component: ComponentInfo) -> JsonObject: + """Represent validated component metadata without changing tuple order.""" + parameters: list[JsonValue] = [] + for name, value in component.parameters: + parameters.append([name, _parameter_value_json(value)]) + return { + "component_id": component.component_id, + "version": component.version, + "parameters": parameters, + "applicability": component.applicability, + } + + +def _proposal_reference_json(change: Change) -> JsonObject: + reference = change.proposal_ref + return { + "component_position": reference.component_position, + "snapshot_sha256": reference.snapshot_sha256, + "proposal_index": reference.proposal_index, + } + + +def _text_edit_json(edit: TextEdit) -> JsonObject: + return { + "snapshot_sha256": edit.snapshot_sha256, + "span": {"start": edit.span.start, "end": edit.span.end}, + "expected_text": edit.expected_text, + "replacement": edit.replacement, + } + + +def _proposed_change_json(proposal: ProposedChange) -> JsonObject: + edits: list[JsonValue] = [_text_edit_json(edit) for edit in proposal.edits] + return { + "snapshot_sha256": proposal.snapshot_sha256, + "reason": proposal.reason, + "edits": edits, + } + + +def _residual_proposal_json(residual: ResidualProposal) -> JsonObject: + reference = residual.proposal_ref + return { + "component_id": residual.component_id, + "component_version": residual.component_version, + "component_position": residual.component_position, + "proposal_ref": { + "component_position": reference.component_position, + "snapshot_sha256": reference.snapshot_sha256, + "proposal_index": reference.proposal_index, + }, + "proposal": _proposed_change_json(residual.proposal), + } + + +def _run_error_json(error: RunError) -> JsonObject: + return { + "component_id": error.component_id, + "component_version": error.component_version, + "component_position": error.component_position, + "stage": error.stage.value, + "error_type": error.error_type, + "message": error.message, + } + + +def _line_column(markdown: str, offset: int) -> tuple[int, int]: + if offset < 0 or offset > len(markdown): + raise ReportingError("a change offset is outside its recorded snapshot") + + line = 1 + column = 1 + position = 0 + while position < offset: + character = markdown[position] + if character == "\r": + line += 1 + column = 1 + if position + 1 < offset and markdown[position + 1] == "\n": + position += 2 + else: + position += 1 + elif character == "\n": + line += 1 + column = 1 + position += 1 + else: + column += 1 + position += 1 + return line, column + + +def _validate_change_identity(change: Change, components: tuple[ComponentInfo, ...]) -> None: + if change.component_position < 0 or change.component_position >= len(components): + raise ReportingError("a change has no matching component position") + component = components[change.component_position] + if change.component_id != component.component_id or change.component_version != component.version: + raise ReportingError("a change identity does not match the recorded component") + if change.proposal_ref.component_position != change.component_position: + raise ReportingError("a change proposal reference has the wrong component position") + if change.proposal_ref.snapshot_sha256 != change.before_sha256: + raise ReportingError("a change proposal reference targets the wrong snapshot") + + +def _replay_changes(input_markdown: str, result: TransformResult) -> tuple[tuple[tuple[int, int], ...], str]: + if markdown_sha256(input_markdown) != result.input_sha256: + raise ReportingError("input Markdown does not match the transform result") + + current = input_markdown + locations: list[tuple[int, int]] = [] + cursor = 0 + while cursor < len(result.changes): + first = result.changes[cursor] + batch_key = (first.component_position, first.before_sha256, first.after_sha256) + batch: list[Change] = [] + while cursor < len(result.changes): + candidate = result.changes[cursor] + candidate_key = (candidate.component_position, candidate.before_sha256, candidate.after_sha256) + if candidate_key != batch_key: + break + batch.append(candidate) + cursor += 1 + + if markdown_sha256(current) != first.before_sha256: + raise ReportingError("a change batch does not follow the recorded snapshot chain") + + for change in batch: + _validate_change_identity(change, result.components) + if change.before_sha256 != first.before_sha256 or change.after_sha256 != first.after_sha256: + raise ReportingError("a change batch contains inconsistent snapshot hashes") + if change.span.end > len(current): + raise ReportingError("a change span is outside its recorded snapshot") + if len(change.before) != change.span.end - change.span.start: + raise ReportingError("a change before value does not match its span length") + if current[change.span.start : change.span.end] != change.before: + raise ReportingError("a change before value does not match its recorded snapshot") + locations.append(_line_column(current, change.span.start)) + + application_order = sorted( + batch, + key=lambda change: ( + change.span.start, + change.span.end, + change.proposal_ref.proposal_index, + change.edit_index, + ), + reverse=True, + ) + for change in application_order: + current = current[: change.span.start] + change.after + current[change.span.end :] + if markdown_sha256(current) != first.after_sha256: + raise ReportingError("replayed changes do not produce the recorded batch hash") + + current_markdown = result.output_markdown if result.status is RunStatus.SUCCESS else result.partial_markdown + if current_markdown is None: + raise ReportingError("a transform result does not contain its status-specific Markdown") + if current != current_markdown or markdown_sha256(current) != result.current_sha256: + raise ReportingError("replayed changes do not produce the transform result's current snapshot") + return tuple(locations), current + + +def _change_json(change: Change, location: tuple[int, int]) -> JsonObject: + line, column = location + return { + "component_id": change.component_id, + "component_version": change.component_version, + "component_position": change.component_position, + "proposal_ref": _proposal_reference_json(change), + "edit_index": change.edit_index, + "reason": change.reason, + "span": {"start": change.span.start, "end": change.span.end}, + "location": {"line": line, "column": column}, + "before": change.before, + "after": change.after, + "before_sha256": change.before_sha256, + "after_sha256": change.after_sha256, + } + + +def encode_json(payload: JsonObject) -> bytes: + """Encode a schema object using the approved stable local representation.""" + return (json.dumps(payload, ensure_ascii=False, indent=2) + "\n").encode("utf-8") + + +def build_unified_diff(document_id: str, before: str, after: str) -> bytes: + """Build the human review view using logical document names only.""" + if before == after: + return b"" + lines = unified_diff( + before.splitlines(keepends=True), + after.splitlines(keepends=True), + fromfile=f"a/{document_id}.md", + tofile=f"b/{document_id}.md", + n=3, + lineterm="\n", + ) + normalized: list[str] = [] + line_endings = ("\r\n", "\n", "\r", "\v", "\f", "\x1c", "\x1d", "\x1e", "\x85", "\u2028", "\u2029") + for line in lines: + for ending in line_endings: + if line.endswith(ending): + normalized.append(line[: -len(ending)] + "\n") + break + else: + normalized.append(line + "\n\\ No newline at end of file\n") + return "".join(normalized).encode("utf-8") + + +def build_document_report( + *, + document_id: str, + source_label: str, + input_markdown: str, + result: TransformResult, +) -> DocumentReport: + """Render one core result after replaying and verifying every applied batch.""" + locations, current_markdown = _replay_changes(input_markdown, result) + changes: list[JsonValue] = [ + _change_json(change, location) for change, location in zip(result.changes, locations, strict=True) + ] + errors: list[JsonValue] = [_run_error_json(error) for error in result.errors] + residual_proposals: list[JsonValue] = [ + _residual_proposal_json(residual) for residual in result.residual_proposals + ] + + if result.status is RunStatus.SUCCESS: + cleaned_path: JsonValue = "cleaned.md" + diff_path: JsonValue = "changes.diff" + cleaned_markdown = current_markdown.encode("utf-8") + diff = build_unified_diff(document_id, input_markdown, current_markdown) + else: + cleaned_path = None + diff_path = None + cleaned_markdown = None + diff = None + + payload: JsonObject = { + "schema_version": 1, + "document": { + "document_id": document_id, + "source_label": source_label, + }, + "status": result.status.value, + "input_sha256": result.input_sha256, + "current_sha256": result.current_sha256, + "changes": changes, + "errors": errors, + "residual_proposals": residual_proposals, + "output": { + "cleaned_path": cleaned_path, + "diff_path": diff_path, + }, + } + return DocumentReport( + document_id=document_id, + source_label=source_label, + status=result.status, + input_sha256=result.input_sha256, + current_sha256=result.current_sha256, + change_count=len(result.changes), + result_json=encode_json(payload), + cleaned_markdown=cleaned_markdown, + diff=diff, + ) diff --git a/tests/test_artifact_store.py b/tests/test_artifact_store.py new file mode 100644 index 0000000..f778888 --- /dev/null +++ b/tests/test_artifact_store.py @@ -0,0 +1,268 @@ +from __future__ import annotations + +import json +import stat +from hashlib import sha256 +from pathlib import Path + +import pytest + +import mdpolish.artifact_store as artifact_store +from mdpolish.artifact_store import ArtifactStoreError, StoredDocument, publish_run + + +def stored_document(document_id: str = "paper", output: bytes | None = b"cleaned\n") -> StoredDocument: + status = "success" if output is not None else "failed" + current_sha256 = sha256(output).hexdigest() if output is not None else "2" * 64 + payload = { + "schema_version": 1, + "document": {"document_id": document_id, "source_label": f"inputs/{document_id}.md"}, + "status": status, + "input_sha256": "1" * 64, + "current_sha256": current_sha256, + "changes": [], + "errors": [] if status == "success" else [{"error_type": "SyntheticError"}], + "residual_proposals": [], + "output": { + "cleaned_path": "cleaned.md" if output is not None else None, + "diff_path": "changes.diff" if output is not None else None, + }, + } + return StoredDocument( + document_id=document_id, + result_json=(json.dumps(payload, indent=2) + "\n").encode(), + cleaned_markdown=output, + diff=b"" if output is not None else None, + output_sha256=sha256(output).hexdigest() if output is not None else None, + ) + + +def manifest_json( + run_date: str, + run_id: str, + documents: tuple[StoredDocument, ...], +) -> bytes: + indexes: list[dict[str, object]] = [] + statuses: list[str] = [] + change_count = 0 + for document in documents: + report = json.loads(document.result_json) + status = report["status"] + statuses.append(status) + change_count += len(report["changes"]) + base = f"documents/{document.document_id}" + indexes.append( + { + "document_id": document.document_id, + "source_label": report["document"]["source_label"], + "status": status, + "input_sha256": report["input_sha256"], + "current_sha256": report["current_sha256"], + "change_count": len(report["changes"]), + "result_path": f"{base}/result.json", + "cleaned_path": f"{base}/cleaned.md" if status == "success" else None, + "diff_path": f"{base}/changes.diff" if status == "success" else None, + } + ) + failed_count = statuses.count("failed") + unstable_count = statuses.count("unstable") + overall_status = "failed" if failed_count else "unstable" if unstable_count else "success" + payload = { + "schema_version": 1, + "run": {"run_id": run_id, "run_date": run_date, "status": overall_status}, + "documents": indexes, + "summary": { + "document_count": len(documents), + "success_count": statuses.count("success"), + "failed_count": failed_count, + "unstable_count": unstable_count, + "change_count": change_count, + }, + } + return (json.dumps(payload, indent=2) + "\n").encode() + + +def test_publish_run_creates_private_date_layout_and_status_specific_files(tmp_path: Path) -> None: + artifacts_root = tmp_path / "artifacts" + documents = (stored_document("success"), stored_document("failed", None)) + + run_directory = publish_run( + artifacts_root=artifacts_root, + run_date="2026-08-22", + run_id="example-run", + manifest_json=manifest_json("2026-08-22", "example-run", documents), + documents=documents, + ) + + assert run_directory == artifacts_root / "2026-08-22" / "runs" / "example-run" + assert json.loads((run_directory / "manifest.json").read_bytes())["run"]["status"] == "failed" + assert (run_directory / "documents/success/result.json").is_file() + assert (run_directory / "documents/success/cleaned.md").read_bytes() == b"cleaned\n" + assert (run_directory / "documents/success/changes.diff").read_bytes() == b"" + assert (run_directory / "documents/failed/result.json").is_file() + assert not (run_directory / "documents/failed/cleaned.md").exists() + assert not (run_directory / "documents/failed/changes.diff").exists() + + for directory in ( + artifacts_root, + artifacts_root / "2026-08-22", + artifacts_root / "2026-08-22/runs", + run_directory, + run_directory / "documents", + run_directory / "documents/success", + ): + assert stat.S_IMODE(directory.stat().st_mode) == 0o700 + for artifact_file in run_directory.rglob("*"): + if artifact_file.is_file(): + assert stat.S_IMODE(artifact_file.stat().st_mode) == 0o600 + + +def test_publish_run_rejects_existing_target_without_overwriting(tmp_path: Path) -> None: + artifacts_root = tmp_path / "artifacts" + documents = (stored_document(),) + first_manifest = manifest_json("2026-08-22", "same-run", documents) + run_directory = publish_run( + artifacts_root=artifacts_root, + run_date="2026-08-22", + run_id="same-run", + manifest_json=first_manifest, + documents=documents, + ) + + with pytest.raises(ArtifactStoreError, match="already exists"): + publish_run( + artifacts_root=artifacts_root, + run_date="2026-08-22", + run_id="same-run", + manifest_json=first_manifest, + documents=documents, + ) + + assert (run_directory / "manifest.json").read_bytes() == first_manifest + + +@pytest.mark.parametrize( + ("run_date", "run_id"), + [ + ("2026-8-22", "valid"), + ("2026-02-30", "valid"), + ("2026-08-22", "Uppercase"), + ("2026-08-22", "../escape"), + ("2026-08-22", "two..dots"), + ], +) +def test_publish_run_rejects_unsafe_date_and_run_id(tmp_path: Path, run_date: str, run_id: str) -> None: + documents = (stored_document(),) + with pytest.raises(ArtifactStoreError): + publish_run( + artifacts_root=tmp_path / "artifacts", + run_date=run_date, + run_id=run_id, + manifest_json=manifest_json(run_date, run_id, documents), + documents=documents, + ) + + +def test_publish_run_rejects_inconsistent_or_duplicate_document_artifacts(tmp_path: Path) -> None: + valid = stored_document() + bad_hash = StoredDocument("paper", valid.result_json, b"output", b"", "0" * 64) + with pytest.raises(ArtifactStoreError, match="output hash"): + publish_run( + artifacts_root=tmp_path / "artifacts-a", + run_date="2026-08-22", + run_id="bad-hash", + manifest_json=manifest_json("2026-08-22", "bad-hash", (bad_hash,)), + documents=(bad_hash,), + ) + + duplicates = (stored_document(), stored_document()) + with pytest.raises(ArtifactStoreError, match="unique"): + publish_run( + artifacts_root=tmp_path / "artifacts-b", + run_date="2026-08-22", + run_id="duplicate", + manifest_json=manifest_json("2026-08-22", "duplicate", duplicates), + documents=duplicates, + ) + + +def test_publish_run_rejects_manifest_path_or_document_mismatch(tmp_path: Path) -> None: + documents = (stored_document(),) + wrong_date = manifest_json("2026-08-21", "review", documents) + with pytest.raises(ArtifactStoreError, match="identity"): + publish_run( + artifacts_root=tmp_path / "artifacts-a", + run_date="2026-08-22", + run_id="review", + manifest_json=wrong_date, + documents=documents, + ) + + payload = json.loads(manifest_json("2026-08-22", "review", documents)) + payload["documents"][0]["change_count"] = 99 + mismatched_index = (json.dumps(payload, indent=2) + "\n").encode() + with pytest.raises(ArtifactStoreError, match="index"): + publish_run( + artifacts_root=tmp_path / "artifacts-b", + run_date="2026-08-22", + run_id="review", + manifest_json=mismatched_index, + documents=documents, + ) + + +def test_publish_race_does_not_replace_a_new_target( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + documents = (stored_document(),) + original_rename = artifact_store._rename_no_replace + + def create_competing_target(source: Path, target: Path) -> None: + target.mkdir(mode=0o700) + (target / "keep").write_bytes(b"existing") + original_rename(source, target) + + monkeypatch.setattr(artifact_store, "_rename_no_replace", create_competing_target) + with pytest.raises(ArtifactStoreError, match="already exists"): + publish_run( + artifacts_root=tmp_path / "artifacts", + run_date="2026-08-22", + run_id="raced", + manifest_json=manifest_json("2026-08-22", "raced", documents), + documents=documents, + ) + + target = tmp_path / "artifacts/2026-08-22/runs/raced" + assert (target / "keep").read_bytes() == b"existing" + assert not any(path.name.startswith(".raced.") for path in target.parent.iterdir()) + + +def test_write_failure_cleans_temporary_directory_and_does_not_publish( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + original_write = artifact_store._write_private_file + call_count = 0 + + def fail_second_write(path: Path, content: bytes) -> None: + nonlocal call_count + call_count += 1 + if call_count == 2: + raise OSError("synthetic write failure") + original_write(path, content) + + monkeypatch.setattr(artifact_store, "_write_private_file", fail_second_write) + documents = (stored_document(),) + + with pytest.raises(OSError, match="synthetic"): + publish_run( + artifacts_root=tmp_path / "artifacts", + run_date="2026-08-22", + run_id="broken", + manifest_json=manifest_json("2026-08-22", "broken", documents), + documents=documents, + ) + + runs_directory = tmp_path / "artifacts/2026-08-22/runs" + assert list(runs_directory.iterdir()) == [] diff --git a/tests/test_experiment.py b/tests/test_experiment.py new file mode 100644 index 0000000..6c23f9c --- /dev/null +++ b/tests/test_experiment.py @@ -0,0 +1,335 @@ +from __future__ import annotations + +import json +from collections.abc import Mapping +from datetime import UTC, datetime, timedelta, timezone +from pathlib import Path + +import pytest + +from mdpolish import Component, DocumentSnapshot, Pipeline, ProposedChange, RunStatus, TextEdit, TextSpan +from mdpolish.experiment import ExperimentError, InputDocument, ToolMetadata, run_experiment + + +class ConditionalComponent(Component): + @property + def component_id(self) -> str: + return "test.conditional" + + @property + def version(self) -> str: + return "1.0.0" + + @property + def parameters(self) -> Mapping[str, object]: + return {"needle": "old", "replacement": "new"} + + @property + def applicability(self) -> str: + return "替换测试标记 old，遇到 boom 时模拟组件失败，排除其他内容。" + + def _propose_changes(self, snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]: + if snapshot.markdown == "boom": + raise RuntimeError("private source") + position = snapshot.markdown.find("old") + if position < 0: + return () + edit = TextEdit(snapshot.sha256, TextSpan(position, position + 3), "old", "new") + return (ProposedChange(snapshot.sha256, "replace old test marker", (edit,)),) + + +class NonIdempotentComponent(Component): + @property + def component_id(self) -> str: + return "test.non-idempotent" + + @property + def version(self) -> str: + return "1.0.0" + + @property + def parameters(self) -> Mapping[str, object]: + return {} + + @property + def applicability(self) -> str: + return "在测试文本末尾反复插入标记，只用于验证 unstable 产物边界。" + + def _propose_changes(self, snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]: + position = len(snapshot.markdown) + edit = TextEdit(snapshot.sha256, TextSpan(position, position), "", "!") + return (ProposedChange(snapshot.sha256, "append synthetic marker", (edit,)),) + + +def tool_metadata() -> ToolMetadata: + return ToolMetadata( + name="mdpolish", + package_version="0.1.0", + python_version="3.13.11", + platform="linux-x86_64", + git_commit="a" * 40, + git_dirty=False, + ) + + +def input_document(path: Path, document_id: str) -> InputDocument: + return InputDocument(document_id, path, f"inputs/{path.name}") + + +def test_run_experiment_publishes_manifest_reports_diff_and_preserves_inputs(tmp_path: Path) -> None: + first_path = tmp_path / "first.md" + second_path = tmp_path / "second.md" + first_path.write_bytes(b"before old\r\nafter\r\n") + second_path.write_text("unchanged\n", encoding="utf-8") + original_first = first_path.read_bytes() + original_second = second_path.read_bytes() + local_timezone = timezone(timedelta(hours=8)) + started_at = datetime(2026, 8, 22, 10, 30, tzinfo=local_timezone) + completed_at = datetime(2026, 8, 22, 2, 31, tzinfo=UTC) + + result = run_experiment( + pipeline=Pipeline([ConditionalComponent()]), + documents=(input_document(first_path, "first"), input_document(second_path, "second")), + run_id="local-review", + artifacts_root=tmp_path / "artifacts", + started_at=started_at, + completed_at=completed_at, + tool=tool_metadata(), + ) + + assert result.status is RunStatus.SUCCESS + assert result.run_directory == tmp_path / "artifacts/2026-08-22/runs/local-review" + assert result.document_count == 2 + assert result.success_count == 2 + assert result.change_count == 1 + assert first_path.read_bytes() == original_first + assert second_path.read_bytes() == original_second + assert (result.run_directory / "documents/first/cleaned.md").read_bytes() == b"before new\r\nafter\r\n" + assert (result.run_directory / "documents/second/cleaned.md").read_bytes() == original_second + assert (result.run_directory / "documents/second/changes.diff").read_bytes() == b"" + + manifest = json.loads((result.run_directory / "manifest.json").read_bytes()) + assert manifest["run"]["run_date"] == "2026-08-22" + assert manifest["run"]["utc_offset"] == "+08:00" + assert manifest["run"]["started_at_utc"] == "2026-08-22T02:30:00Z" + assert manifest["run"]["completed_at_utc"] == "2026-08-22T02:31:00Z" + assert manifest["run"]["retention_until"] == "2026-09-21T02:31:00Z" + assert manifest["pipeline"]["components"][0]["component_id"] == "test.conditional" + assert manifest["pipeline"]["components"][0]["parameters"] == [ + ["needle", "old"], + ["replacement", "new"], + ] + assert manifest["summary"] == { + "document_count": 2, + "success_count": 2, + "failed_count": 0, + "unstable_count": 0, + "change_count": 1, + } + first_report = json.loads((result.run_directory / "documents/first/result.json").read_bytes()) + assert first_report["changes"][0]["location"] == {"line": 1, "column": 8} + assert first_report["changes"][0]["before"] == "old" + assert b"--- a/first.md\n+++ b/first.md\n" in ( + result.run_directory / "documents/first/changes.diff" + ).read_bytes() + + +def test_document_failure_is_isolated_and_never_writes_partial_markdown(tmp_path: Path) -> None: + good_path = tmp_path / "good.md" + bad_path = tmp_path / "bad.md" + good_path.write_text("old", encoding="utf-8") + bad_path.write_text("boom", encoding="utf-8") + + result = run_experiment( + pipeline=Pipeline([ConditionalComponent()]), + documents=(input_document(good_path, "good"), input_document(bad_path, "bad")), + run_id="mixed", + artifacts_root=tmp_path / "artifacts", + started_at=datetime(2026, 8, 22, 10, tzinfo=UTC), + completed_at=datetime(2026, 8, 22, 10, 1, tzinfo=UTC), + tool=tool_metadata(), + ) + + assert result.status is RunStatus.FAILED + assert result.success_count == 1 + assert result.failed_count == 1 + assert (result.run_directory / "documents/good/cleaned.md").read_text() == "new" + assert (result.run_directory / "documents/bad/result.json").is_file() + assert not (result.run_directory / "documents/bad/cleaned.md").exists() + assert not (result.run_directory / "documents/bad/changes.diff").exists() + bad_report = json.loads((result.run_directory / "documents/bad/result.json").read_bytes()) + assert bad_report["status"] == "failed" + assert "partial_markdown" not in bad_report + + +def test_preflight_rejects_invalid_utf8_without_running_or_creating_artifacts(tmp_path: Path) -> None: + invalid_path = tmp_path / "invalid.md" + invalid_path.write_bytes(b"\xff") + artifacts_root = tmp_path / "artifacts" + + with pytest.raises(ExperimentError, match="strict UTF-8"): + run_experiment( + pipeline=Pipeline([ConditionalComponent()]), + documents=(input_document(invalid_path, "invalid"),), + run_id="invalid-input", + artifacts_root=artifacts_root, + started_at=datetime(2026, 8, 22, tzinfo=UTC), + tool=tool_metadata(), + ) + + assert not artifacts_root.exists() + + +def test_preflight_rejects_invalid_document_manifests_before_creating_artifacts(tmp_path: Path) -> None: + source_path = tmp_path / "paper.md" + source_path.write_text("content", encoding="utf-8") + directory_path = tmp_path / "directory" + directory_path.mkdir() + missing_path = tmp_path / "missing.md" + cases = ( + (input_document(source_path, "duplicate"), input_document(source_path, "duplicate")), + (input_document(source_path, "first"), input_document(source_path, "second")), + (input_document(missing_path, "missing"),), + (input_document(directory_path, "directory"),), + (input_document(source_path, "two..dots"),), + ) + + for index, documents in enumerate(cases): + artifacts_root = tmp_path / f"artifacts-{index}" + with pytest.raises(ExperimentError): + run_experiment( + pipeline=Pipeline([]), + documents=documents, + run_id="preflight", + artifacts_root=artifacts_root, + started_at=datetime(2026, 8, 22, tzinfo=UTC), + tool=tool_metadata(), + ) + assert not artifacts_root.exists() + + +def test_preflight_rejects_an_output_path_nested_under_an_input_file(tmp_path: Path) -> None: + source_path = tmp_path / "paper.md" + source_path.write_text("content", encoding="utf-8") + + with pytest.raises(ExperimentError, match="cannot overlap"): + run_experiment( + pipeline=Pipeline([]), + documents=(input_document(source_path, "paper"),), + run_id="overlap", + artifacts_root=source_path, + started_at=datetime(2026, 8, 22, tzinfo=UTC), + tool=tool_metadata(), + ) + + assert source_path.read_text(encoding="utf-8") == "content" + + +def test_utf8_bom_crlf_and_missing_final_newline_are_preserved_exactly(tmp_path: Path) -> None: + source_path = tmp_path / "bom.md" + empty_path = tmp_path / "empty.md" + source_bytes = b"\xef\xbb\xbfhead\r\nlast" + source_path.write_bytes(source_bytes) + empty_path.write_bytes(b"") + + result = run_experiment( + pipeline=Pipeline([]), + documents=(input_document(source_path, "bom"), input_document(empty_path, "empty")), + run_id="byte-preservation", + artifacts_root=tmp_path / "artifacts", + started_at=datetime(2026, 8, 22, tzinfo=UTC), + completed_at=datetime(2026, 8, 22, 0, 1, tzinfo=UTC), + tool=tool_metadata(), + ) + + assert source_path.read_bytes() == source_bytes + assert (result.run_directory / "documents/bom/cleaned.md").read_bytes() == source_bytes + assert (result.run_directory / "documents/bom/changes.diff").read_bytes() == b"" + assert empty_path.read_bytes() == b"" + assert (result.run_directory / "documents/empty/cleaned.md").read_bytes() == b"" + assert (result.run_directory / "documents/empty/changes.diff").read_bytes() == b"" + + +def test_unstable_document_only_publishes_a_result_json(tmp_path: Path) -> None: + source_path = tmp_path / "paper.md" + source_path.write_text("content", encoding="utf-8") + + result = run_experiment( + pipeline=Pipeline([NonIdempotentComponent()]), + documents=(input_document(source_path, "paper"),), + run_id="unstable", + artifacts_root=tmp_path / "artifacts", + started_at=datetime(2026, 8, 22, tzinfo=UTC), + completed_at=datetime(2026, 8, 22, 0, 1, tzinfo=UTC), + tool=tool_metadata(), + ) + + assert result.status is RunStatus.UNSTABLE + assert result.unstable_count == 1 + assert (result.run_directory / "documents/paper/result.json").is_file() + assert not (result.run_directory / "documents/paper/cleaned.md").exists() + assert not (result.run_directory / "documents/paper/changes.diff").exists() + manifest = json.loads((result.run_directory / "manifest.json").read_bytes()) + assert manifest["run"]["status"] == "unstable" + + +def test_existing_same_date_run_is_rejected_without_overwriting(tmp_path: Path) -> None: + source_path = tmp_path / "paper.md" + source_path.write_text("old", encoding="utf-8") + artifacts_root = tmp_path / "artifacts" + pipeline = Pipeline([ConditionalComponent()]) + documents = (input_document(source_path, "paper"),) + started_at = datetime(2026, 8, 22, tzinfo=UTC) + completed_at = datetime(2026, 8, 22, 0, 1, tzinfo=UTC) + tool = tool_metadata() + first = run_experiment( + pipeline=pipeline, + documents=documents, + run_id="same", + artifacts_root=artifacts_root, + started_at=started_at, + completed_at=completed_at, + tool=tool, + ) + original_manifest = (first.run_directory / "manifest.json").read_bytes() + + with pytest.raises(ExperimentError, match="already exists"): + run_experiment( + pipeline=pipeline, + documents=documents, + run_id="same", + artifacts_root=artifacts_root, + started_at=started_at, + completed_at=completed_at, + tool=tool, + ) + + assert (first.run_directory / "manifest.json").read_bytes() == original_manifest + + +def test_same_run_id_on_another_local_date_uses_a_separate_directory(tmp_path: Path) -> None: + source_path = tmp_path / "paper.md" + source_path.write_text("unchanged", encoding="utf-8") + artifacts_root = tmp_path / "artifacts" + + first = run_experiment( + pipeline=Pipeline([]), + documents=(input_document(source_path, "paper"),), + run_id="daily", + artifacts_root=artifacts_root, + started_at=datetime(2026, 8, 22, 23, tzinfo=UTC), + completed_at=datetime(2026, 8, 22, 23, 1, tzinfo=UTC), + tool=tool_metadata(), + ) + second = run_experiment( + pipeline=Pipeline([]), + documents=(input_document(source_path, "paper"),), + run_id="daily", + artifacts_root=artifacts_root, + started_at=datetime(2026, 8, 23, 0, tzinfo=UTC), + completed_at=datetime(2026, 8, 23, 0, 1, tzinfo=UTC), + tool=tool_metadata(), + ) + + assert first.run_directory.parent.parent.name == "2026-08-22" + assert second.run_directory.parent.parent.name == "2026-08-23" diff --git a/tests/test_reporting.py b/tests/test_reporting.py new file mode 100644 index 0000000..b2e7522 --- /dev/null +++ b/tests/test_reporting.py @@ -0,0 +1,177 @@ +from __future__ import annotations + +import json +from collections.abc import Mapping +from dataclasses import replace + +import pytest + +from mdpolish import Component, DocumentSnapshot, Pipeline, ProposedChange, RunStatus, TextEdit, TextSpan +from mdpolish.reporting import ReportingError, build_document_report, build_unified_diff + + +class ReplaceComponent(Component): + def __init__(self, needle: str, replacement: str, component_id: str) -> None: + self.needle = needle + self.replacement = replacement + self._component_id = component_id + + @property + def component_id(self) -> str: + return self._component_id + + @property + def version(self) -> str: + return "1.0.0" + + @property + def parameters(self) -> Mapping[str, object]: + return {"needle": self.needle, "replacement": self.replacement} + + @property + def applicability(self) -> str: + return "处理精确测试标记，要求完整匹配，排除其他文本。" + + def _propose_changes(self, snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]: + position = snapshot.markdown.find(self.needle) + if position < 0: + return () + edit = TextEdit( + snapshot.sha256, + TextSpan(position, position + len(self.needle)), + self.needle, + self.replacement, + ) + return (ProposedChange(snapshot.sha256, f"replace {self.needle}", (edit,)),) + + +class ExplodingComponent(ReplaceComponent): + def __init__(self) -> None: + super().__init__("unused", "unused-replacement", "test.exploding") + + def _propose_changes(self, snapshot: DocumentSnapshot) -> tuple[ProposedChange, ...]: + raise RuntimeError(f"private: {snapshot.markdown}") + + +def test_success_report_replays_multiple_component_snapshots_and_derives_locations() -> None: + markdown = "x\r\ntarget\n" + pipeline = Pipeline( + [ + ReplaceComponent("x", "XX", "test.expand"), + ReplaceComponent("target", "done", "test.target"), + ] + ) + result = pipeline.transform(markdown) + + report = build_document_report( + document_id="paper", + source_label="data/md/paper.md", + input_markdown=markdown, + result=result, + ) + payload = json.loads(report.result_json) + + assert result.status is RunStatus.SUCCESS + assert report.cleaned_markdown == b"XX\r\ndone\n" + assert payload["changes"][0]["location"] == {"line": 1, "column": 1} + assert payload["changes"][1]["location"] == {"line": 2, "column": 1} + assert payload["changes"][1]["before"] == "target" + assert payload["changes"][1]["after"] == "done" + assert payload["changes"][1]["before_sha256"] == result.changes[1].before_sha256 + assert payload["changes"][1]["after_sha256"] == result.changes[1].after_sha256 + assert report.diff is not None + assert b"--- a/paper.md\n+++ b/paper.md\n" in report.diff + assert b"data/md/paper.md" not in report.diff + + +def test_zero_change_success_still_has_cleaned_markdown_and_empty_diff() -> None: + markdown = "中文\nCafe\u0301\n" + result = Pipeline([]).transform(markdown) + + report = build_document_report( + document_id="unchanged", + source_label="论文.md", + input_markdown=markdown, + result=result, + ) + + assert report.cleaned_markdown == markdown.encode() + assert report.diff == b"" + assert report.change_count == 0 + assert report.result_json.endswith(b"\n") + assert "论文".encode() in report.result_json + assert b"\\u4e2d" not in report.result_json + + +def test_diff_preserves_final_newline_changes_and_uses_report_newlines() -> None: + diff = build_unified_diff("paper", "same\r\n", "same") + + assert b"--- a/paper.md\n+++ b/paper.md\n" in diff + assert b"-same\n+same\n\\ No newline at end of file\n" in diff + assert b"\r" not in diff + + +def test_failed_report_keeps_change_audit_but_does_not_render_partial_markdown() -> None: + pipeline = Pipeline( + [ + ReplaceComponent("a", "b", "test.first"), + ExplodingComponent(), + ] + ) + result = pipeline.transform("a") + + report = build_document_report( + document_id="failed", + source_label="failed.md", + input_markdown="a", + result=result, + ) + payload = json.loads(report.result_json) + + assert result.status is RunStatus.FAILED + assert report.cleaned_markdown is None + assert report.diff is None + assert payload["output"] == {"cleaned_path": None, "diff_path": None} + assert len(payload["changes"]) == 1 + assert payload["errors"][0]["error_type"] == "RuntimeError" + assert "private" not in payload["errors"][0]["message"] + assert "partial_markdown" not in payload + + +def test_unstable_report_serializes_residual_proposals_without_partial_markdown() -> None: + pipeline = Pipeline( + [ + ReplaceComponent("bad", "good", "test.to-good"), + ReplaceComponent("good", "bad", "test.to-bad"), + ] + ) + result = pipeline.transform("bad") + + report = build_document_report( + document_id="unstable", + source_label="unstable.md", + input_markdown="bad", + result=result, + ) + payload = json.loads(report.result_json) + + assert result.status is RunStatus.UNSTABLE + assert report.cleaned_markdown is None + assert report.diff is None + assert payload["residual_proposals"][0]["proposal"]["reason"] == "replace bad" + assert payload["residual_proposals"][0]["proposal"]["edits"][0]["expected_text"] == "bad" + assert "partial_markdown" not in payload + + +def test_report_rejects_a_tampered_change_hash_chain() -> None: + result = Pipeline([ReplaceComponent("a", "b", "test.replace")]).transform("a") + tampered_change = replace(result.changes[0], after_sha256="0" * 64) + tampered_result = replace(result, changes=(tampered_change,)) + + with pytest.raises(ReportingError, match="batch hash"): + build_document_report( + document_id="tampered", + source_label="tampered.md", + input_markdown="a", + result=tampered_result, + )