Compare commits
29 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| a28d94451e | |||
| a04d87e8d7 | |||
| c60011b034 | |||
| 9f7d407120 | |||
| 0572611af7 | |||
| fe616cf91d | |||
| adc069447a | |||
| 463eca380d | |||
| 0a6d6225db | |||
| 166b2865d0 | |||
| b0ab39edbf | |||
| a07a6b096b | |||
| 53d2f089e1 | |||
| 5e2c35a812 | |||
| b1bc06e2b1 | |||
| da77b123ec | |||
| 9474c76ab0 | |||
| 1ff83bbfe0 | |||
| 6c640fcca3 | |||
| d2455e8fd4 | |||
| ab00aa4457 | |||
| 433039be79 | |||
| b4812e12c8 | |||
| 067b15be48 | |||
| 7b2f6105f3 | |||
| 393f2bf617 | |||
| 87c261bf73 | |||
| 300ced5dbd | |||
| a81cc91124 |
@@ -69,6 +69,17 @@ LLM_CIRCUIT_BREAKER_COOLDOWN=60 # 或 LLM__BREAKER__COOLDOWN_S
|
||||
# ── 在几毫秒内死掉且 MAX_ATTEMPTS 一格用不上。wait 不削弱保护(等待期照样
|
||||
# ── 不发请求),只是把最坏墙钟拉长到 BACKPRESSURE__STALL_WINDOW_S ──
|
||||
# LLM__CIRCUIT_OPEN=fail_fast # 熔断开路: fail_fast(默认) | wait
|
||||
# LLM__CALL_DEADLINE_S= # 一次逻辑调用的墙钟硬边界(秒);缺省不设 = 不启用
|
||||
# ── 治理对象是"等待"(退避/配额轮询/熔断冷却/结构化重问/embedding 分批共享一份),
|
||||
# ── 不是单次 HTTP 超时(那是 TIMEOUT_S)。清理仍在 finally 跑完: 返回时刻 = 期限 + 清理耗时,
|
||||
# ── 且到期 ≠ 未产出、≠ 未计费。到期抛 CallDeadlineExceeded(不属四分类、
|
||||
# ── 不属 GatewayUnavailableError 族、无 retry_after_s);非法值(0/负/nan/inf)装配期报错 ──
|
||||
# LLM__HEDGE__AFTER_S= # 长尾对冲触发阈值(秒);缺省不设 = 关闭(仅 chat 生效)
|
||||
# LLM__HEDGE__MAX_EXTRA=1 # 每次逻辑调用最多对冲路数;v1 仅单路生效(>1 仅装配期 warning)
|
||||
# ── 触发语义: 流式 = 超阈值且首 token 未至(不误杀慢生成);非流式 = 纯总时长阈值(无中途信号,
|
||||
# ── 「挂起 vs 慢生成」物理不可分,建议取源 p50 的数倍)。对冲向异源并发再发一次,走完整限流/熔断准入,
|
||||
# ── 拿不到配额静默放弃。成本含义: 开启即用配额换延迟——触发窗口内 in-flight 翻倍,输家被取消后按 est
|
||||
# ── 保留预扣(不喂熔断),且输家可能已被上游计费、取消止不住。须与 CALL_DEADLINE_S 组合时强制阈值 < 期限 ──
|
||||
|
||||
# ══ 装配选择(PGW_*)══
|
||||
PGW_LIMITER_BACKEND=memory # memory | redis(redis 需 REDIS_URL;多进程 worker 必须 redis)
|
||||
@@ -113,6 +124,9 @@ PGW_TELEMETRY_BACKEND=none # sqlite | postgres | none(必填)
|
||||
# # 也要能拿原样的请求复现与重放;截断后这两件事都做不成,而既有下游正依赖这一行为。
|
||||
# # 反面同样要看清: 不截断意味着客户合同、标书全文无限期留在 llm_calls 里,
|
||||
# # 多租户下还混在同一张表。真在意留存面的部署应显式设一个上限,并配保留期与访问控制。
|
||||
# # 1.3.5 补充: error_body(网关响应正文摘要)沿用库内 summarize_body 上限,
|
||||
# # 结构化耗尽终态行的 error 说明自带有界限长(不含模型原始正文);
|
||||
# # **两者都不在本键的覆盖面内**,估算留存面时要单独计。
|
||||
# PGW_PRICING_PATH=config/prices.json # 可选: {"<model>": {"input_per_1m": x, "output_per_1m": y}};缺省 cost 恒 None
|
||||
# # 可选第三档 "cached_input_per_1m": z —— 供应商 prompt cache 命中部分的单价;
|
||||
# # 不填即命中部分也按 input 全额计(库不猜折扣率),cost 会偏高
|
||||
|
||||
+114
@@ -1,5 +1,119 @@
|
||||
# Changelog
|
||||
|
||||
## 1.3.7(2026-09-10)
|
||||
|
||||
给 chat 链路加了一条**默认关闭**的长尾对冲(issue #24):一次尝试挂起超过阈值时,并发向**另一个等价源**再发一次,先回者赢、输家取消。另给 `CallStats` 追加三个字段,其中 `generation_ms`(裸生成时间)对四条链路全部生效,与是否开启对冲无关。
|
||||
|
||||
### 关于对冲,请先读这三句
|
||||
|
||||
| # | 承诺 | 展开 |
|
||||
| --- | --- | --- |
|
||||
| 1 | **默认关闭,开启即用配额换延迟** | 不配 `{SCOPE}__HEDGE__AFTER_S` 时行为逐字等于 1.3.6(默认关闭回归门: 既有 unit/contracts 全套件一行断言未改)。开启后触发窗口内 in-flight 翻倍——两路各自走完整准入(配额闸/熔断门/pacer),拿不到配额**静默放弃**、原请求继续等,饱和期不添乱 |
|
||||
| 2 | **输家可能已被上游计费,取消止不住** | 输家落既有取消路径按 est **保留预扣**(闸内保守记账,不是上游真实计费的计量);取消能止住等待,止不住上游已经烧掉的钱。对账靠遥测: 输家 attempt 行 `error="hedge_cancelled"`,与赢家行共享同一 `logical_call_id` 可 join |
|
||||
| 3 | **对冲只对 chat 生效** | `EmbeddingClient`/`OcrClient` 不加对冲参数;它们本版的收益是 `CallStats.generation_ms` 计时(见下) |
|
||||
|
||||
### 触发语义
|
||||
|
||||
| 调用形态 | 触发判据 |
|
||||
| --- | --- |
|
||||
| 流式 | 已过 `hedge_after_s` **且首 token 未至**——慢生成不会被误对冲 |
|
||||
| 非流式 | 已过 `hedge_after_s`,纯总时长阈值。非流式无中途信号,「挂起 vs 慢生成」物理不可分,只能靠阈值取值控制误对冲(建议取源 p50 的数倍) |
|
||||
|
||||
输家取消**不喂熔断/健康分**(挂起 ≠ 源死亡);两路都失败才进既有重试循环且**只计一次重试预算**(对冲是一次尝试的加速形态,不是两次独立尝试;两路皆 429 时按既有规则免预算并退 stall 账,一路 429 一路真失败则计一次不退还)。
|
||||
|
||||
### CallStats 三新字段(全带默认值,1.3.6 的构造方式不炸)
|
||||
|
||||
| 字段 | 口径 |
|
||||
| --- | --- |
|
||||
| `generation_ms: int = 0` | 裸生成时间: 赢家/成功那次 transport 调用的墙钟,**排除** admission 排队、重试退避、对冲触发前等待与清理遥测;与 `total_latency_ms` 的差值即「在等不在生成」的波动开销。结构化重问取最后一轮(覆盖),embedding 为各批 transport 之和(累加),缓存命中恒 0(未产生 transport 调用,0 是实测) |
|
||||
| `hedges: int = 0` | 实际并发发出的对冲路数(触发但准入失败静默不计);未启用对冲恒 0 |
|
||||
| `hedge_won: bool = False` | 赢家是否为对冲路;无对冲恒 False |
|
||||
|
||||
### 与 `call_deadline_s` 的关系
|
||||
|
||||
两者正交、可独立配置: deadline 让长尾**更早失败**,hedge 让调用**更快成功**。组合时装配守卫强制 `hedge_after_s < call_deadline_s`(违反即 `ValueError`);`hedge_after_s ≥ min(源 timeout_s)` 同样装配期报错(对冲永不可能触发);`hedge_after_s ≥ min(已设 ttft_timeout_s)` 与单源 scope 设阈值降为装配期 **warning**(流式档已被看门狗先行切断 / 运行期自然静默)。per-call `chat(call_deadline_s=X)` 使 X < 对冲阈值时该次调用对冲不触发,属合法语义不告警。
|
||||
|
||||
### 配置面
|
||||
|
||||
| 键 | 值域 | 缺省 |
|
||||
| --- | --- | --- |
|
||||
| `{SCOPE}__HEDGE__AFTER_S` | 有限正数秒(复用期限同款值域校验) | 未设 = 关闭 |
|
||||
| `{SCOPE}__HEDGE__MAX_EXTRA` | int ∈ [1,3] | 1;**v1 仅单路对冲生效**,>1 接受但装配期 warning(梯次追加为预留) |
|
||||
|
||||
`GatewayClient(...)` 直传两 keyword-only 参数(`hedge_after_s` / `hedge_max_extra`)走同一份装配守卫;`from_settings`/`from_env` 照常透传。embedding/OCR 的 settings 嵌 `GatewaySettings` 故守卫照常跑,但对冲键对这两条链路不生效。
|
||||
|
||||
## 1.3.6(2026-09-10)
|
||||
|
||||
给一次逻辑调用加了一条**可选**墙钟硬边界(issue #22),并修好取消路径的 TPM 结算与 `Retry-After` 非有限值防御。
|
||||
|
||||
### 关于调用期限,请先读这三句
|
||||
|
||||
| # | 承诺 | 展开 |
|
||||
| --- | --- | --- |
|
||||
| 1 | **期限治理的是「等待」,不是「返回时刻」** | 到期后取消在飞的尝试,但清理(遥测写入、限流结算、缓存收尾)仍在 `finally` 里跑完,**允许超出期限**。实测构造(两次慢遥测写入)里返回时刻达期限的 **5–7 倍**;库只对「等待被切断」给承诺,对「多久返回」不给上界 |
|
||||
| 2 | **到期 ≠ 未产出、≠ 未计费** | 上游可能已经算完并计费,只是结果在返回路上被丢弃(缓存写入慢于期限就是一例)。把 `CallDeadlineExceeded` 当成「这次没花钱」会低估成本 |
|
||||
| 3 | **不配置就是 1.3.5 语义,逐字不变** | `call_deadline_s` 缺省 `None` 时根本不进 `asyncio.timeout` 上下文。故 1.3.5 的两条长等仍在: 纯 429 序列(429 不消耗重试预算)仍可能长时间等待;**有限大的 `Retry-After`(如 3600s)仍照睡**——库有意不用 `backoff_max_s` 去夹它,唯一制约手段就是本版这条期限 |
|
||||
|
||||
### 公共面新增(三项,全为纯新增)
|
||||
|
||||
| # | 位置 | 内容 |
|
||||
| --- | --- | --- |
|
||||
| 1 | `polygateway.CallDeadlineExceeded` | 新异常;带 `scope` / `deadline_s`,**无 `retry_after_s`**(到期不含「何时可再试」,给 `0.0` 会指示下游立刻重打饱和渠道) |
|
||||
| 2 | 配置键 `{SCOPE}__CALL_DEADLINE_S` | 缺省不设 = 不启用;非法值(0/负/`nan`/`inf`/非数)在**装配期**当场 `ValueError` |
|
||||
| 3 | 三个 client 构造参数 + 四个公开方法的 keyword-only 参数 | `GatewayClient` / `EmbeddingClient` / `OcrClient` 的 `call_deadline_s`;`chat` / `embed` / `recognize_text` / `parse_layout` 可 per-call 覆盖(`None` = 继承装配值,**不提供「本次关闭」**)。一次 `embed` 的 N 个批次共享同一份期限,不随批数放大 |
|
||||
|
||||
> [!WARNING]
|
||||
> **`except GatewayUnavailableError` 接不住 `CallDeadlineExceeded`。** 新异常直接继承 `PolyGatewayError`,既不属四分类,也不在 `GatewayUnavailableError` 族内——期限到期是**调用方自己设的边界**,不是网关不可用。只有显式配了期限的调用方才会遇到它,需要处理就单列一条 `except`。遥测侧无需改动: 三个边界既有的 `except PolyGatewayError` 会接住它并照常写一条 `terminal_failure` 行(`error_type='CallDeadlineExceeded'`),**零新增列**。
|
||||
|
||||
### 行为变更:取消路径的 TPM 结算口径
|
||||
|
||||
| 情形 | 1.3.5 | 本版 |
|
||||
| --- | --- | --- |
|
||||
| 取消发生在**端口已开始、结算尚未确定**时 | `settle(0)`,入场预扣整笔退还 | 按 `est` **保留预扣**(方向是宁多扣不空退: 上游可能已计费) |
|
||||
| 结算已确定(含真实 usage 恰为 0 的成功、已判 `SourceDead` 的 `0`) | 按已算出的值 | **一字不变**,取消不覆写 |
|
||||
| 未被四分类接住的异常逃逸(`RuntimeError` 等) | `0` | **仍按 `0`**,本版不扩大语义(已登记为残留) |
|
||||
|
||||
启用期限后库自身会常规性触发取消路径,故这条记账修复与期限同版交付。OCR 的 `settle(0)` 不变——无 token 是事实而非「未知」。非取消路径的最终结算值与 1.3.5 逐字相同,只是算得更早(失败分支的结算决定前移到其第一个 `await` 之前)。
|
||||
|
||||
### 其他
|
||||
|
||||
- `Retry-After: inf` / `1e999`(`float()` 会把它舍成 `inf`)此前会原样进入退避并让该次尝试睡到天荒地老;现按「无提示」处理,退回纯指数退避,并发**一条**带源名与判据词的 warning(不回显原始头,429 风暴下会淹掉真信号)。`nan`、空串、负数、HTTP-date 的既有值语义一字未动。
|
||||
- 限流 Lua、`Permit` 端口签名、遥测 schema、缓存 key 公式、重试预算与退避算法、熔断语义**均未改动**。
|
||||
|
||||
## 1.3.5(2026-09-09)
|
||||
|
||||
把治理单位从「一次尝试」补齐到「一次逻辑调用」(issue #19、#23)。此前重试、换源、结构化重问、embedding 分批都各自独立可见,而「这一次调用总共打了几次、总共花了多久、最后为什么失败」在库外拼不出来;结构化耗尽、embedding/OCR 的无源与准入拒绝更是**一条遥测行都没有**。
|
||||
|
||||
### 公共面四项变更
|
||||
|
||||
| # | 位置 | 变更 | 谁会当场断 |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | `polygateway.CallStats` | 新导出的 frozen dataclass(`logical_call_id` / `attempts` / `total_latency_ms`) | 无(纯新增) |
|
||||
| 2 | `LLMResponse` / `EmbeddingResponse` / `OcrTextResult` / `OcrLayoutResult` | 各追加**末尾**字段 `call_stats: CallStats \| None = None` | 按位置解包这四个类型的代码 |
|
||||
| 3 | `ports.TelemetryRecorder.record_llm_call()` | 新增 **10 个无默认值 keyword-only 参数**(26 → 36 参) | 任何自建 recorder——且**在装配期当场报错**,不再是运行期静默丢行 |
|
||||
| 4 | 遥测表 `llm_calls` | 追加 10 列(INSERT 字段 26 → 36,物理列 27 → 37);新增 `event_kind='terminal_failure'` 行 | 按旧口径计失败调用数的 SQL |
|
||||
|
||||
第 3 条的装配期报错是有意的:`_record` 的 `except Exception` 会把旧签名 recorder 的 `TypeError` 吞成 warning,后果是下游升级后 **100% 丢遥测且调用照常成功**。降级方向的铁律管的是运行期写失败,不是装配错误。`**fields` 形态的 recorder 不受影响。
|
||||
|
||||
### 下游必须做的事
|
||||
|
||||
| 动作 | 说明 |
|
||||
| --- | --- |
|
||||
| **计失败调用改 `WHERE event_kind = 'terminal_failure'`** | 一次逻辑调用恰好一条终态行。`error IS NOT NULL` 跨尝试行与终态行,升级后计数会变大 |
|
||||
| `AVG(latency_ms)` 按 `event_kind` 分组 | 终态行的 `latency_ms` 是整个逻辑调用的总耗时,与单次尝试不同量纲 |
|
||||
| 自建 recorder 补齐 10 参 | 或改成 `**fields`;同时同步自己的 schema 与 INSERT 字段 |
|
||||
| PG manual 档下游补列 | `telemetry_schema_sql("postgres")` 自取;不补则这 10 个维度按现有列裁剪后静默不落库(库发一条点名警告) |
|
||||
| 注意失败行可能带 `http_status_code = 200` | MonkeyOCR 的 `success != true` 就是 200 下的失败,该列不可当成败判据 |
|
||||
|
||||
费用口径**不变**:终态行 `cost IS NULL`、`usage_source='unavailable'`、token 为 0,不参与 `SUM(cost)`。400 密集负载下错误行会翻倍(既有 attempt 错误行 + 新增终态行),这是已批准的下游可见变化。
|
||||
|
||||
### 其他
|
||||
|
||||
- 新列全部可空、无默认值、追加在现有末列之后;**旧行不回填**(NULL 表达「补列之前根本没记过这件事」),滚动升级期新旧进程可写同一张表。
|
||||
- `error_body` 沿用 `summarize_body` 上限,结构化耗尽终态行的 `error` 带有界说明(不含模型原始正文);**两者都不在 `PGW_TELEMETRY_TEXT_CAP` 覆盖面内**。
|
||||
- 修正历史误标:`embed()` 非 200 的失败现在报 `operation="embedding"`。遥测新列 `operation` 由调用点给定,与 `exc.operation` 是两个语义。
|
||||
- 重试预算与退避、429 免预算与 stall 算法、取消结算、限流/熔断语义、缓存 key 公式一字未改。
|
||||
|
||||
## 1.3.4(2026-09-09)
|
||||
|
||||
> [!WARNING]
|
||||
|
||||
@@ -91,7 +91,7 @@ make ci # 只读验证(check + test)
|
||||
| 1 | **更新 README** | 打包会把当时的 README 固化进 sdist,**发布后再改就来不及了**(包里那份永远是旧的)。逐项核对: 安装命令的版本约束(`==1.1.*` 这类**极易漏改**,漏了下游就被锁在旧版)、能力表是否覆盖新行为、数字型断言是否仍成立(如遥测字段数,须用 `inspect.signature` 实测而非凭记忆) |
|
||||
| 2 | CHANGELOG 定版 | "未发布" → `## X.Y.Z(日期)` |
|
||||
| 3 | 版本号 | `pyproject.toml` + `src/polygateway/__init__.py` 两处必须一致 |
|
||||
| 4 | 合并 main + push | `--no-ff`;合并后在 main 上重跑 `make lint` 与全套件,**外加 `pytest -m slow`** ——真实网关 e2e 与 Redis 时间语义变体被 `addopts = "-m 'not slow'"` 默认排除,**不显式跑就等于没跑**(约 20-40 分钟,取决于网关快慢)。它们不进日常提交是有意的: pre-commit 关卡跑全套件,网关一抖就挡住与之无关的提交,久了会把"测试红了先怀疑网关"变成惯性,真 bug 也会被当成抖动重试掉;代价是这道门必须由本清单兜住 |
|
||||
| 4 | 合并 main + push | `--no-ff`;合并后在 main 上重跑 `make lint` 与全套件,**外加与本次 diff 有交集的 slow 子集**(2026-09-10 起): `pytest -m slow` 只跑被本版改动触及的模块——Redis 时间语义变体在动限流/退避/取消时跑,真实网关冒烟在动公开入口时跑;**`test_thinking_live.py` 全模型能力矩阵不再每次发布都跑**(付费且额度耗尽渠道会走完整超时链,一晚数小时无信号),仅在动 `thinking.py`/能力注册表/相关 e2e 设施或人类明确要求时跑,否则复用最近一次有效矩阵证据并如实登记。slow 不进日常提交是有意的: pre-commit 关卡跑全套件,网关一抖就挡住与之无关的提交,久了会把"测试红了先怀疑网关"变成惯性,真 bug 也会被当成抖动重试掉;代价是这道门必须由本清单兜住 |
|
||||
| 5 | **打 tag 并 push** | `git tag -a vX.Y.Z -m "..."` + `git push origin vX.Y.Z`。历史上多个版本漏打 |
|
||||
| 6 | 构建 | `rm -rf dist && python -m build && python -m twine check dist/*` |
|
||||
| 7 | **上传 registry** | 凭据在 `~/.config/tea/config.yml`(tea CLI 的 Gitea token,**不在** `~/.pypirc`);token 走 `TWINE_PASSWORD` 环境变量,不进命令行<br>`TWINE_USERNAME=iomgaa TWINE_PASSWORD=$TOKEN python -m twine upload --repository-url https://gitea.iomgaa.online/api/packages/iomgaa/pypi dist/*` |
|
||||
@@ -115,7 +115,7 @@ Gitea 包 registry 是 **owner 级**(`/iomgaa/-/packages/`)不是仓库级;PyPI
|
||||
- 覆盖率目标 80%;并发/韧性行为是一等测试对象: 重试穿透取消、熔断开路半开、限流结算退款、Redis 掉线降级方向、缓存 key 隔离。
|
||||
- Redis 相关测试用真实 Redis(integration),不 mock Lua 行为;限流契约测试随实现一起交付(参考 CHSAnalyzer `tests/contracts_limiter.py`)。
|
||||
- 涉及真实 LLM 的测试输出结构化 Markdown 至 `tests/outputs/<module>/<test>_<ts>.md`。
|
||||
- **成败取决于外部服务当下状态的测试一律标 `slow`**(`tests/e2e/` 四个文件与 Redis 时间语义变体):它们默认不进日常套件,由发布清单第 4 步统一跑。判据是"重跑一次可能就绿了"——这种测试留在提交关卡里会污染信号。同理,给它们的超时不得紧于 `.env` 的生产配置,否则是设计上就会间歇红。
|
||||
- **成败取决于外部服务当下状态的测试一律标 `slow`**(`tests/e2e/` 四个文件与 Redis 时间语义变体):它们默认不进日常套件,由发布清单第 4 步**按 diff 交集选子集**跑(全量 `pytest -m slow` 仅在交集不清或人类要求时用)。判据是"重跑一次可能就绿了"——这种测试留在提交关卡里会污染信号。同理,给它们的超时不得紧于 `.env` 的生产配置,否则是设计上就会间歇红。
|
||||
|
||||
## 5. 项目结构
|
||||
|
||||
|
||||
@@ -16,11 +16,14 @@
|
||||
| 熔断 | 双通道(连续失败 + 失败率窗口,健康证据抑制误熔);半开单探针带租约(持有者死亡自动回收);epoch fencing 拒绝迟到写回;开路时长指数递增;**开路时当场失败还是等冷却可配**(`CIRCUIT_OPEN`,单源 scope 应配 `wait`) |
|
||||
| 自适应并发 | AIMD:429 削减、成功缓升,防止打爆上游 |
|
||||
| 背压与判死 | 配额满与熔断开路**各自**可选等待或快速失败(`QUOTA_FULL` / `CIRCUIT_OPEN`,两键不可互相替代);等待期按双条件判死(本地非生产性等待与全局无进展**同时**超窗)。stall 窗口只计**非生产性**等待(429 退避/配额轮询/熔断冷却),与 `TIMEOUT_S` 无耦合 |
|
||||
| 调用期限 | 一次逻辑调用可选一条**墙钟硬边界**(`{SCOPE}__CALL_DEADLINE_S` 或 `chat(call_deadline_s=...)`,缺省不启用):治理的是**等待**——重试退避、配额轮询、熔断冷却、结构化重问与 embedding 分批共享同一份期限。三条须知:①**返回时刻 = 期限 + 清理耗时**(遥测/结算/缓存收尾在 `finally` 里跑完,允许超期;实测构造达期限的 5–7 倍),库只承诺切断等待、不给返回上界;②**到期 ≠ 未产出、≠ 未计费**,上游可能已算完并计费;③**不配置即逐字保持 1.3.5 语义**(纯 429 序列仍可能长等、有限大 `Retry-After` 仍照睡)。到期抛 `CallDeadlineExceeded`,**不属四分类、不属 `GatewayUnavailableError` 族** |
|
||||
| 长尾对冲 | **默认关闭**的可选加速(`{SCOPE}__HEDGE__AFTER_S`,仅 chat):一次尝试挂起超阈值时向**异源**并发再发一次,先回者赢、输家取消。流式以「首 token 未至」为触发判据(不误杀慢生成),非流式只有纯总时长阈值;对冲走完整限流/熔断准入,拿不到配额静默放弃。成本须知:开启即用配额换延迟(触发窗口内 in-flight 翻倍),输家取消按 est 保留预扣且**可能已被上游计费**(取消止不住上游);输家不喂熔断,遥测 attempt 行标 `error="hedge_cancelled"` 可按 `logical_call_id` join 对账;两路皆败只计一次重试预算;与 `call_deadline_s` 正交组合(装配守卫强制对冲阈值 < 期限)
|
||||
| 响应缓存 | Redis/内存;key 含 model + messages 摘要 + namespace(缓存隔离单位)+ salt + 采样参数 + 请求级推理档位(同 messages 跑 low 与 max 不互相命中),多模态 content 先摘要再 hash(防毒化);可 per-call 绕过(科研重采样) |
|
||||
| 流式看门狗 | TTFT / inter-token / 总超时三层活性;thinking token 刷活性不计结果;截断流(缺 `[DONE]`)判瞬时不入缓存 |
|
||||
| 推理可观测性 | "这次到底推理没推理"由多信号裁定(推理正文压倒 usage 明细),三态落在 `LLMResponse.thinking_observation`:`observed` / `absent` / `unknown`——**`unknown` 是"本次判不出",不是"没推理"**;本次实发档位与实测观测矛盾时按 `(源, 模型, 生效档位)` 各告警一次(能力表过期、开启未生效、注入了却观测不到;同一模型的 low 与 max 是两个独立的矛盾,不共用节流键);裁定结果随遥测落库 |
|
||||
| 推理档位 | 推理是**八档**(`none`/`auto`/`minimal`/`low`/`medium`/`high`/`xhigh`/`max`)而非开关:源级 `REASONING_EFFORT` + 请求级 `chat(reasoning_effort=...)`,`ENABLE_THINKING` 保留为语法糖;库带 24 条能力表(逐条 evidence 自报实测/文档推定),档位打空**默认报错并给出该模型最省的可用档与该配的键**,要静默映射需显式配 `EFFORT_FALLBACK=nearest`;实发档随 `LLMResponse.applied_effort` 与遥测落库 |
|
||||
| 遥测与成本 | 每次调用(含缓存命中与失败)必录 26 字段;SQLite / Postgres 后端(表已存在时**不需要** schema 建表权限,最小权限账号可直接用);按价格表折算成本落库(注意 `LLMResponse.cost` 本身恒为 `None`,成本只进遥测);多模态内容摘要落库不存原图 |
|
||||
| 遥测与成本 | 每次调用(含缓存命中与失败)必录 36 字段;三类行(`event_kind` = `attempt` / `cache_hit` / `terminal_failure`)加逐源诊断列(`http_status_code` / `error_type` / `cause_type` / `error_body`);SQLite / Postgres 后端(表已存在时**不需要** schema 建表权限,最小权限账号可直接用);按价格表折算成本落库(注意 `LLMResponse.cost` 本身恒为 `None`,成本只进遥测);多模态内容摘要落库不存原图 |
|
||||
| 逻辑调用统计 | 治理单位是**一次逻辑调用**而非一次尝试:四种响应(chat / embedding / OCR 两种)带 `call_stats`(`logical_call_id` / `attempts` / `total_latency_ms`,及裸生成时间 `generation_ms`——赢家那次 transport 调用的墙钟,排除准入排队/退避/触发前等待,与 `total_latency_ms` 的差值即「在等不在生成」——与对冲计数 `hedges` / `hedge_won`),重试、换源、结构化重问、embedding 分批共享同一逻辑 ID;每次**领域失败**另落一条 `terminal_failure` 行,失败调用数从此是一条 `WHERE event_kind = 'terminal_failure'`,详见[1.3.5 逻辑调用统计与失败诊断](#135-逻辑调用统计与失败诊断) |
|
||||
| 遥测的资源与降级 | Postgres 池**闲时占 0 条连接**、忙时上限可配(`PGW_TELEMETRY_PG_POOL_MAX`,缺省 4),每次写入有硬预算(`PGW_TELEMETRY_PG_WRITE_TIMEOUT_S`,缺省 5s);后端不可用是**可恢复的降级**(冷却 60s 后自动重试,DBA 建完表/放开权限即自愈),永久失能只留给 DSN 本身写错;降级状态可编程查询——`client.telemetry_status` 给出 `degraded`/`fatal`/`reason`/`dropped_rows` 等只读快照,不必再靠人工对账。**对账要同时看 `degraded` 与 `dropped_rows`**: 池饱和超预算丢的行走行级丢弃,`degraded` 保持 `False`(后端没挂,是本进程并发超了),只按 `degraded` 告警会看不见这一类丢行——而它恰是 `pool_max` 配小了的唯一信号 |
|
||||
| 调用方维度 | 每次调用可带 `tenant_id`(遥测表的真实列,可挂 RLS、可建复合索引)与 `meta`(≤16 个自定义 KV);四个公共方法全覆盖,校验超限即报错;**库只交付列,不启用 RLS、不建索引** |
|
||||
| 遥测表治理 | `llm_calls` 是**下游的表**:PG 侧缺省**不再自动 `ALTER` 补列**(`PGW_TELEMETRY_SCHEMA_MODE` 三态,不设则 sqlite→auto、postgres→manual),manual 档点名缺列并按现有列裁剪写入;`telemetry_schema_sql(backend)` 自取可粘进迁移文件的建表/补列 SQL;`PGW_TELEMETRY_TEXT_CAP` 限正文长度(**不设 = 存全文**);保留期与访问控制走[生产部署 DDL 模板](#生产部署-ddl-模板postgresql)加 `tools/telemetry_retention.py` |
|
||||
@@ -71,13 +74,50 @@ M8 包括 reasoning_effort、enable_thinking、thinking、thinking_budget、reas
|
||||
|
||||
**本版验收例外(2026-09-09 用户正式批准)**:不再补全模型矩阵;既有失败、UNKNOWN、不可达、缺轮及下游现行配置缺证据如实保留,不改成 PASS。M2 两型的定向成功不代表全模型通过;三项目实际配置迁移仍未核验,合成兼容测试不能代替,缓存迁移操作前置也未被豁免。逐项实测、网络诊断与证据索引见[1.3.4 验证记录](research-wiki/findings/2026-09-09-134-thinking-contracts-validation.md)。
|
||||
|
||||
## 1.3.5 逻辑调用统计与失败诊断
|
||||
|
||||
> [!WARNING]
|
||||
> **升级后失败行会变多,旧的"失败调用数" SQL 会多数。** 每次领域失败除逐次尝试行外另写一条 `terminal_failure` 行;自建 recorder 若未跟进 10 个新参数,会在**装配期**就报错(不是运行期静默丢行)。
|
||||
|
||||
本版把治理单位从"一次尝试"补齐到"一次逻辑调用"。两件事: 四种响应带上 `call_stats`;遥测表补 10 列并新增一类行。
|
||||
|
||||
### 读 `call_stats`
|
||||
|
||||
```python
|
||||
resp = await client.chat([{"role": "user", "content": "hi"}])
|
||||
stats = resp.call_stats # EmbeddingResponse / OcrTextResult / OcrLayoutResult 同名字段
|
||||
stats.logical_call_id # 一次逻辑调用一个 ID:重试/换源/重问/分批共享
|
||||
stats.attempts # 真实打出去的尝试次数(免预算 429 也计;缓存命中为 0)
|
||||
stats.total_latency_ms # 含缓存 IO、退避、准入等待、重问与内联遥测收尾的墙钟
|
||||
```
|
||||
|
||||
`call_stats` 为 `None` 意为**未知**(如第三方合成的响应、或旧缓存条目回放),不得读成 0。失败时异常对象上**不挂任何统计字段**——要归因请查遥测表。
|
||||
|
||||
### 下游 SQL 迁移五项
|
||||
|
||||
| # | 改什么 | 理由 |
|
||||
| --- | --- | --- |
|
||||
| 1 | 计失败调用改成 `WHERE event_kind = 'terminal_failure'` | 一次逻辑调用恰好一条终态行 |
|
||||
| 2 | `error IS NOT NULL` 不再是失败调用的判据 | 它同时命中尝试错误行与终态行,升级后计数变大 |
|
||||
| 3 | `AVG(latency_ms)` 须按 `event_kind` 分组 | 终态行的 `latency_ms` 是**整个逻辑调用**的总耗时,与单次尝试不同量纲 |
|
||||
| 4 | 费用口径**不变** | 终态行 `cost IS NULL`、`usage_source='unavailable'`、token 为 0,不参与 `SUM(cost)` |
|
||||
| 5 | 失败行可能带 `http_status_code = 200` | MonkeyOCR 的 `success != true` 就是 200 下的失败,该列不可当成败判据 |
|
||||
|
||||
归因查询的典型形态: 一条 `WHERE logical_call_id = :lcid` 同时拿到整池终态原因(`terminal_failure` 行的 `error` / `error_type`)与逐源现场(`attempt` 行的 `source_name` / `http_status_code` / `cause_type` / `error_body`)。**终态行的 `http_status_code`/`cause_type`/`error_body` 恒为 NULL**: 拿最后一个源的现场冒充整池归因是错的。
|
||||
|
||||
### 存储侧升级
|
||||
|
||||
新增 10 列全部可空、无默认值、追加在现有末列之后(`scope`、`operation`、`logical_call_id`、`event_kind`、`http_status_code`、`error_type`、`cause_type`、`error_body`、`attempts`、`total_latency_ms`),INSERT 字段 26 → 36、物理列 27 → 37。auto 档自动追加;manual 档一条 DDL 不发、按现有列裁剪写入并点名缺列(补列 SQL 由 `telemetry_schema_sql("postgres")` 自取)。**旧行的新列一律 NULL 且不回填**——NULL 表达的是"补列之前根本没记过这件事";滚动升级期新旧进程可写同一张表。
|
||||
|
||||
自建 `TelemetryRecorder` 的下游请同步补齐这 10 个 keyword-only 无默认值参数(或改成 `**fields` 形态),并同步自己的 schema 与 INSERT 字段。装配期形状闸只能证明形状能被接受,证不了函数体真的落这些列。
|
||||
|
||||
## 安装
|
||||
|
||||
发布在实验室 Gitea PyPI(公开包,匿名可装):
|
||||
|
||||
```bash
|
||||
pip install --extra-index-url https://gitea.iomgaa.online/api/packages/iomgaa/pypi/simple/ \
|
||||
"polygateway[redis,postgres,structured]>=1.3.4,<2"
|
||||
"polygateway[redis,postgres,structured]>=1.3.7,<2"
|
||||
```
|
||||
|
||||
核心仅依赖 `httpx` + `pydantic`;按需选 extras:
|
||||
@@ -147,13 +187,17 @@ vectors = (await embed.embed(["文本 a", "文本 b"])).vectors
|
||||
### 4. 业务侧异常处理
|
||||
|
||||
```python
|
||||
from polygateway import GatewayUnavailableError, RequestRejectedError
|
||||
from polygateway import CallDeadlineExceeded, GatewayUnavailableError, RequestRejectedError
|
||||
|
||||
try:
|
||||
resp = await client.chat(messages)
|
||||
except GatewayUnavailableError as exc:
|
||||
# 整个 scope 暂时无源可用: 延期重投,不消耗业务失败预算
|
||||
schedule_retry(after_s=exc.retry_after_s) # exc.reason / exc.per_source_reasons 供诊断
|
||||
except CallDeadlineExceeded as exc:
|
||||
# 只在自己配了调用期限时出现: 不在 GatewayUnavailableError 族内,上一条接不住;
|
||||
# 且无 retry_after_s(到期不含"何时可再试"),重投时机由业务侧定
|
||||
schedule_retry(after_s=None) # exc.scope / exc.deadline_s 供诊断
|
||||
except RequestRejectedError:
|
||||
... # 请求本身有问题(400/格式拒绝): 不重试,直接失败
|
||||
```
|
||||
@@ -398,6 +442,8 @@ PGW_TELEMETRY_TEXT_CAP=2000 # 落库正文的字符上限;不设 = 存全
|
||||
|
||||
**`PGW_TELEMETRY_TEXT_CAP` 的覆盖面必须说清,否则合规判断会出错。** cap 落在四处:`messages` 里每条消息的字符串 `content`、多模态 content 数组中 `type == "text"` 的 part 的 `text`,以及 `response` 与 `thinking` 两列。消息侧的这个面与缓存摘要函数 `digest_messages` 一致——**只碰 `content`**,消息里别的字段一概不碰。所以调用方自己塞进 `tool_calls.function.arguments`、`name` 等字段的内容**不在覆盖范围内**:开了 cap 不等于表里没有全文残留。另需知道:缺省是**不截断**(存全文),而截断之后遥测不再是可复现重放的证据。
|
||||
|
||||
**1.3.5 新增的两处诊断文本同样不在 cap 覆盖内**:`error_body` 记网关响应正文,沿用库内既有的 `summarize_body` 上限(与 cap 无关,也不随它变化);`error` 在结构化耗尽的终态行上带一段有界说明(修复原因至多 200 字符 + 至多 3 条校验错误 × 200 字符,**不含模型原始正文**),自带独立限长。判断留存面时这两列按各自上限估算,不能算进 cap 的四处。
|
||||
|
||||
### 7. SQLite 侧的保留期
|
||||
|
||||
SQLite 侧**不建议**对着一个大库文件跑 `DELETE` + `VACUUM`,而应**按天/按实验轮转库文件**——`runs/<date>.db`、`runs/<experiment>.db` 这样,到期直接删文件。这是三个现有下游(Video-Tree-TRM5 / CHSAnalyzer / dissect)天然就有的形态,比删行省事也安全得多:删文件是 O(1) 且不可能删错行,而 `VACUUM` 会重写整库、期间需要一倍磁盘空间,还会把并发写入方挡在外面。
|
||||
@@ -423,6 +469,8 @@ SQLite 侧**不建议**对着一个大库文件跑 `DELETE` + `VACUUM`,而应**
|
||||
|
||||
**网关拒绝的理由不会丢失**(1.2.0 起):非 2xx 的响应体经折叠与截断后同时进入异常 message 与 `exc.body_text`,故遥测表的 `error` 列里就能看到网关的原话——不必再为查一次 400 单独埋点。截断保头保尾(总长 2048 字符),JSON 错误体尾部的 `code` / `request_id` 不会被切掉。**经中转部署时请注意**:第三方中转服务自身抖动也会回 400,从状态码上与"你的输入有问题"无法区分;库仍按确定性失败处理(直连供应商时重试只会白烧配额),批处理下游宜据 `body_text` 自备兜底分类。
|
||||
|
||||
**有限大的 `Retry-After` 仍照睡**: 能力表那条「尊重 `Retry-After`」是字面意思——库**有意不用 `backoff_max_s` 去夹服务端给的提示**(夹住就是提前重打已明确说「还没好」的网关),服务端给 3600s 就真睡 3600s;**唯一的制约手段是 1.3.6 的调用期限**(上表「调用期限」行)。自 1.3.6 起,`inf` / `-inf` / `1e999` 这类**非有限**取值按「无提示」处理(退回纯指数退避 + 一条带源名的 warning),不再造成无限等待;`nan`、空串、负数、HTTP-date 的既有语义不变。
|
||||
|
||||
### 哪些异常会到达调用方
|
||||
|
||||
上表的"库内行为"一列描述的是**治理动作**,不是调用方要处理的东西。四类里有两类**根本到不了调用方**——它们被重试循环接住,预算耗尽时统一包成 `AllSourcesExhausted`。这个区分只看类型树和 docstring 是读不出来的,曾让下游据此写错整段设计文档,故在此列明:
|
||||
@@ -433,9 +481,12 @@ SQLite 侧**不建议**对着一个大库文件跑 `DELETE` + `VACUUM`,而应**
|
||||
| `RequestRejectedError` | `SourceDeadError`(立即熔断该源并换源,同上) |
|
||||
| `ResultInvalidError` | |
|
||||
| `SourceNotConfiguredError` | |
|
||||
| `CallDeadlineExceeded`(1.3.6 起) | |
|
||||
|
||||
**`GovernanceBackendError` 属于第一列**: 限流/熔断的状态后端(如 Redis)自身故障时库 fail-closed——一个请求都发不出去,这就是"整个 scope 暂时不可用"。它继承 `GatewayUnavailableError`,所以 §4 那段 `except GatewayUnavailableError` 一条即覆盖完整,无需为它单列分支。`retry_after_s` 默认 5 秒(后端恢复时间不可知,取 0 会让积压任务零延迟冲击已挂掉的后端)。
|
||||
|
||||
**`CallDeadlineExceeded` 既不属四分类、也不属 `GatewayUnavailableError` 族**: 它直接继承 `PolyGatewayError`,表达的是「调用方自己设的墙钟边界到了」而非网关不可用,故**只有显式配了 `call_deadline_s` / `{SCOPE}__CALL_DEADLINE_S` 的调用方才可能遇到它**,不配就永远不会出现。它**无 `retry_after_s`**,且 §4 那段 `except GatewayUnavailableError` **接不住**它——要处理就得单列一条分支。
|
||||
|
||||
**`SourceNotConfiguredError` 有意不在第一列的族内**: 源名不在限流后端的配置字典中是**装配缺陷**而非暂时故障,它应当消耗失败预算、进死信、让人看见——归入可重投家族只会让配置写错的任务永远重投且无人告警。
|
||||
|
||||
## 配置参考
|
||||
|
||||
+2
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "polygateway"
|
||||
version = "1.3.4"
|
||||
version = "1.3.7"
|
||||
description = "PolyGateway:实验室统一的大语言模型(LLM/VLM/OCR)调度与中转库——多源、限流、重试、熔断、缓存、遥测"
|
||||
# registry 包页面的正文只认这一项:缺了页面就是一片空白(1.1.2 的教训,twine 会警告
|
||||
# long_description missing 但不阻塞上传)。README 在打包时被固化进产物,发布后再改无效。
|
||||
@@ -82,6 +82,7 @@ layers = [
|
||||
"polygateway.middleware",
|
||||
"polygateway.transports | polygateway.backends | polygateway.telemetry | polygateway.structured",
|
||||
"polygateway.thinking",
|
||||
"polygateway.deadline",
|
||||
"polygateway.providers : polygateway.sources",
|
||||
"polygateway.ports : polygateway.types : polygateway.errors : polygateway.streaming",
|
||||
]
|
||||
|
||||
@@ -562,9 +562,9 @@ flowchart TB
|
||||
|
||||
### 7.8 遥测与成本
|
||||
|
||||
**必录字段**(继承三项目 15 字段规范;当前 26 个 INSERT 字段,物理表列 27 = 26 + 数据库自填的 `created_at`,两套口径的区分见 `telemetry/schema.py` 模块 docstring): call_id、parent_call_id、session_id、model、provider、source_name、messages(JSON)、response、thinking、prompt_tokens、completion_tokens、usage_source、latency_ms、ttft_ms、max_inter_token_ms、cache_hit、error、**cost**、**cached_prompt_tokens**、**model_reported**、**sampling**、**reasoning_tokens**、**tenant_id**、**meta**、**thinking_observation**、**reasoning_effort**。
|
||||
**必录字段**(继承三项目 15 字段规范;当前 36 个 INSERT 字段,物理表列 37 = 36 + 数据库自填的 `created_at`,两套口径的区分见 `telemetry/schema.py` 模块 docstring): call_id、parent_call_id、session_id、model、provider、source_name、messages(JSON)、response、thinking、prompt_tokens、completion_tokens、usage_source、latency_ms、ttft_ms、max_inter_token_ms、cache_hit、error、**cost**、**cached_prompt_tokens**、**model_reported**、**sampling**、**reasoning_tokens**、**tenant_id**、**meta**、**thinking_observation**、**reasoning_effort**、**scope**、**operation**、**logical_call_id**、**event_kind**、**http_status_code**、**error_type**、**cause_type**、**error_body**、**attempts**、**total_latency_ms**。
|
||||
|
||||
**`sampling` 列(2026-07-31,issue #4,端口 20 → 21)**: 列语义 = 「调用方采样意图 ⊎ 生效源 `extra_body`」的 canonical JSON,空则 NULL。**不含**结构化注入的 `response_format`——列名是采样参数,schema 不是,且数 KB schema 逐行落库会让审计表无谓膨胀。三个 emit 入口口径必须各自定死,否则同一列在不同行含义不同: `emit_attempt`(RetryMW 调用,**唯一**有生效源者)并上 `source.extra_body`;`emit_cache_hit` / `emit_terminal_failure`(TelemetryMW 最外层调用)无 source 可言,只记调用级——与 `model`/`source_name` 在终态行置空是同一先例,且缓存命中行无损(`sampling` 已进缓存 key,能命中即意味调用级参数与历史那次逐字相同)。三者统一读 `request.sampling` 而非 `request.overlay`(后者在 RetryMW 处已被结构化注入污染、在 TelemetryMW 处未被污染,直接用必然三行分叉)。OCR/embedding 路径因决策 G 剥离 `extra_body`,该列恒 NULL。
|
||||
**`sampling` 列(2026-07-31,issue #4,端口 20 → 21)**: 列语义 = 「调用方采样意图 ⊎ 生效源 `extra_body`」的 canonical JSON,空则 NULL。**不含**结构化注入的 `response_format`——列名是采样参数,schema 不是,且数 KB schema 逐行落库会让审计表无谓膨胀。三个 emit 入口口径必须各自定死,否则同一列在不同行含义不同: `emit_attempt`(RetryMW 调用,**唯一**有生效源者)并上 `source.extra_body`;`emit_cache_hit` / `emit_terminal_failure`(前者由 TelemetryMW 最外层调用,后者自 1.3.5 起由三个 client 的公开边界经 `emit_terminal_once` 统一写出)无 source 可言,只记调用级——与 `model`/`source_name` 在终态行置空是同一先例,且缓存命中行无损(`sampling` 已进缓存 key,能命中即意味调用级参数与历史那次逐字相同)。三者统一读 `request.sampling` 而非 `request.overlay`(后者在 RetryMW 处已被结构化注入污染、在 TelemetryMW 处未被污染,直接用必然三行分叉)。OCR/embedding 路径因决策 G 剥离 `extra_body`,该列恒 NULL。
|
||||
|
||||
**`reasoning_tokens` 列(2026-08-11,issue #6,端口 21 → 22)**: 推理 token 已计入 `completion_tokens`,故成本总额一直是对的——这不是计费缺口而是**归因**缺口:缺了它,"这次调用花的钱里有多少花在推理上"无法区分,也就无从判断某个 scope 该不该关推理。供应商不报时记 NULL 而非 0(不可得 ≠ 为零,与 `usage_source='unavailable'` 同一纪律)。
|
||||
|
||||
@@ -581,6 +581,20 @@ flowchart TB
|
||||
|
||||
实际档分析须 `cache_hit=false AND error IS NULL`。embedding/OCR text/layout 的成功与失败尝试由 reasoning_applies=False 保证 NULL,真实 client→emitter→临时 SQLite 与 chat 阳性共同守卫,不能用空行集合证明。生产 emitter 单一出口、端口字段数和 DDL 不变。
|
||||
|
||||
**逻辑调用十列(2026-09-09,issue #19/#23,端口 26 → 36)**: 本版把治理单位从"一次尝试"补齐到"一次逻辑调用"。十列按同一顺序追加在 `reasoning_effort` 之后,全部可空、无默认值、不回填旧行(NULL 表达的是"补列之前根本没记过这件事",与任何哨兵值不同;这一点与 `tenant_id` 故意相反——后者是 RLS 可见性需要哨兵,前者是归因需要区分真实缺口)。
|
||||
|
||||
| 列 | 语义 |
|
||||
| --- | --- |
|
||||
| `scope` | 池名,`TelemetryEmitter` 构造期注入(三个 client 各一行),三类行都带;**不拿 `source_name` 顶替**(终态失败可能根本没选出源) |
|
||||
| `operation` | `chat` / `embed` / `recognize_text` / `parse_layout`,**由调用点给定**。它与 `PolyGatewayError.operation` 是两个语义: 后者是异常自报的出错环节(可为 `download_result` 这类子步骤),链路上任何位置不得读它来填本列 |
|
||||
| `logical_call_id` | 一次逻辑调用一个 ID;上下文缺席(库内现场构造的请求)落 NULL,**不造 ID** |
|
||||
| `event_kind` | 三态: `attempt` / `cache_hit` / `terminal_failure`,三类行的唯一机械判据 |
|
||||
| `http_status_code` / `cause_type` / `error_body` | 只在失败的 `attempt` 行上非空(成功行不统一填 200: 那会让"有状态码"不再等价于"失败了");终态行三列恒 NULL(**C1 红线**) |
|
||||
| `error_type` | 该行自身错误的类名;取消路径传字符串,故为 NULL |
|
||||
| `attempts` / `total_latency_ms` | 只属终态行(同一份冻结快照,与该行 `latency_ms` 同值),其余两类行 NULL |
|
||||
|
||||
两条不变量: **I3** —— 每次领域失败至多一条终态行(`_CallContext.claim_terminal()` 去重;recorder 写失败仍只 warning,故 SQL 可见 ≤ 1);**I4** —— 非领域异常(编程错)**零条**终态行、原样传播、分类不被改写。终态行仍 `cost=NULL`、`usage_source='unavailable'`、token 0,费用聚合口径不变。诊断值的提取收敛在 Emitter 内一个纯 helper(`_error_fields`),只读领域异常的既有属性,**不遍历任意对象、不解析字符串猜诊断**。另立**装配期形状闸**: 旧签名 recorder 在构造 `TelemetryEmitter` 时当场 `ValueError`——降级方向的铁律管的是**运行期写失败**,装配错误放行的后果是下游 100% 丢遥测且调用照常成功。
|
||||
|
||||
**`thinking_observation` 列(2026-08-25,issue #16/#17,端口 24 → 25)**: 落 `LLMResponse.thinking_observation` 的裸取值(`observed` / `absent` / `unknown`,两端均为可空 `TEXT`),语义见 §5.1。它补的是 `reasoning_tokens` 补不上的那一格: 后者为 NULL 时「没推理」与「没上报」不可区分,而供应商停报 `completion_tokens_details` 是会真实发生的事(MiniMax 这一路 2026-08-25 实测已停报,qwen 与 deepseek 在同一网关同一 key 上照常返回),届时按 `reasoning_tokens IS NULL OR = 0` 统计「未推理」会把推理了的调用一并算进去。有了本列,口径改为按本列取值分组,`unknown` 独立成一档而不再被并进「未推理」。
|
||||
|
||||
**recorder 收到的必须是裸 `str` 而非枚举实例**: `TelemetryEmitter` 的 `_AttemptUsage` 内部持 `ThinkingObservation` 类型,`_record` 下沉时取 `.value`。`StrEnum` 虽是 `str` 子类,asyncpg 的参数编码对 `str` 子类不保证接受,而遥测写失败只降级为一条 warning——这类问题不会当场炸,只会让 Postgres 那一路悄悄少一列数据。归一化放在 emitter 侧,与 `tenant_id`/`meta`/`sampling` 由 emitter 定型后再交 recorder 是同一分工(recorder 只落库,不做语义判断)。列序纪律同上: 新列排在最末,两端 DDL 与两份 backfill 同步。
|
||||
@@ -589,7 +603,7 @@ flowchart TB
|
||||
|
||||
**schema 单一事实源、档位与冲突目标(2026-08-19,issue #13,决策见 D15)**: 列序、两端 DDL、两端补列语句、`INSERT` 构造与缺列告警收敛进 `telemetry/schema.py`——此前在两个 recorder 各存一份,而公共函数 `telemetry_schema_sql` 打印给下游的 SQL 必须与库真正执行的 DDL **同源**,三份必然漂移,漂移的表现是"下游照打印的 SQL 建完表,库仍报缺列"。补列自此由 `PGW_TELEMETRY_SCHEMA_MODE` 控制(三态: 不设按后端派生 sqlite→auto / postgres→manual,显式设置两侧均可覆盖): manual 档一条 DDL 都不发,改为按探测到的现有列**裁剪 `INSERT`**(裁剪是关掉 ALTER 的前提,否则缺列旧表每行写入都被拒 = 遥测全失)并发**一条**点名缺列、附可执行 SQL 的 warning;auto 档行为不变,且补列失败时**不裁剪**(该档承诺"把列补上",补不上就让缺列以逐行 warning 暴露)。**库内执行的补列语句与打印给人的那份是两套文本**: 库内不用 `ADD COLUMN IF NOT EXISTS`(它即便列已存在也先取 ACCESS EXCLUSIVE 锁,故库侧一律先探测后 ALTER),打印的那份带,以保证下游可重复执行。同批把 PG 写入的 `ON CONFLICT (call_id) DO NOTHING` 改为**无冲突目标**的 `ON CONFLICT DO NOTHING`: 带目标的语句要求恰好匹配 `(call_id)` 的唯一约束,而 PG 要求分区表的唯一约束必须包含分区键——按 `created_at` 分区(issue #12)后主键变成 `(call_id, created_at)`,该语句被 PG 直接拒收,而写失败只逐行 warning,表现为分区部署下遥测全线静默丢数据;无目标版本在两种表形态上都合法,普通表上语义逐字等价(表上只有主键这一个唯一约束),SQLite 的 `INSERT OR IGNORE` 本就无目标。
|
||||
|
||||
**正文截断(2026-08-19,issue #12)**: `PGW_TELEMETRY_TEXT_CAP` 给落库正文一个可配置的字符上限,**缺省不设 = 不截断**(人类决策 E-a): 截断后的遥测不再是审计证据,也无法拿原样的请求复现与重放,而这正是既有下游在依赖的行为,默认改动即破坏;代价是 issue 那句"无限期保留全部租户全文不应是默认状态"只被解决一半——默认仍是全文,但下游第一次有了不写全文的手段。截断落在 `TelemetryEmitter._record`(全库唯一遥测出口,单一 helper 铁律)内,位于 `digest_messages` 之后、`json.dumps` 之前,作用面四处: 每条消息的字符串 `content`、多模态 part 中 `type == "text"` 的 `text`、`response`、`thinking`;超出部分头部硬切并附 `…(略 N 字)`。**按每条文本切而不是切整串 JSON**——后者会往不做任何校验的 TEXT 列里写进非法 JSON,让此后一切按 JSON 解析该列的分析全废。**且只产出新对象、绝不就地修改**: `digest_messages` 对非 list 的 `content` 原样透传同一个 dict 对象,就地截断会同时污染调用方持有的 messages、后续重试的请求体与缓存写入的 key 且全程无报错——红线由"cap 开与关两态下 `build_cache_key` 输出逐字节相同"的测试钉死。覆盖面须诚实声明: 只碰 `content`(与 `digest_messages` 处理面一致),调用方放进 `tool_calls.function.arguments` 等字段的内容不在其中。embedding 与 OCR 两条链路各自既有的 200 字符上限保留不动,与新 cap 是取更严者的关系。
|
||||
**正文截断(2026-08-19,issue #12)**: `PGW_TELEMETRY_TEXT_CAP` 给落库正文一个可配置的字符上限,**缺省不设 = 不截断**(人类决策 E-a): 截断后的遥测不再是审计证据,也无法拿原样的请求复现与重放,而这正是既有下游在依赖的行为,默认改动即破坏;代价是 issue 那句"无限期保留全部租户全文不应是默认状态"只被解决一半——默认仍是全文,但下游第一次有了不写全文的手段。截断落在 `TelemetryEmitter._record`(全库唯一遥测出口,单一 helper 铁律)内,位于 `digest_messages` 之后、`json.dumps` 之前,作用面四处: 每条消息的字符串 `content`、多模态 part 中 `type == "text"` 的 `text`、`response`、`thinking`;超出部分头部硬切并附 `…(略 N 字)`。**按每条文本切而不是切整串 JSON**——后者会往不做任何校验的 TEXT 列里写进非法 JSON,让此后一切按 JSON 解析该列的分析全废。**且只产出新对象、绝不就地修改**: `digest_messages` 对非 list 的 `content` 原样透传同一个 dict 对象,就地截断会同时污染调用方持有的 messages、后续重试的请求体与缓存写入的 key 且全程无报错——红线由"cap 开与关两态下 `build_cache_key` 输出逐字节相同"的测试钉死。覆盖面须诚实声明: 只碰 `content`(与 `digest_messages` 处理面一致),调用方放进 `tool_calls.function.arguments` 等字段的内容不在其中。embedding 与 OCR 两条链路各自既有的 200 字符上限保留不动,与新 cap 是取更严者的关系。**1.3.5 新增的两处诊断文本同样不在 cap 覆盖内**: `error_body` 沿用 `summarize_body` 的既有上限,结构化耗尽终态行的 `error` 说明自带有界限长(`repair=` 至多 200 字符 + `validation=` 至多 3 条 × 200 字符,**不含 `raw_text`**——模型正文的预算已由 attempt 行的 `response` 列承担)。两段限长常量与 `StructuredMW` 重问反馈共用同一份(`format_bounded_errors`),数值只有一份定义。
|
||||
|
||||
- 后端: `SQLiteRecorder`(默认;WAL + busy_timeout、`INSERT OR IGNORE` 幂等、`asyncio.to_thread` 桥接、初始化/写入失败全降级不冒泡)与 `PostgresRecorder`。
|
||||
- **单一 helper 铁律**: 遥测调用点收敛为一个内部函数/上下文管理器;Video-Tree 与 GovDoc 各有 4-5 处逐字复制的 `record_llm_call(15 个参数)` 是本条的直接教训。
|
||||
|
||||
@@ -68,6 +68,8 @@
|
||||
|
||||
音频端口实现(D10)、SDK transport(openai/anthropic 原生协议,D2 预留)、GLM OCR invoker(D9 预留)、内网 pip index(Q1)、多项目共用 Redis 的 namespace 治理、harness-eval 评估流水线激活。
|
||||
|
||||
**已交付增补(2026-09-10)**: 长尾对冲(issue #24,1.3.7)——chat 单路并发对冲,触发=流式首 token 未至/非流式纯时长阈值,异源走完整准入,默认关闭;`CallStats` 增 `generation_ms`(裸生成时间)/`hedges`/`hedge_won`。设计 designs/2026-09-10-24-hedged-requests-design.md(人类已批准 H1-H8),实施 plans/2026-09-10-24-hedged-requests.md。后续储备:梯次多路对冲(H5 预留)、分位数触发、embedding/OCR 对冲(现不做)。
|
||||
|
||||
## 7. 开放决策依赖
|
||||
|
||||
| 决策 | 阻塞点 | 需拍板时间 |
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
# 1.3.5:逻辑调用统计与结构化失败诊断
|
||||
|
||||
> 状态:独立审查及定向复审通过,**人类已于 2026-09-09 正式批准**(§10 六项批准项全数获批,可据此实施公共 API)。实施计划见 `research-wiki/plans/2026-09-09-135-call-observability.md`。
|
||||
> 日期:2026-09-09。范围:issue #19、#23。基线源码 HEAD e71a623。
|
||||
> 复用现有 Emitter / schema / 三条治理循环,不引入追踪平台、不做 deadline / hedging、不在库内复制上下游关联状态。
|
||||
|
||||
## 1. 问题与边界
|
||||
|
||||
治理单位是一次逻辑调用,但当前 `latency_ms` / `call_id` 只描述**单次尝试**:重试几次、等了多久、缓存有没有参与,SQL 答不出来。诊断侧,transport 抛出的领域错误已带 `status_code` / `operation` / `body_text`(errors.py:52),进 emitter 前却被 `str()` 压平成一列自由文本。终态失败可能没有选中源,但 **scope 始终已知**。
|
||||
|
||||
中转把供应商 529 改写成 503 后,库只能如实记 503,不能猜回 529;未知成本、未知状态一律 NULL。本版不改重试预算、429 语义、stall、推理能力表与取消结算算法,不新增核心依赖。
|
||||
|
||||
## 2. 方案
|
||||
|
||||
| 方案 | 收益 | 代价 |
|
||||
| --- | --- | --- |
|
||||
| A:只在 RetryMW 加两个计数、错误列细分 | 补丁最小 | 漏缓存命中、结构化重问、embedding 分批与三类失败终态,回答不了"整次调用" |
|
||||
| B:每调用局部统计 + 领域异常下沉到单一出口,复用现有遥测行 | 统一边界、字段保真、不默认加成功行、不双计费用 | 增加内部上下文、响应字段与 recorder 字段;须补齐缺失的失败终态行 |
|
||||
| C:新增独立逻辑调用表 / 通用事件端口 | 完整追踪与任意事件分析 | 新存储与运维面,超出当前需求 |
|
||||
|
||||
推荐 **B**。显式传递局部对象,不用 client 共享可变计数、不用模块级 ContextVar;不统一三条治理循环,只统一计数与诊断出口。
|
||||
|
||||
## 3. 逻辑边界与公开统计
|
||||
|
||||
四种响应(`LLMResponse`、`EmbeddingResponse`、`OcrTextResult`、`OcrLayoutResult`)追加 `call_stats: CallStats | None = None`。新增 frozen `CallStats` 并由包根导出——四份平铺字段会立刻漂移。
|
||||
|
||||
| CallStats 字段 | 语义 |
|
||||
| --- | --- |
|
||||
| `logical_call_id: str` | 每次公开调用一个 UUID;重试、重问、分批共享;不占用既有 `parent_call_id` |
|
||||
| `attempts: int` | 准入后实际调用 transport 端口的次数;含免预算 429 与端口本地拒绝;**不是 HTTP 请求条数** |
|
||||
| `total_latency_ms: int` | 从输入校验通过到返回/异常传播前的单调时钟快照;含缓存、等待、重问、分批、内联记账与资源收尾 |
|
||||
|
||||
输入校验异常发生在统计边界之外,保持原行为。第三方合成响应的 `None` 表示未知,不得默认伪造 0。
|
||||
|
||||
- **缓存命中**:`attempts=0`、新 logical ID、本次缓存路径耗时;不回放历史统计。缓存持久化排除 `call_stats`(`_serialize` 显式剔除),`_rehydrate` 显式覆盖为 `None`——`_RESPONSE_FIELDS` 过滤会放行历史 dict,不覆盖就会有 dict 冒充 `CallStats`。缓存 key 白名单不变。
|
||||
- **embed 空输入**(embedding.py:180-190):合法零尝试,返回真实统计(`attempts=0`),**不写任何遥测行**——与 cache_hit 不同,不要按"必录"推断它有台账行。
|
||||
- **OCR layout** 的 POST + ZIP GET 在同一 transport 调用内(monkey_ocr.py:262-296),计 1 次尝试。
|
||||
- **chat 重问 / embedding 多批**计入同一上下文,不重置计数(重问经 `call_next` 重入 RetryMW,已核对)。
|
||||
|
||||
`latency_ms` / `call_id` / `parent_call_id` 语义不变;"总耗时减最后一次尝试耗时"不等于纯等待(含其他本地工作)。不在响应里挂每次尝试的明细列表,避免公共响应无界增长。
|
||||
|
||||
### 3.1 失败与取消
|
||||
|
||||
本版**不向异常对象附加可变 `call_stats`**:第三方可能复用同一异常实例,first-write-wins 会把首次调用的统计误读成本次,覆盖写则串扰;复制任意异常又保证不了构造签名与自定义属性。异常类型与分类原样保留,失败侧的统计走 §6 的终态行。
|
||||
|
||||
- 取消:`CancelledError` 保持原类型与语义,不在其上加字段;取消路径只**尽力**写一条终态,不 shield、不开新后台任务。
|
||||
- 任意内部非领域异常(编程错)原样传播,本版**不承诺**为其提供任何统计或终态行,也不偷偷改分类。
|
||||
|
||||
## 4. 最小内部接缝
|
||||
|
||||
`CallStats` 与私有可变 `_CallContext` 都落 `types.py`:`types.py` 不反向依赖实现层,不产生循环,也不动 import-linter 分层(`ports : types : errors` 并列最内层)。私有上下文只持计数、单调时钟与必要去重状态,不做 I/O。
|
||||
|
||||
`ChatRequest` 追加内部上下文字段(`default=None, compare=False, repr=False`);`StructuredMW` 的 `dataclasses.replace` 保留同一引用(structured.py:96-110,已核对)。响应统计只在公共出口经 `replace` 附加。`GatewayClient` 需自存注入的 `now`(client.py 现未保存),**冻结快照是同步动作,不 await**。
|
||||
|
||||
- 上下文创建/冻结:`GatewayClient` 在公开入口创建、`finally` 冻结;`RetryMW` 只在 transport 调用前登记一次尝试。
|
||||
- **OCR 例外(M1)**:`image` 的类型/空校验在 `_call` 内(ocr.py:238-241)而非公开方法,故上下文在该校验**通过之后**创建,§3 的"校验在边界外"对 OCR 才成立。
|
||||
- `EmbeddingClient` / `OcrClient` 显式把同一上下文传到每批/每次尝试;统计生效与否**不由 telemetry 是否启用决定**。
|
||||
- 每次尝试 ID 仍在当前循环产生,与上下文的 logical ID 一起交给 Emitter。不持久化断点,不引入任务恢复。
|
||||
|
||||
## 5. 诊断字段与归因方式
|
||||
|
||||
保留现有 `error` 字符串供人阅读;Emitter 改为接受**领域异常对象**而非调用方先 `str()`,由单一 helper 提取有限诊断字段。普通超时/网络翻译在 transport 侧保留直接 `__cause__` 类型,空 `str()` 退回类名;**不遍历任意异常对象、不猜测正文**。
|
||||
|
||||
| 新增 INSERT 列 | 值域 / 来源 |
|
||||
| --- | --- |
|
||||
| `scope` | 配置池名,构造期注入 Emitter(见下);不拿 `source_name` 顶替 |
|
||||
| `operation` | **公开方法固定四值**:`chat` / `embed` / `recognize_text` / `parse_layout`;由三个 client 在调用点给定 |
|
||||
| `logical_call_id` | 当前调用上下文的 UUID;上下文缺席(库内现场构造的 `ChatRequest`)→ **NULL,不造 ID** |
|
||||
| `event_kind` | `attempt` / `cache_hit` / `terminal_failure`;旧行 NULL,不回填 |
|
||||
| `http_status_code` | **仅 attempt 行**:失败异常实收状态;无 HTTP 或未知 NULL;成功行不统一填 200 |
|
||||
| `error_type` | 该行自身错误的领域类名;成功行 NULL |
|
||||
| `cause_type` | **仅 attempt 行**:transport 直接捕获的底层异常类名,未知 NULL |
|
||||
| `error_body` | **仅 attempt 行**:既有 `summarize_body` 有界摘要,未知 NULL;不存 `raw_text`、不存全量原文 |
|
||||
| `attempts` / `total_latency_ms` | **仅 terminal_failure 行**填逻辑快照,其余行 NULL;快照在写入前冻结 |
|
||||
|
||||
全部可空,追加到物理列末尾(与旧表 ALTER 追加位置一致,`schema.py` 的 DDL / BACKFILL / COLUMNS 三处同改)。
|
||||
|
||||
**归因方式(C1,决定性)**:`GatewayUnavailableError` 家族从不携带 `status_code` / `body_text`(errors.py:118-166),终态行的 `http_status_code` / `cause_type` / `error_body` 因此**保持 NULL,这是它自身的真实状态**——不把最后一次 attempt 的状态码与正文搬上来伪装成整池诊断(那正是"不拿最后一个源冒充整池归因"的同一条红线)。终态行的 `error_type` 落它自己的类名(`AllSourcesExhausted` / `CircuitOpenError` / …),scope 级 reason 沿用**已有 error 文案**(`str(exc)` 已是 `"{scope} 网关暂时不可用: {reason}"`,不新增列)。逐源现场由同一 `logical_call_id` 的 attempt 行给出。
|
||||
|
||||
```sql
|
||||
-- #19 验收:一次逻辑失败调用的完整现场(终态 + 各次尝试)
|
||||
SELECT event_kind, source_name, http_status_code, error_type, cause_type, error_body, error
|
||||
FROM llm_calls WHERE logical_call_id = :lcid ORDER BY created_at;
|
||||
```
|
||||
|
||||
该查询必须同时给出"整池为何失败"(终态行 error 文案)与"每个源怎么死的"(attempt 行状态码/正文),测试按它断言。因此**本版不再为 `reason` / `per_source_reasons` 扩列**。
|
||||
|
||||
**结构化耗尽的可定位性(C2)**:`ResultInvalidError("结构化输出阶梯耗尽")` 的 message 不含 `validation_errors` / `repair_error`(structured.py:80-86),而该失败发生在 StructuredMW 之上——RetryMW 侧的 attempt 行全是成功行,终态行是唯一记录。做法:**不写 `error_body`**(该列只属 attempt),由 Emitter 的单一 helper 对 `ResultInvalidError` 生成**有界结构化说明**并入现有 `error` 字符串,复用 structured.py 已有的取材口径(至多 3 条、每条 200 字符,与 `_format_errors` 同参数,拼装函数收敛在 helper 一处)。`raw_text` **不重复落库**(它是模型正文,attempt 行的 `response` 列已按 `text_cap` 记过一份;再存一份等于绕过既有正文预算)。该说明可能包含模型输出片段,故遵循与 `error` 现有正文相同的隐私边界,不额外扩大保留范围。
|
||||
|
||||
**operation 的数据源(I1/I2)**:openai_compat.py:169 的 `_status_to_error` 硬编码 `operation="chat"`,而 `embed()` 的非 200 分支(:512)也走它 → 现存所有 embedding HTTP 失败的 `exc.operation` 都是错的。(行号勘误 2026-09-09:本句原写 "(:512、:527)",实测 `:527` 属 `_complete_stream`即流式 chat,`embed` 只有 `:512` 一处;归属以实施计划 §2 表为准,决策未变。)修正:`_status_to_error` 增 keyword `operation`,`embed` 传 **`"embedding"`**(沿用该异常侧既有词表,不改 `chat` / `ocr_text` / `parse` / `download_result` 四值)。新列 `operation` 与 `exc.operation` 是**两个语义**:前者是公开方法,后者是 HTTP 子操作;新列由调用点给定,**绝不读 `exc.operation`**,两者不做自动转换。OCR 两个公开方法各自在调用级给定自己的值。
|
||||
|
||||
**scope 注入(M6)**:`TelemetryEmitter` 现在不知道 scope,`CacheMW` / `TelemetryMW` 自己也拿不到。构造期注入(三个 client 各一行),使 attempt / cache_hit / terminal_failure 三类行都带 scope,避免改三条调用链。model / provider / source 未选出时仍留原空值。
|
||||
|
||||
**成功侧不加承诺**:成功行不承诺 HTTP 状态与错误体;**本版不宣称 SQL 可直接统计所有成功逻辑调用的总耗时**(成功不加终态行)。`error_body` 沿用 `summarize_body` 的既有上限,**不纳入 `PGW_TELEMETRY_TEXT_CAP` 覆盖面**(该键现覆盖四处,详见 §8)。
|
||||
|
||||
## 6. 行语义与终态:只补确实缺失的失败
|
||||
|
||||
保留每次 attempt 与 cache_hit 的既有行,**不为成功新增终态行**。终态行不得复制已有 attempt 的 token 与成本。所有统计边界内的领域失败均尝试写终态,**包括已有 attempt 错误行的直接 RequestRejectedError/ResultInvalidError**;不再沿用“该类错误已录所以外层不录”的旧假设,400 密集负载的错误行可能翻倍,调用失败计数必须只取 terminal_failure。
|
||||
|
||||
**不变量(I3,统一措辞)**:
|
||||
|
||||
| 结束形态 | 终态行数 |
|
||||
| --- | --- |
|
||||
| 以**领域错误**结束的逻辑调用 | 每次调用**尝试写一条**;持久化 best effort(recorder 写失败按既有降级只落 warning),故 SQL 可见行数 ≤ 1 |
|
||||
| 取消 | 三个 client **同策略尽力写一条**,允许 0 条 |
|
||||
| 非领域异常(编程错) | **0 条**,原样传播,本版无统计保证 |
|
||||
|
||||
需要补的路径:chat 结构化耗尽(发生在 transport 成功之后,现无任何失败行)、embedding / OCR 的无源、准入拒绝、重试耗尽与尝试外取消。**取消口径统一(I4)**:chat 现由 TelemetryMW 对任何取消补终态(telemetry.py:487-495,含尝试内取消),embedding / OCR 按同一口径尽力补,避免下游按 `event_kind` 统计取消时拿到路径相关的结果。
|
||||
|
||||
终态与 attempt **不是重复事实**(前者描述逻辑终态,后者描述尝试),用 `event_kind` 区分;**禁止按 `error IS NOT NULL` 跨两类直接计失败调用次数**。chat 现有 TelemetryMW 终态路径收敛到公开边界的单一 helper,避免两处同时写;embedding / OCR 复用该 helper。
|
||||
|
||||
**终态行的请求摘要(M4)**:复用既有 200 字符输入摘要口径,描述**本次调用的整体输入**,但不扩大单行正文预算——embedding 终态取 `<embed texts=N batches=M>` 计数占位 + 第一批(至多 `batch_size` 条、每条 200 字符,与逐批行同款构造);OCR 终态沿用 `<ocr:{kind} image_bytes=…>` 占位,图像 bytes 永不入库。**失败批的具体文本由同 `logical_call_id` 的 attempt 行给出**,终态行不保存全量原输入。
|
||||
|
||||
**错误文本口径(I7)**:OCR 现落 `"类名: msg"`(ocr.py:445-449,按类名归组的既有 metric 口径),chat / embed 落裸 `str(exc)`。改成"Emitter 收异常对象"后,该差异由统一出口的**显式文本策略参数**保留(OCR 保留类名前缀),**不再在三处复制参数列表**。
|
||||
|
||||
**取消时的终态写(残余风险,显式定策)**:该 `await` 本身是新的取消点。策略是**取消优先、不屏蔽**:外部取消落在这一 await 上时,`CancelledError` 照常传播(调用方可能因此看到 `CancelledError` 而非领域错误,与 TelemetryMW 现有行为同款);不 shield、不建新后台任务。冻结统计快照是同步动作,不 await。
|
||||
|
||||
终态行成本 NULL、usage `unavailable`;聚合费用仍只由 attempt / cache_hit 行决定。终态快照在写入前冻结,故不含自身写入耗时;成功响应快照包含其返回前已完成的内联遥测耗时。失败异常不附快照,不为对齐再 UPDATE 旧行。
|
||||
|
||||
取消 attempt 既有字符串 `"cancelled"` 保留:Emitter接收 `PolyGatewayError | str | None`,字符串不解析猜测诊断,error_type/cause_type/http_status_code/error_body均NULL;终态取消同样使用明确取消文案。终态既有latency_ms与新增total_latency_ms取同一冻结快照,避免双时钟微差。
|
||||
|
||||
## 7. recorder 兼容:装配期机械闸(C3)
|
||||
|
||||
`TelemetryEmitter._record` 的 `except Exception` 会把旧 recorder 的 `TypeError` 吞成 warning(telemetry.py:463),后果是自定义 recorder 在下游升级后**100% 丢遥测且调用照常成功**——正是"遥测必录"要防的形态。文档级迁移清单挡不住它。
|
||||
|
||||
机械闸:在 `TelemetryEmitter.__init__`(三个 client 的唯一汇流点,与 `text_cap` 值域校验同处)对 `recorder.record_llm_call` 做**一次** `inspect.signature(...).bind(**<完整新 kwargs 形状>)`,**不执行写入**;含 `**kwargs`(`VAR_KEYWORD`)者自动通过。校验失败 → 装配期抛错。若目标不可 inspect(C 实现等),同样按**配置错误**当场报错,不进入"运行期静默丢行"。
|
||||
|
||||
边界诚实声明:签名 bind 只证明该形状能被接受,**不能证明函数体真的落这些列**;这是一道装配闸,不是行为验证。它既不是被否掉的 `runtime_checkable` 判定,也不是捕 `TypeError` 重试写入。是否扩主 `TelemetryRecorder` Protocol(备选:独立扩展端口保留旧实现)是 §10 的人类批准项;本草案不同时实现两套接口,自带两个 recorder 的 `**fields` 签名可接受新增参数,但其 schema、INSERT 字段与契约测试仍须同步,不能称后端完全不受影响。
|
||||
|
||||
校验所用参数名从 `TelemetryRecorder.record_llm_call` 的协议签名派生,不手抄第四份字段清单;只用哨兵值做bind形状校验,不读取真实请求数据。结构化说明的限条数/限长复用现有常量,若需命名常量则在既有规则所有者中定义并由两个消费者引用,不复制数值。
|
||||
|
||||
## 8. 下游可见变更与文档同步
|
||||
|
||||
**SQL 迁移清单(I6,批准项须按此逐条看)**:
|
||||
|
||||
| 影响面 | 变化 | 下游动作 |
|
||||
| --- | --- | --- |
|
||||
| 失败行数 | 新增 `terminal_failure` 行(每失败调用至多 1) | 计失败调用改 `WHERE event_kind = 'terminal_failure'` |
|
||||
| `error IS NOT NULL` | 同时命中 attempt 与 terminal 两类 | 不再作为"失败调用数"的判据 |
|
||||
| `AVG(latency_ms)` | terminal 行携带**逻辑总耗时**,量级大于单次尝试 | 时延看板一律按 `event_kind` 分组或过滤 |
|
||||
| 费用聚合 | terminal 行 `cost` 恒 NULL、usage `unavailable` | 费用仍只由 attempt / cache_hit 行决定,口径不变 |
|
||||
| 成功侧 | **不加**任何成功汇总行 | 成功逻辑调用总耗时仍从响应 `call_stats` 读,不从 SQL 读 |
|
||||
| `http_status_code` | 失败行上可能是 200(monkey_ocr.py:265-270 的 `success != true` 带 200 上抛) | 该列不可作失败判据 |
|
||||
|
||||
**文档同步(M5,发布前必须同批)**:README:23 的"必录 26 字段"、README:399 与 ARCHITECTURE.md:592 的 `PGW_TELEMETRY_TEXT_CAP` 覆盖面四处枚举(须明确:`error_body` 沿用 `summarize_body` 上限,`error` 保留既有文本口径并仅对新增结构化说明限长;二者**不在 cap 覆盖内**)、ARCHITECTURE.md:565 的必录字段清单与 §7.8 补列一节、`.env.example` 相关注释、CHANGELOG 与 wiki(docs-convention §2)。**字段数与物理列数一律以 `inspect.signature` / `len(COLUMNS)` 实测改写,不凭记忆**(现状:26 个 INSERT 字段 + `created_at` = 27 物理列;本版新增 10 列)。
|
||||
|
||||
## 9. 非功能与测试矩阵
|
||||
|
||||
| 维度 | 要求 |
|
||||
| --- | --- |
|
||||
| 并发 | 每调用独立对象;同一 client 并发不串 logical ID / 计数 / 统计;无全局状态 |
|
||||
| 取消 | 各等待点穿透;`finally` 释放既有资源;终态写取消优先;不新增 shield 与后台任务 |
|
||||
| 降级 | recorder 写失败不改统计与主结果;准入后端仍 fail-closed;收尾失败仍原 warning |
|
||||
| 持久化 | schema 单一事实源;SQLite auto / PG manual 裁剪 INSERT 保持;不 ALTER 默认生产 PG、不改旧列、不回填旧行 |
|
||||
| 幂等 | attempt ID 唯一,终态独立 ID,不重复写同一终态;缓存命中不复制历史统计;不 UPDATE 计费 |
|
||||
|
||||
验收优先离线:真实 client + 内存后端 + FakeClock + MockTransport + 临时 SQLite,复用 1.3.4 设施;不重跑未变的模型能力矩阵。
|
||||
|
||||
| 测试族 | 必须证明 |
|
||||
| --- | --- |
|
||||
| logical 计数 | 一次成功、失败重试、免预算 429、多源拒绝、缓存命中、结构化重问、embedding 多批、OCR 双 HTTP、空输入(0 尝试且 0 遥测行) |
|
||||
| 计时 | 缓存 IO、退避、准入等待、重问、收尾均计入;关闭 recorder 仍正确;毫秒/秒不混用 |
|
||||
| 失败与取消 | 领域失败恰一次尝试写终态;取消三条路径同策略(允许 0 行);**终态写 await 上被取消 → `CancelledError` 传播**;permit / 探针释放不变;非领域异常 0 行且原样传播 |
|
||||
| 保真诊断 | 503 包装不改回 529;直接 529 记 529;空 Connect/Read/Write/PoolTimeout 文案有类型;embedding HTTP 失败的 `exc.operation` 为 `embedding`;新列 `operation` 恒为四值之一且不随异常变化 |
|
||||
| 归因 SQL | §5 那条按 `logical_call_id` 的查询同时给出终态 reason 文案与逐源状态码/正文;终态行三列为 NULL;结构化耗尽的 `error` 含有界 validation/repair 说明且不含 `raw_text` |
|
||||
| 行语义 | 三类行均带 scope;每失败调用至多一条终态;attempt 与 terminal 区分;费用不重复;`AVG(latency_ms)` 按 `event_kind` 分组的断言 |
|
||||
| 装配闸 | 旧签名 recorder → 装配期报错(非 warning);`**kwargs` recorder 通过;不可 inspect → 配置错误 |
|
||||
| 存储兼容 | SQLite 新旧表、PG manual 缺列裁剪 / auto 追加、旧行 NULL、新旧进程混写 |
|
||||
| 变异 | 计数位置、上下文复制、缓存历史回放、提前 `str()` 压平、终态双计费用分别红→绿;PG 真实集成 + 常规全套 + 独立验证 |
|
||||
|
||||
## 10. 集中人类批准项
|
||||
|
||||
| # | 决策 | 推荐 |
|
||||
| --- | --- | --- |
|
||||
| 1 | 公开响应结构 | 一个 `CallStats` 对象而非四类响应各铺三字段;需确认命名与消费便利性 |
|
||||
| 2 | 失败侧统计范围 | 不改/不复制异常;失败统计只落终态行,调用方仅在成功响应读 `call_stats`;需确认该取舍可接受 |
|
||||
| 3 | recorder 兼容路线 | 扩主 `TelemetryRecorder` Protocol + 装配期 bind 闸;备选独立扩展端口保留旧实现;需确认是否存在必须兼容的自定义 recorder |
|
||||
| 4 | 新增失败终态行 | 补漏但不加成功汇总;须批准 §8 表中**全部五项**下游可见变化(不止行数) |
|
||||
| 5 | 取消时终态写取消优先 | 调用方可能看到 `CancelledError` 而非领域错误(同 TelemetryMW 现状);需确认接受 |
|
||||
| 6 | `operation` 值域与异常侧修正 | 新列固定四值;`_status_to_error` 增 `operation` 参数、`embed` 传 `embedding`(修正现存误标,属下游可见的历史数据口径变化) |
|
||||
|
||||
## 11. 独立审查处理表
|
||||
|
||||
| 项 | 结论 | 落点 |
|
||||
| --- | --- | --- |
|
||||
| C1 终态缺诊断来源 | **不采纳"定向读 `__cause__` / `per_source_reasons` 扩列"**;终态保留自身 NULL 状态,归因由 `logical_call_id` 关联 attempt 行 + 既有 reason 文案完成,并写死 SQL 验收 | §5 |
|
||||
| C2 结构化耗尽可定位 | 采纳(变形):有界说明并入现有 `error` 字符串,不写 `error_body`、不重复存 `raw_text` | §5 |
|
||||
| C3 recorder 静默失败 | 采纳:装配期一次 `signature.bind` 形状校验,含 `**kwargs`,不可 inspect 即配置错误;明确不验证函数体 | §7 |
|
||||
| I1 / I2 operation 污染与未归一 | 采纳:新列固定四值由调用点给定,绝不读 `exc.operation`;只修 embedding 误标,其余异常侧词表不动 | §5 |
|
||||
| I3 / I4 终态不变量与取消口径 | 采纳(定稿措辞):领域失败每调用尝试写一条、持久化 best effort;取消三路同策略尽力允许 0;非领域异常 0 条 | §6 |
|
||||
| I5 上下文缺席语义 | 采纳:`None` → NULL,不造 ID | §5 |
|
||||
| I6 迁移影响不止行数 | 采纳:列全五项 SQL 影响并进批准项 | §8、§10 |
|
||||
| I7 OCR error 文本口径 | 采纳:保留类名前缀,由统一出口的显式文本策略参数承载,不复制参数列表 | §6 |
|
||||
| M1 OCR 校验与统计边界 | 采纳:上下文在 `image` 校验通过后创建 | §4 |
|
||||
| M2 embed 空输入 | 采纳:0 尝试且不写任何遥测行 | §3 |
|
||||
| M3 失败行可能带 200 | 采纳:写明该列不可作失败判据 | §8 |
|
||||
| M4 终态请求摘要未定 | 采纳:复用 200 字符口径,终态描述整体输入,不扩预算,不存全量原输入 | §6 |
|
||||
| M5 文档同步缺项 | 采纳:列全六处并要求实测改数字 | §8 |
|
||||
| M6 Emitter 不知 scope | 采纳:构造期注入 | §5 |
|
||||
| 残余风险(终态写成新取消点) | 采纳:显式定策"取消优先不屏蔽",并进测试族与批准项 | §6、§9、§10 |
|
||||
|
||||
自审:方案 B 复用既有 Emitter / schema 与三条循环,不需要 #22 / #24 的新调度。未采纳的两项(异常 first-write-wins、终态搬运最后一次 attempt 的状态与正文)理由已写在正文,不是遗漏。本文档无代码实现与测试通过声明;**§10 六项已于 2026-09-09 获人类批准**,实施边界与红绿证据要求以上述实施计划为准。
|
||||
@@ -0,0 +1,313 @@
|
||||
# 1.3.6:可选调用期限(issue #22),兼 `Retry-After` 有限值防御
|
||||
|
||||
- 状态: **人类已于 2026-09-10 批准**(§9 七项批准项全数获批,H3 取 (a′) 档);实施计划见 `research-wiki/plans/2026-09-10-136-call-deadline.md`
|
||||
- 基线: main `ab00aa4` / 1.3.5,工作区 HEAD `d2455e8`;分支 `feature/1.3.6-call-budgets`
|
||||
- 范围收敛(2026-09-09 人类定夺,2026-09-10 追认): **只做方案 B 的后半**——保留 chat 的 429 免次数预算,新增**可选**整体调用期限,缺省不启用;**不做** 429 次数预算(原 D2/D3)、**不做** issue #24(未批,不得捆绑)
|
||||
- 已批准契约(2026-09-10 逐条定案): `call_deadline_s` 缺省 `None`;单调用参数 `None` = 继承装配值;`CallDeadlineExceeded` 为 `PolyGatewayError` **直接**子类;合作式清理会超出期限、**到期可能丢弃已计费的成功**;chat 429 免次数预算保留原样
|
||||
- 取消结算(TPM)修复**并入本版**: 不再作为「先立独立 issue、修好再启用期限」的前置阻塞,改为本版第一个原子提交(§6.3 矩阵即其精确契约)
|
||||
- 输入: issue #22 原文;设计审查 `a544b789/design136/review.md`(BL1-BL5)与独立复审 B(B1-B5,含离线 asyncio 探针实测);本轮独立探针 `/tmp/pgw_deadline_probe.py`、`/tmp/pgw_deadline_probe2.py`(3.12.13 实测,§5.2);1.3.5 源码逐行现读
|
||||
- 关联: ARCHITECTURE §7.2/§7.3、`designs/2026-08-06-issue8-stall-budget-design.md`、`designs/2026-09-09-135-call-observability-design.md`
|
||||
|
||||
## 1. 目标与非目标
|
||||
|
||||
| 项 | 内容 |
|
||||
| --- | --- |
|
||||
| 目标 1 | 让调用方能对**一次逻辑调用**设墙钟上限;不配置时,库行为逐字保持 1.3.5 |
|
||||
| 目标 2 | 修 `Retry-After` **非有限值**防御缺口(`inf`/`1e999` → `sleep(inf)` 永久挂起) |
|
||||
| 目标 3 | 期限是**单一硬边界**,覆盖缓存 IO/准入排队/退避 sleep/transport/结构化重问/embedding 分批,不是轮首软检查 |
|
||||
| 非目标 A | 不新增 429 次数预算、不改 429 退避指数分账、不收紧任何缺省(原 §4.1/§4.2 已删除) |
|
||||
| 非目标 B | 不改三条循环各自的既有差异(chat 429 免预算 + stall 退还;embed/OCR 无条件计数) |
|
||||
| 非目标 C | ~~本设计不含取消路径 `settle(0)` 结算修复~~ → **2026-09-10 改判**: 该修复**已获批并并入本版**,是期限落地前的第一个原子提交(§6.3 给出精确结算矩阵)。它只改**取消路径**的结算取值,不动成功/拒绝/失败三条既有分支的口径 |
|
||||
| 非目标 D | 不做 issue #24(长尾对冲);不新增遥测列;不新增后台任务/`shield` |
|
||||
|
||||
## 2. 1.3.5 现状核实(现读源码,不引用旧报告)
|
||||
|
||||
| 事实 | 证据 | 对本设计的意义 |
|
||||
| --- | --- | --- |
|
||||
| chat 429 免重试预算并退还 stall 账 | `middleware/retry.py:158-164`、`:250-257` | **保留不动**;期限是正交的第二层 |
|
||||
| embed/OCR 无条件 `fails += 1` | `embedding.py:319`、`ocr.py:339` | **保留不动**;两条循环次数有界、时长无界,由期限兜 |
|
||||
| 退避与 `Retry-After` 取大且不夹上限 | `retry.py:64-77` `max(delay, retry_after)` | 有限大值**照睡**(§6.2 说明为何不夹) |
|
||||
| `_parse_retry_after` 放行 `inf`,但 `nan` **已被忽略** | `transports/openai_compat.py:111-120`:`float()` 成功后 `seconds > 0`;`nan > 0` 恒假 → `nan` 现已返回 `None` | 真实缺口只有 `inf`/`1e999` 一族(唯一一处;`monkey_ocr.py:58` 不解析该头) |
|
||||
| 三个公开边界均在**输入校验之后**建 `_CallContext` | `client.py:381`、`embedding.py:192`、`ocr.py:272` | 期限起点与 `total_latency_ms` 同口径,校验时间不计入 |
|
||||
| 洋葱与两条循环全在同一 `await` 树下 | `client.py:398`、`embedding.py:209`、`ocr.py:274` | 一个 `asyncio.timeout` 即可覆盖全部等待,**无须**改 `backoff_delay`/`SourceAdmission._nap` 签名(解 BL3) |
|
||||
| 终态行唯一出口 + 去重 | `telemetry.py:670-716` `emit_terminal_once` / `types.py:355` `claim_terminal` | 到期终态复用该出口,`error_type` 列自动落新类名,**无新增列** |
|
||||
| 取消路径 attempt 行写 `error="cancelled"` 后穿透 | `retry.py:320-323` | 到期时 attempt 行仍记 cancelled(保留),**终态行**必须记 deadline |
|
||||
| 库内已有 `asyncio.timeout` + `cm.expired()` 范式 | `streaming.py:43-58`、`telemetry/postgres.py:419-423` | 本设计沿用同一范式,不发明新写法 |
|
||||
| `asyncio.timeout` 的**公共契约**: 到期投递取消并在退出时转 `TimeoutError`,外部取消原样上抛,擦边成功不遗留游离取消 | 标准库文档 + 受支持 Python 矩阵上的行为测试(§10"形态区分"批次;复审 B 在 3.12.13 上以离线探针复核) | 外部取消优先与"擦边成功"竞态由运行时判定,库不自判(§5.2);**不依据 CPython 私有实现细节立论**,跨版本保证由测试矩阵给 |
|
||||
|
||||
## 3. 实现范式的三个备选
|
||||
|
||||
| 方案 | 做法 | 判定 |
|
||||
| --- | --- | --- |
|
||||
| **A 单一硬边界(推荐)** | 在三个公开边界各用一次 `asyncio.timeout(d)` 包住整条 `await` 树;到期由运行时取消在途 await,边界处转成 `CallDeadlineExceeded` | 覆盖面天然完整(缓存/准入/sleep/transport/重问/分批);零签名扩散;代价是**到期即取消在途尝试**(§6.3) |
|
||||
| B 轮首软检查 + sleep 夹紧 | 三条循环轮首判剩余预算,退避 `min(delay, 剩余)` | **否决**: 名为期限实为"轮次粒度上限"——一次 300s 的慢尝试或一次准入排队即可整体越限,且要改 `backoff_delay`、`_nap`、三处轮首共 5 个点(BL3 原形态)。用软检查冒充硬期限正是 issue #8 "不要用一个预算冒充另一个预算"的同形错误 |
|
||||
| C 每层各自超时 | 缓存、准入、transport 各配一份超时 | **否决**: N 个键相加才是总时长,调用方仍拿不到"最多等 N 秒"的承诺;配置面爆炸 |
|
||||
|
||||
**推荐 A**,且缺省 `None` = 不启用:opt-in 才能保证存量下游行为逐字不变(1.3.x 内不做默认行为变更)。
|
||||
|
||||
## 4. 方案 A 的具体形态
|
||||
|
||||
### 4.1 新模块 `src/polygateway/deadline.py`(约 50 行,只依赖 stdlib + `errors.py`)
|
||||
|
||||
```python
|
||||
def ensure_call_deadline(value: object, origin: str) -> float | None:
|
||||
"""全装配路径共用的值域校验: None 或有限正数,否则 ValueError(消息含 origin)。"""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
raise ValueError(...) # bool 先判:不得把 True 当成 1 秒
|
||||
v = float(value)
|
||||
if not math.isfinite(v) or v <= 0:
|
||||
raise ValueError(...) # NaN / inf / 0 / 负
|
||||
return v
|
||||
|
||||
|
||||
async def with_call_deadline[T](aw: Awaitable[T], *, deadline_s: float | None, scope: str) -> T:
|
||||
"""给一次逻辑调用施加单一硬边界;None = 不进任何上下文,逐字走旧路径。
|
||||
|
||||
入参已由调用方在**构造 `aw` 之前**校验(见 §4.3),本函数不再校验。
|
||||
"""
|
||||
if deadline_s is None:
|
||||
return await aw
|
||||
inner_timeout: BaseException | None = None
|
||||
try:
|
||||
async with asyncio.timeout(deadline_s) as cm:
|
||||
try:
|
||||
return await aw
|
||||
except TimeoutError as exc:
|
||||
inner_timeout = exc # 体内(含清理路径)自抛,不是本层期限
|
||||
raise
|
||||
except TimeoutError as exc:
|
||||
if cm.expired() and exc is not inner_timeout:
|
||||
raise CallDeadlineExceeded(scope=scope, deadline_s=deadline_s) from None
|
||||
raise # 内层失败原样上抛,绝不贴 deadline 标签
|
||||
```
|
||||
|
||||
**为何不能只用 `cm.expired()`(本轮独立探针实测,3.12.13)**: `Timeout.expired()` 在 `EXPIRING`/`EXPIRED` 两态都返 True(`inspect.getsource` 现读),而计时器触发后的**清理路径**若自抛 `TimeoutError`(第三方端口实现自抛,或其内部另一个 `asyncio.timeout` 到期),该异常会被只看 `expired()` 的写法**改标成 `CallDeadlineExceeded`**(探针 E1/E2 实测均为误标)。`__cause__` 启发式也不够: 内层 `asyncio.timeout` 抛的 `TimeoutError` 其 `__cause__` 同样是 `CancelledError`(E1 实测仍误标)。故采用**局部变量身份比较**这一最小辅助机制: 本层 `asyncio.timeout` 转出的是 `raise TimeoutError from exc_val` 新建对象,与体内那个实例必不同一,判据确定、不依赖任何 CPython 私有实现。它**不新增公共配置、不开后台任务、不改异常对象**;`cm.expired()` 作为第二道守卫保留。
|
||||
|
||||
```text
|
||||
探针证据(/tmp/pgw_deadline_probe2.py, python 3.12.13):
|
||||
D1 到期命中 → CallDeadlineExceeded 耗时 0.050s cancelling=0
|
||||
D2 清理内层 timeout → TimeoutError(原样) 耗时 0.060s ← 只看 expired() 会误标
|
||||
D3 清理裸 TimeoutError→ TimeoutError(原样) 耗时 0.050s ← 只看 expired() 会误标
|
||||
D4 未到期内层自抛 → TimeoutError(原样) 耗时 0.010s
|
||||
D5 到期窗口内领域异常 → 领域异常(期限让位) 耗时 0.200s
|
||||
D6 慢清理 → CallDeadlineExceeded 耗时 0.251s(期限 5×)
|
||||
D7 清理期外部取消 → CancelledError cancelling=1(外部取消优先)
|
||||
D8 外部取消先到 → CancelledError
|
||||
D9 未启用(None) → 逐字旧路径
|
||||
```
|
||||
|
||||
写成**接收 awaitable 的函数**而非 `@asynccontextmanager`: 后者要把 `yield` 包进 timeout,取消经 `athrow` 回注生成器,语义正确但绕(`streaming.py:60-70` 的 docstring 已记录同类陷阱);函数形态只有一条直路。`None` 分支**不进** `asyncio.timeout`,故未启用时连"必须在 Task 内运行"这一新约束都不引入。
|
||||
|
||||
**两个实现约束(实施时不得变形)**:
|
||||
|
||||
1. **校验先于构造 awaitable**: 公开方法入口先跑 `ensure_call_deadline`,通过后才构造 `self._handler(request)` 等协程。若把校验放进 `with_call_deadline`,非法参数抛错时会遗留**未 await 的协程**(RuntimeWarning + 未释放资源)。
|
||||
2. **分层合法**: `deadline.py` 只 import stdlib 与 `errors.py`,并作为**新一层**进 import-linter 契约(`pyproject.toml` layers 中置于 `polygateway.thinking` 之下、内核行之上)。`config.py` 位在更上层,import 它合法且**不会依赖任何具体实现**(transports/backends/telemetry 一律不引入)。
|
||||
|
||||
### 4.2 三处接入点(唯一三处;严禁在循环内层再建 scope)
|
||||
|
||||
| 文件:行 | 现状 | 改后 |
|
||||
| --- | --- | --- |
|
||||
| `client.py:398` | `response = await self._handler(request)` | `await with_call_deadline(self._handler(request), deadline_s=d, scope=self._scope)` |
|
||||
| `embedding.py:209` | `return await self._embed_all(...)` | 同款包住 `_embed_all(...)`(**整次调用一份**,分批共享) |
|
||||
| `ocr.py:274` | `return await self._run(...)` | 同款包住 `_run(...)` |
|
||||
|
||||
三处均在既有 `try` 之内、`_CallContext` 之后,故到期路径照走 `except PolyGatewayError → emit_terminal_once`(§7)。`StructuredMW._run_ladder`(`structured.py:67-98`)与 `_embed_batch`(`embedding.py:295`)**不得**新建 scope:同级重试/重问/分批共享同一期限,否则期限被轮数放大 N 倍即等于没有。
|
||||
|
||||
### 4.3 装配路径与 per-call 覆盖(实际签名)
|
||||
|
||||
| 层 | 签名变化 | 语义 |
|
||||
| --- | --- | --- |
|
||||
| 配置键 | `{SCOPE}__CALL_DEADLINE_S`,经 `_first` 读(**不用** `_require`,否则是破坏性配置变更) | 未设 = `None` = 不启用 |
|
||||
| `GatewaySettings` | 末尾追加 `call_deadline_s: float \| None = None` + `_validate_call_deadline()` 进 `__post_init__`(见 `config.py:186-193`) | 有默认值,不扰动既有位置构造;校验覆盖 env/直接构造/`dataclasses.replace` 三条路 |
|
||||
| 值域校验 | **全装配路径共用** `deadline.ensure_call_deadline`(§4.1):`GatewaySettings.__post_init__`、三个 client `__init__`、四个公开方法各调一次,实现只一份 | 拒收: `bool`(True 不得当 1 秒)、非数值类型、`NaN`、`inf`、`0`、负数 → `ValueError`(消息含来源)。**不与 `timeout_s` 耦合**:期限短于单次超时是合法选择 |
|
||||
| 三个 client `__init__` | 追加 keyword-only `call_deadline_s: float \| None = None`,**入口即校** | 与 `now`/`sleep`/`rng` 同款注入位;全量注入是正式装配路,不得只靠 `GatewaySettings` 守门(否则 `inf` 静默失效、`NaN` 每次调用当场失败) |
|
||||
| 三条 `from_settings` | 传 `settings.call_deadline_s`(embed/OCR 取 `settings.gateway.call_deadline_s`) | `from_env` 无签名变化(经 settings 透传) |
|
||||
| 四个公开方法 | `chat`/`embed`/`recognize_text`/`parse_layout` 追加 keyword-only `call_deadline_s: float \| None = None` | `None` = **继承装配值**;正数 = 本次覆盖;**不提供"本次关闭"**(需要不同期限就装配两个 client;三态哨兵不值这个公共面复杂度) |
|
||||
| 校验时点 | per-call 值在 `_CallContext` 创建**之前**、也在构造被包裹协程之前校验(与既有三项校验同列;OCR 两个入口经 `_call` 两级透传,与 `image` 校验同列) | 输入校验边界保持:非法期限抛裸 `ValueError`,不进统计边界、不写终态行、不遗留未 await 协程 |
|
||||
|
||||
### 4.4 新错误类型
|
||||
|
||||
```python
|
||||
class CallDeadlineExceeded(PolyGatewayError):
|
||||
"""调用方设定的整体期限到期;不是网关不可用、也不是源故障。"""
|
||||
def __init__(self, *, scope: str, deadline_s: float) -> None:
|
||||
super().__init__(f"{scope} 调用期限 {deadline_s}s 到期")
|
||||
self.scope, self.deadline_s = scope, deadline_s
|
||||
```
|
||||
|
||||
| 决策 | 取法 | 理由 |
|
||||
| --- | --- | --- |
|
||||
| 父类 | `PolyGatewayError` **直接**子类 | 不用 `TransientError`: 那是"退避后可重试、计熔断"的源级瞬时故障,下游按它无限外层重试只会把同一份期限再等一遍;不用 `GatewayUnavailableError`: 其消息硬编码"{scope} 网关暂时不可用"(`errors.py:156`),把调用方自选的期限报告成 scope 死亡,正是要避免的"用一个时钟冒充另一个"(BL4) |
|
||||
| `scope` 字段 | **有** | 多 scope 部署时的诊断分组,免得下游从消息串里解析 |
|
||||
| `retry_after_s` 字段 | **无** | 期限到期不含"何时可再试"的信息;给 `0.0` 会按 `errors.py` 既定语义指示下游**立刻重打仍饱和的渠道** |
|
||||
| `SCOPE_REASONS` | **不新增值** | 它不是 `GatewayUnavailableError` 家族成员,与 `reason` 无关 |
|
||||
| 四分类 | 不变 | 它是**调用方策略**的终止信号,不是四分类里的失败;文档须明写 |
|
||||
| 导出 | 进 `__init__.py` 的 `__all__` | 下游要能 `except CallDeadlineExceeded` |
|
||||
|
||||
**醒目**: `except GatewayUnavailableError` / `except AllSourcesExhausted` 的存量代码**接不住**本异常——这是有意设计,且只在显式配置期限后才可能出现。CHANGELOG / wiki / README 必须以此措辞列出。
|
||||
|
||||
## 5. 时钟与失败模式的边界
|
||||
|
||||
### 5.1 两个时钟不混用
|
||||
|
||||
| 时钟 | 用途 | 纪律 |
|
||||
| --- | --- | --- |
|
||||
| 事件循环时钟(`loop.time()`,`asyncio.timeout` 内部) | 期限的**唯一**计时源 | 只传**相对时长** `deadline_s`;严禁把 `_CallContext._started`(注入 `now`)加上偏移当绝对截止时刻传进去 |
|
||||
| 注入 `now`(`types.py:335`、三个 client) | `CallStats.total_latency_ms`、`StallClock`、退避 | 不读、不改;测试替换它不会影响期限判定,这一点必须在测试里明确 |
|
||||
|
||||
代价写实: 两者不同源,故 `total_latency_ms` 与 `deadline_s` 之间存在微小偏差(注入钟被伪造时可任意大)。这是**有意**的——统一它们要么强迫调用方注入 loop 钟,要么自建定时器,两者都比这点偏差贵。
|
||||
|
||||
### 5.2 四种"到期周边形态"必须分开
|
||||
|
||||
| 现象 | 判据 | 结果 |
|
||||
| --- | --- | --- |
|
||||
| 本层期限到期 | `except TimeoutError` 且 `cm.expired()` | `CallDeadlineExceeded`,终态行记 deadline |
|
||||
| 内层自抛 `TimeoutError`(未到期) | `cm.expired()` 为假 | 原样上抛,不吞不改判(同 `streaming.py:52-58`);探针 D4 |
|
||||
| **到期后清理路径自抛 `TimeoutError`** | `cm.expired()` 为真但异常对象**就是体内那一个**(身份比较) | 原样上抛该 `TimeoutError`,**不改标成 deadline**(探针 D2/D3;只看 `expired()` 的写法在此会误标)。代价: 该异常不是 `PolyGatewayError`,三个边界的 `except PolyGatewayError` 接不住→**无终态行**(与 1.3.5 已有的裸 `TimeoutError` 穿透行为同口径,非本版新增) |
|
||||
| 外部取消 | `asyncio.timeout` 契约:不是本层计时器造成的取消 → `CancelledError` 原样上抛 | 走既有 `except asyncio.CancelledError` 分支,终态行记 `"cancelled"`;**不会**同时出现两条终态行(`claim_terminal()` 去重)。探针 D7/D8 实测: 外部取消**无论先于还是晚于到期**(含清理期到达)都胜出 |
|
||||
| **到期窗口内体内先抛领域异常** | 计时器已触发、取消尚未投递到达时,体内先 `raise AllSourcesExhausted(...)` 等 | **领域异常原样逐层上抛,期限静默让位**;终态行记该领域异常而非 deadline。该窗口在重试循环真实存在(一次尝试刚结束与计时器同刻),复审 B 探针 E2/E8 已实测 |
|
||||
|
||||
故本设计只承诺"到期**通常**得 `CallDeadlineExceeded`",**不承诺 100%**;实现与测试均不得写成无条件断言。擦边竞态(计时器已触发但调用体已成功返回)不会遗留游离取消——这是 `asyncio.timeout` 的公共行为,库不自判形态、也不依赖任何 CPython 私有实现;跨版本保证由受支持 Python 矩阵上的行为测试提供(§10)。
|
||||
|
||||
### 5.3 期限治理的是"等待",不是返回时刻(必须写进 wiki,不得含糊)
|
||||
|
||||
asyncio 是**合作式取消**: 到期只是向任务投递一次取消,真正返回的时刻取决于在途 `await` 何时到达取消点,以及**清理路径**跑多久。已知会在期限之后继续跑的三段:
|
||||
|
||||
| 段 | 位置 | 性质 |
|
||||
| --- | --- | --- |
|
||||
| permit 结算与释放 | `retry.py:341` → `admission.py:46-60` | Redis 后端是两次网络往返;不受期限管辖 |
|
||||
| 取消路径的 attempt 遥测行 | `retry.py:320-323` | 一次落库;丢了就丢了本次现场 |
|
||||
| 终态遥测行 | 三个边界的 `emit_terminal_once` | **有意留在期限之外**:诊断行如果自己被期限切掉,期限到期这件事就没有台账 |
|
||||
|
||||
**量级不是毫秒级**: 复审 B 探针 E4 以上述真实形态(取消分支 0.2s 遥测 + `finally` 0.1s 结算)实测:期限 0.05s → 返回时刻 0.351s(**约 7 倍**);本轮独立探针 D6(清理 0.2s)复现同一形态: 0.251s(**5 倍**)。故对外措辞必须是"**返回时刻 = 期限 + 清理耗时**",而不是"最多 N 秒返回";清理耗时取决于 permit/遥测后端,可远超期限本身。§10 以量化断言把越限量变成可观测、可回归的量。
|
||||
|
||||
**反向代价(同等重要)**: 成功之后的旁路 IO **在期限之内**——`middleware/cache.py:199` 的 `_safe_set` 在 `call_next` 返回之后执行,`middleware/telemetry.py:734` 的缓存命中写同理。期限落在这两步 → 一个已完成、**已计费**的 `LLMResponse` 被丢弃,调用方只拿到 `CallDeadlineExceeded`。故 wiki 必须写明"期限到期**不等于**未产出、未计费";本版**不引入 `shield`** 去抢救它。
|
||||
|
||||
取舍是显式的: **宁可超出期限也要留下资源清理与诊断**,而不是引入 `shield`/后台任务去"抢救"(那会把取消语义弄脏,违反"取消可穿透"铁律)。若下游端口实现(自实现 transport / 遥测)在清理里长时间阻塞,期限的超出量就是那段阻塞时长——库不为第三方实现兜底。
|
||||
|
||||
## 6. 与既有算法的关系(明确不改的三件事)
|
||||
|
||||
### 6.1 F1:`Retry-After` 非有限值(纯 bug 修复)
|
||||
|
||||
`_parse_retry_after` 加判据: **解析成功但为无穷**(`inf`/`-inf`/`1e999`,判据 `math.isinf(seconds)`)时显式忽略并记一条 `warning`,按"服务端没给提示"处理,不伪造缺省值。`nan` 维持现状——它被既有的 `seconds > 0` 恒假拦下,**静默 `None` 且不告警**(§2),本版**不给它加告警、不改判据顺序**。
|
||||
|
||||
告警要有主语但不得回显不可信输入,故函数签名改为 `_parse_retry_after(raw: str | None, *, source_name: str) -> float | None`:`source_name` 是**必填 keyword-only 私有参数**(带前导下划线的模块内函数,不属公共面,无需 keyword 默认值兜底),唯一调用处 `_translate_429`(`transports/openai_compat.py:140`)传 `source_name=source.name`。warning 只写源名与判据词(如 `retry_after_not_finite`),**不拼接、不截断、不打印原始头字符串**。其余形态(HTTP-date、空串、负数、不可解析)**维持静默返回 `None`**--HTTP-date 是 RFC 7231 合法形态、空/负是常见噪声,逐次 warning 会在 429 风暴时把真缺陷的信号淡掉。不抛 `RequestRejectedError`:一个坏响应头不该把一次可重试的 429 判死。
|
||||
|
||||
### 6.2 有限大值的 `Retry-After` 不夹上限
|
||||
|
||||
审查 BL1 建议 `min(retry_after, backoff_max_s)`,**本设计不采纳**:夹小的直接后果是提前重打一个明确说了"3600 秒后再来"的饱和渠道,把服务端调度指令改写成库的猜测。有限大值的处置只有一条正路——调用方设期限,由 §4 的硬边界在到期时切断那次 sleep。未配期限即维持 1.3.5 语义(等满 `Retry-After`),这一残余必须在 wiki 明写。
|
||||
|
||||
### 6.3 取消路径结算修复(**已获批,本版第一个原子提交**)
|
||||
|
||||
现状(现读): `retry.py:281 actual = 0` → `:320` 取消分支 → `:341 finally` → `admission.py:46-60 settle_and_release(permit, actual)`;`settle` 算的是 `delta = actual - est`(`backends/memory/limiter.py:48-55`、`backends/redis/limiter.py:136-158`),故 `actual=0` = **把入场预扣的 TPM 整笔退还**。取消发生在 transport 在途时,上游可能已经计费——退款就是把已消耗的额度退回闸里。启用期限后库自己会常规性触发该路径,故先修后启用。
|
||||
|
||||
**精确结算矩阵**(以“这一刻库到底知道什么”为唯一判据;`est` 指 `source.effective_est_tokens()`):
|
||||
|
||||
| # | 取消落点(精确位置) | 库此时知道的事实 | `settle()` 取值 | 本版是否改变 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| S1 | 准入阶段(`admission.pick`:`try_enter` 异常、开路分支) | **确定未调用 transport** | `0`(全额退) | 否(既有行为即此) |
|
||||
| S2 | 退避 `sleep` / 配额轮询 / 熝断等待 | 本轮未持 permit(上一轮已在 `finally` 结清) | 无 permit 可结 | 否 |
|
||||
| S3 | `_attempt` 内、transport 在途(`retry.py:288` 的 await 未返回) | **端口已开始但用量未知** | `est`(delta==0,保留预扣) | **是**——原为 `0` |
|
||||
| S4 | transport 已返回、`actual` 已算出后的任一 await(记账写回/逐次遥测) | **完整 usage 已知**(measured/estimated) | 保留已算出的真实 `actual`,**不得被取消分支覆盖回 `est`** | 否(现行为已正确,修复不得弄坏) |
|
||||
| S5 | **已处理领域失败**分支内的 await(`record_failure`/`_emit`)中途取消 | 已知失败类型,结算决定已算出 | 该分支的决定值:瞬时 = `est`;`SourceDead` = **`0`**;`RequestRejected`/`ResultInvalid` = `0` | 是——决定移到该分支同步处理之后、紧贴首个 await 之前,故 `SourceDead` 的既有 `0` 在取消下**被保住**而不再退化成 `est` |
|
||||
| S6 | OCR 任何位置(`ocr.py:449 settle_and_release(permit, 0)`) | **OCR 无 token 是事实**,不是“未知” | `0` | 否(事实即 0,不得改成 `est`) |
|
||||
| S7 | embedding 与 chat 同构两处(`embedding.py:392-407` 取消分支) | 同 S3/S4 | 同 S3/S4 | **是**(与 chat 同口径同时改) |
|
||||
| S8 | **未被任何 except 接住**的异常(`RuntimeError`、`KeyError` 等未分类逃逸) | 库对用量一无所知,且**不在本版批准范围** | `0`(与 1.3.5 逐字一致) | 否——本版**不**把"端口开始 = 可能已计费"推广到未分类异常 |
|
||||
|
||||
**实现形态**(实施时不得变形;人类只批准了"取消路径"这一条,故语义扩大**必须**被限制在取消分支内):
|
||||
|
||||
`actual` 初值**保持 `0` 不动**;另设一个**局部阶段变量** `settlement_known: bool = False`(纯函数内局部,不是新公共面、不进任何签名、不进配置)。置位规则只有两条——成功路径拿到用量(真实 usage 或 `usage_source == "unavailable"` 的 `est`)后置 `True`;三个**已处理领域失败**分支在**进入分支后的第一条语句**算出结算值并置 `True`。取消分支只在 `settlement_known` 仍为 `False` 时才赋 `actual = est`。
|
||||
|
||||
```python
|
||||
actual = 0
|
||||
settlement_known = False # 局部阶段变量: 该刻库是否已算出确定结算
|
||||
# 成功: actual = 真实 usage 或 est → settlement_known = True
|
||||
# RequestRejected / ResultInvalid: actual = 0; settlement_known = True(分支首句)
|
||||
# SourceDead / Transient: actual = 0 if dead else est; settlement_known = True(分支首句)
|
||||
except asyncio.CancelledError:
|
||||
if not settlement_known: # 端口已开始、结算未定 → 保守保留预扣
|
||||
actual = source.effective_est_tokens()
|
||||
...
|
||||
```
|
||||
|
||||
三条不变量(同级 `except CancelledError` 接不住其他 `except` 块内的取消,故必须在其首个 await 前先确定结算,而非只靠标志位):① **已确定的值一律不覆写**,包括真实 usage 恰为 `0`(源真返回 0 token 是事实,不是"未知");② "确定结算"的界桩按**当前代码里 failure 记账 await 的前后位置**划定——失败分支的结算决定被前移到 `record_failure` / `_emit` **之前**,故取消无论落在这两个 await 的哪一侧,拿到的都是该失败类型本来的值;③ 未被任何 `except` 接住的异常不经上述任一分支,`actual` 保持 `0` 原样传播(S8)。
|
||||
|
||||
**明确收窄**: 本版**不再**把 S5 泛化成"一切失败按 `est` 结算"。1.3.5 的 `SourceDead` 退全款(`0`)是**有意**的语义(源已判死,不该继续占额度),取消恰好落在它之后时必须保留那个 `0`;"端口开始 = 可能已计费"只是**取消且结算未定**这一格的兜底取值,不是全局通则。
|
||||
|
||||
这样只有取消路径的取值发生改变,其余四条既有路径与未分类异常路径逐字不变(验收矩阵 §10 "结算"批次逐条钉,并新增一条**防越界回归**)。
|
||||
|
||||
**保守结算的诚实边界**(不得写成“修对了”): `await transport.complete(...)` 返回前被取消,只能证明**端口协程已被进入**,不能证明 HTTP 字节已发出、更不能证明上游已计费。故 S3 是一个**保守选择**(宁可多扣不可凭空退款),不是事实性计量;它与既有的“瞬时失败按 `est` 结算”(`retry.py:337`)同一口径、同一理由。库**不新增任何 wire 事件协议**(如“transport 上报字节已发出”)来缩小这个不确定区——那是新公共端口面,且 httpx 层面也给不出可靠信号;不确定性写进文档而不是藏起来。
|
||||
|
||||
**不改且不被本版解决的相邻缺口**(列出以免被读成已修): ① `ResultInvalidError` 路径(如 `embedding.py:350-355` 维度不符、`transports/openai_compat.py:280-308` 响应形态异常)——响应真实返回过(已计费)但仍按 `0` 退全款;② `RequestRejectedError` 同理。两者与取消无关,属另一族记账语义变更,**未获批准即不动**,另行立 issue。
|
||||
|
||||
**共享状态不变**: `permit.release()`、`pacer.leave()`、`breaker.release_probe()`(探针归还)、`mark_progress` 全部保持原样且仍在 `finally`;本修复**只改 `settle()` 的入参取值**,不动限流 Lua、不动端口签名、不动幂等语义。
|
||||
|
||||
## 7. 遥测(无新增列,无新增 DDL)
|
||||
|
||||
| 行 | 到期时的取值 |
|
||||
| --- | --- |
|
||||
| attempt 行 | 被取消的那次仍记 `error="cancelled"`(`retry.py:320-323`)——它描述的是**那次尝试**的真实结局,保留 |
|
||||
| 终态行 | 经既有 `emit_terminal_once` 写出;到期**通常** `error_type='CallDeadlineExceeded'`(例外见 §5.2 第 4 行:体内先抛领域异常时记该异常)、`error` 为其消息串(`telemetry.py:274-305` 既有取值逻辑,零改动),`logical_call_id`/`attempts`/`total_latency_ms` 照旧 |
|
||||
| 计数 | 每逻辑调用至多一条终态行,由 `claim_terminal()` 保证;**不会**同时出现 cancelled 与 deadline 两条 |
|
||||
|
||||
## 8. 变更点清单(反 gold-plating)
|
||||
|
||||
| 类别 | 内容 |
|
||||
| --- | --- |
|
||||
| 新增文件 | `src/polygateway/deadline.py`(1 个校验函数 + 1 个包裹函数) |
|
||||
| 改动文件 | `errors.py`(1 个类)、`__init__.py`(1 个导出)、`config.py`(1 个键 + 1 个 loader + 1 个守卫 + 1 个字段,守卫直调 `deadline.ensure_call_deadline`)、`client.py`/`embedding.py`/`ocr.py`(各 1 处包裹 + 构造参数 + 公开方法参数 + 入口校验 + `from_settings` 透传)、`middleware/retry.py` 与 `embedding.py` 的 `_attempt`(§6.3 结算矩阵:只改 `actual` 取值 + 1 个**局部**阶段变量 `settlement_known`)、`transports/openai_compat.py`(F1 一行判据 + warning + `source_name` 必填私有 kw 及其唯一调用处)、`pyproject.toml`(import-linter layers 新增 `polygateway.deadline` 一层) |
|
||||
| 直接复用 | `_CallContext`、`emit_terminal_once`、`claim_terminal`、`asyncio.timeout` + `cm.expired()` 范式、`settle_and_release` 单一出口、既有 FakeClock/假 transport 测试设施、限流契约套件(Lua 不改) |
|
||||
| **明确不做** | 不改 `backoff_delay`/`SourceAdmission._nap`/`StallClock`/429 分账/限流 Lua/端口签名;不改 `ResultInvalid`/`RequestRejected` 的结算口径;不加遥测列;不加 `shield`/后台任务;不加 429 次数键;不做 #24 |
|
||||
|
||||
## 9. 集中人类批准项(2026-09-10 全数获批)
|
||||
|
||||
| # | 决策 | 结果 | 不采纳的代价 |
|
||||
| --- | --- | --- | --- |
|
||||
| H1 | 采纳 §3 方案 A(单一 `asyncio.timeout` 硬边界),缺省 `None` | **已批** | B/C 只能给轮次粒度或多键相加的"伪期限" |
|
||||
| H2 | `CallDeadlineExceeded` 为 `PolyGatewayError` 直接子类、无 `retry_after_s`、不进 `SCOPE_REASONS` | **已批** | 挂进 `GatewayUnavailableError` 会把调用方的期限报告成网关不可用 |
|
||||
| H3 | 取消结算修复的位置 | **已批 (a′)**: 不再另立前置 issue,改为 **1.3.6 内的第一个原子提交**,口径按 §6.3 矩阵(S1-S8,含 S8 未分类异常维持 `0` 的收窄)逐格定死 | 选 (b) = 把已知记账缺口变成常规路径,正是本项目反复吃过的亏 |
|
||||
| H4 | per-call 覆盖形态: §4.3 的"`None` 继承、无单次关闭" | **已批** | 三态哨兵扩大公共面;完全不给 per-call 则长短调用必须装两个 client |
|
||||
| H5 | §6.2 不夹有限大 `Retry-After`(与审查 BL1 建议相反) | **已批**,残余写进 wiki | 夹小即提前重打饱和渠道 |
|
||||
| H6 | §5.3/§11 的对外承诺形态: 期限治理**等待**,返回时刻 = 期限 + 清理耗时(实测可达 5-7 倍),且到期可能丢弃已计费成功 | **已批**,不引入 `shield` | 写成"最多 N 秒返回"会让下游上层超时被整片击穿 |
|
||||
| H7 | issue #22 关闭判据 = "调用方**能配置**上限";#24 保持 open、**本版不实现** | **已批** | 见 §11 |
|
||||
|
||||
实施边界不得再扩: 本表之外的任何公共面变化(新配置键、新端口方法、新遥测列、其它记账口径变更)均属**未批准**,需停下来报。
|
||||
|
||||
## 10. 离线验收矩阵(实施时须先失败后通过;全部不触网、不付费)
|
||||
|
||||
| 批次 | 断言 |
|
||||
| --- | --- |
|
||||
| 未启用回归 | `call_deadline_s=None` 时三条链路行为逐字不变:现有 `tests/unit/test_retry.py`、`test_backpressure.py`、`test_embedding.py`、`test_ocr_client.py`、`test_client.py` 全绿(不改一行断言) |
|
||||
| 期限命中 | 假 transport 真实 `await asyncio.sleep(0.3)`、`call_deadline_s=0.05` → 抛 `CallDeadlineExceeded`,`scope`/`deadline_s` 正确;计时范式沿用 `tests/unit/test_streaming.py` 的真实 loop 时钟 + 4-10× 余量(已验证稳定,不标 slow) |
|
||||
| 覆盖面 | 分别令 ①退避 sleep(注入真 `asyncio.sleep`)②准入排队(配额满轮询)③结构化重问 ④embedding 多批 各自超期 → 均抛 `CallDeadlineExceeded`;embedding 断言 N 批**共享一份**期限(总时长不随批数放大) |
|
||||
| 形态区分 | ①内层自抛 `TimeoutError`(假 transport 直抛)且未到期 → 原样上抛,不变成 deadline;②外部 `task.cancel()` → 仍抛 `CancelledError`,终态行 `error="cancelled"`;③擦边成功(transport 耗时略小于期限)→ 正常返回,无游离取消;④**到期窗口内体内先抛领域异常**(自旋构造确定性窗口)→ 上抛该领域异常、终态行记它,**不断言必为 deadline**;⑤**到期后清理自抛 `TimeoutError`**(假端口在取消分支里 `raise TimeoutError`)→ 原样上抛该 `TimeoutError`,**断言不是 `CallDeadlineExceeded`**(钉住身份比较机制;只看 `cm.expired()` 的写法在此变红) |
|
||||
| 结算(§6.3 矩阵逐格) | S1 准入取消 → `tpm_used` 回到 0;S3 transport 在途取消 → `tpm_used == est`(**先失败后通过**的核心红绿);S4 取消落在 usage 已知之后 → `tpm_used == 真实 usage`(不被 `est` 覆盖);S6 OCR 取消 → `tpm_used == 0`;S5 取消落在 `SourceDead` 的 `record_failure` await 中途 → `tpm_used == 0`(**不得**变成 `est`);**S8 防越界回归**: 假 transport 抛 `RuntimeError`(未分类)→ `tpm_used == 0` 且异常原样上抛;成功/`SourceDead`/`RequestRejected`/`ResultInvalid` 四条既有路径结算值逐字不变;真实 usage 恰为 0 的成功 → `tpm_used == 0`;permit `release`/`pacer.leave`/`release_probe` 调用次数不变 |
|
||||
| 终态遥测 | 到期恰好一条 `event_kind='terminal_failure'` 行,`error_type='CallDeadlineExceeded'`;被取消的 attempt 行仍为 `cancelled`;两者 `logical_call_id` 一致;列数不变 |
|
||||
| 清理不可越过(含量化) | 到期时 permit 的 `settle`/`release` 与 pacer `leave` 仍被调用(假 permit 记账断言);`CancelledError` 未被吞;**量化断言**: 注入已知 sleep 的假 permit/假 emitter → 返回时刻 ≈ 期限 + 已知清理时长(可远大于期限本身) |
|
||||
| 成功旁路被切 | 假缓存后端 `set` 慢于期限 → 抛 `CallDeadlineExceeded` 且断言上游 transport **已成功调用一次**(已计费成功被丢弃,§5.3),把该行为钉住而非留作偶然 |
|
||||
| 注入钟无关 | 伪造注入 `now`(跳变 10^6 秒)不触发期限;反之期限触发时 `total_latency_ms` 仍来自注入钟 |
|
||||
| 配置(全装配路径) | 键未设 → `None`;`0`/负/`nan`/`inf`/非数/`True`(bool 不得当 1 秒)→ `ValueError` 且消息含来源——**四条路各测一遍**: env、`GatewaySettings` 直接构造、`dataclasses.replace`、**三个 client `__init__` 直传**;per-call 非法值在构造协程前抛错且**无 "coroutine was never awaited" 警告**;`call_deadline_s < timeout_s` **不报错**(允许) |
|
||||
| F1 | `Retry-After` 为 `inf`/`-inf`/`1e999` → 按无提示处理且各记一条 warning(**断言日志含源名、不含原始字符串**);`nan`/空/负/HTTP-date → 静默 `None` 且**无 warning**;`_parse_retry_after` 的 `source_name` 为必填 kw(漏传即 `TypeError`);有限正数仍取大;`insufficient_quota` 仍归 `SourceDead` |
|
||||
|
||||
## 11. issue 闭环判据
|
||||
|
||||
| issue | 判据 | 措辞纪律 |
|
||||
| --- | --- | --- |
|
||||
| #22 | **可关闭**: 调用方通过 `{SCOPE}__CALL_DEADLINE_S` 或 per-call 参数即可给一次调用设上限 | 措辞必须是"**期限治理的是等待,不是返回时刻**;返回时刻 = 期限 + 清理耗时(取决于 permit/遥测后端,实测可达数倍)",不得写成"最多等 N 秒"的硬保证;同时写明"到期不等于未产出、未计费"(§5.3)与"**不配置就保持 1.3.5 旧语义**"(纯 429 序列仍可能长时间等待、有限大 `Retry-After` 仍照睡)——三句均需出现在 CHANGELOG、wiki 与 issue 关闭说明的显要位置 |
|
||||
| #24 | **保持 open、本版不实现**(未获批准),另行设计 | 期限只让长尾**更早失败**,与"更快成功"是两件事;任何文档不得把前者写成后者,也不得因本版而声称 #24 缓解 |
|
||||
|
||||
## 12. 待补证据与残余风险(诚实标注)
|
||||
|
||||
| 项 | 状态 |
|
||||
| --- | --- |
|
||||
| 取消是否让上游停止生成与计费 | 无一手证据 → §6.3 的 S3 只能是**保守选择**而非事实计量;不做任何"取消即省钱"或"已修准"的表述 |
|
||||
| 保守结算的反向偏差 | S3 在“transport 确实未发出字节”时会**多扣** `est`(直到窗口滞后自然过期);与“凭空退款击穿网关”相比这是有意选定的方向(降级方向铁律),但必须写进 CHANGELOG |
|
||||
| 未分类异常的结算 | `RuntimeError` 等未被四分类接住的异常仍按 `0` 退全款(S8)——人类只批准了取消路径,本版**不**将保守口径扩到它们;它们理论上同样可能发生在 transport 在途之后,属已知残留,需时另立 issue |
|
||||
| `ResultInvalid`/`RequestRejected` 的退全款 | 本版**不改**(§6.3 末段);它们与取消无关,属未批准的另一族记账语义变更 |
|
||||
| 跨 Python 版本的取消/超时语义 | 本设计只依赖 `asyncio.timeout` 的**公共行为**与异常对象身份比较,不引 CPython 私有实现作保证;保证由 §10"形态区分"在受支持 Python 矩阵上的测试提供(本轮探针仅覆盖 3.12.13) |
|
||||
| 清理自抛 `TimeoutError` 时无终态行 | 该异常不是 `PolyGatewayError`,三个边界的 `except` 接不住(§5.2);与 1.3.5 已有的裸 `TimeoutError` 穿透同口径,本版不扩大也不修补 |
|
||||
| #22 现场 46.7s/20min 数字 | 未复跑;本设计不依赖其数值,只依赖路径成立性(已由源码证明) |
|
||||
| 第三方端口实现的清理耗时 | 不可控,直接构成期限超出量(§5.3) |
|
||||
| 期限与 `stall_window_s` 的联合调参建议 | 本版不给推荐值:两者治理对象不同(调用方意志 vs scope 活性),给一个"经验公式"就是把两个预算再次绑死 |
|
||||
@@ -0,0 +1,189 @@
|
||||
# 长尾对冲请求(issue #24)设计
|
||||
|
||||
- 状态: **已批准**(2026-09-10 人类批准 §9 全部批准项 H1–H7,含增补 H8:`CallStats` 增 `generation_ms`/`hedge_won` 裸生成时间字段);本文件只做设计与权衡,不含实现
|
||||
- 基线: main `166b286` / 1.3.6(已发布,含 `call_deadline_s` 与取消结算修复)
|
||||
- 输入: issue #24 原文(非流式挂起后正常 200:20 次里 5 次超 60s、中位 15.1s、真实负载 13% 调用吃掉 71% 模型总时间、慢调用输出中位 186 token——在等不在生成、90–96s 窄峰疑似源侧固定机制)
|
||||
- 关联: `designs/2026-09-09-136-call-budgets-design.md`(期限与取消结算,本设计直接站在其 S3 格上);ARCH §6.4 取消语义、§7.2 重试、§7.3 限流;`designs/2026-08-06-issue8-stall-budget-design.md`
|
||||
|
||||
## 1. 目标与非目标
|
||||
|
||||
| 项 | 内容 |
|
||||
| --- | --- |
|
||||
| 目标 1 | 非流式请求挂起超过阈值时,并发向**另一个等价源**再发一次,先回者赢,输家取消——把 p99 从"挂起时长"压到"阈值 + 健康源耗时" |
|
||||
| 目标 2 | 流式请求以 **TTFT 未至**为触发判据(不误杀慢生成);非流式无 TTFT 可观测,用总时长阈值 |
|
||||
| 目标 3 | 默认关闭;开启后的一切行为(配额、熔断、遥测、结算)可观测、可对账 |
|
||||
| 非目标 A | embedding / OCR 不做对冲(无 TTFT 概念、issue 未涉、无配置面) |
|
||||
| 非目标 B | 不做滚动分位数触发(`AFTER_PERCENTILE`);不做同源对冲;不加遥测新列(默认档) |
|
||||
| 非目标 C | 不改 `call_deadline_s`/`deadline.py`/限流 Lua/429 分账;不改既有四分类 |
|
||||
|
||||
## 2. 现状核实(现读 1.3.6 源码,不引用旧报告)
|
||||
|
||||
| 事实 | 证据 | 对本设计的意义 |
|
||||
| --- | --- | --- |
|
||||
| 非流式路径是单 JSON 响应,**仅 total 超时**,无中途进度信号 | `transports/openai_compat.py:646-654`(`_complete_once` docstring 原文)、`:684 ttft_ms=None` | 非流式的对冲触发**物理上只有总时长阈值**一种;"挂起 vs 慢生成"在非流式不可分,只能靠阈值取值与成本上限控制误对冲 |
|
||||
| 流式首 token 观测点已存在 | `openai_compat.py:569-575`(`ttft_ms` 首次赋值处) | TTFT 事件信号只需在该点 `event.set()`,探测成本近零 |
|
||||
| `Transport.complete` 是公共端口签名,库内唯一实现 | `ports.py:51-60` | 加首 token 事件参数 = **公共端口签名变更**,必须进批准项(H2) |
|
||||
| 每次尝试 = 选源 → 熔断门 → 限流 permit → transport,全在 `_attempt` 内 | `middleware/retry.py:276-353`;准入编排 `middleware/admission.py:171-213`(`pick`) | 对冲 = **并发跑第二次 `_attempt`**,配额/熔断/结算/pacer 全部复用,无需发明第二套准入 |
|
||||
| 取消路径结算: `settlement_known=False` 时保留预扣 est | `retry.py:327-331`(1.3.6 S3 格);finally 结算 `:352-353` → `admission.py:46-60` | 对冲输家走取消路径,**结算语义现成**:额外成本上限 = 一份 est 预扣滞留 |
|
||||
| 取消的 attempt 行记 `error="cancelled"` 后穿透 | `retry.py:332-334` | 输家遥测只需换一个区分字符串,零新列(§4.5) |
|
||||
| `_CallContext` 承诺"每调用一个实例的**单任务**对象,计数无需锁" | `types.py:321-337`;`register_attempt` `:339-345`、`claim_terminal` `:354-360` 均为无 await 同步方法 | 对冲引入第二个并发任务,该 docstring 承诺须修订;同步方法在事件循环内天然任务安全(无 await 间隙),机制零改动 |
|
||||
| 成功响应在**返回前**冻结 `call_stats` 快照 | `client.py:442`(`dataclasses.replace(response, call_stats=context.snapshot())`) | 输家取消收口必须**先于**快照,否则 `attempts` 漏计输家(§4.5) |
|
||||
| 期限包整棵树,缺省 None 不进上下文 | `deadline.py:64-86`;接入点 `client.py:419-421`;配置 `config.py:190`、`:695-715` | deadline 与 hedge 正交组合,`deadline.py` 零改动(§6) |
|
||||
| 429 免重试预算且耗时退 stall 账 | `retry.py:158-164`、`:250-257` | 对冲轮内某任务 429 的免预算语义沿用 attempt 级既有机制(§4.6) |
|
||||
| 配置键两段/三段式天然跳过 `_load_sources`;保留段防撞名 | `config.py:401-407`(len==4 判定)、`:57`(`_RESERVED_SEGMENTS`) | 新键 `{SCOPE}__HEDGE__AFTER_S` 为 3 段,天然不被当源字段;`HEDGE` 须加进保留段(§5) |
|
||||
|
||||
## 3. 备选方案与权衡
|
||||
|
||||
| 维度 | **方案 A:并发对冲(推荐)** | 方案 B:取消式投机重试 | 方案 C:仅流式对冲,非流式只靠 deadline |
|
||||
| --- | --- | --- | --- |
|
||||
| 做法 | 阈值到 → 并发向异源发第二次 `_attempt`,`asyncio.wait(FIRST_COMPLETED)`,赢家返回、输家 `cancel()` 并 await 收口 | 阈值到 → 取消在途 attempt,按可重试失败走既有换源重试循环 | 只对 `stream=True` 做 TTFT 对冲;非流式维持 1.3.6 现状(期限切长尾) |
|
||||
| 挂起请求的信号处理 | 输家只是"被取消",**不喂熔断/健康分**——挂起的请求最终正常 200,记 failure 是错误信号(源没坏,是这一跳排队) | 必须新造一类"挂起失败":复用 Transient 会把未死源喂进熔断失败计数(`retry.py:340/346-348`),污染熔断与健康分;新造免预算类别 = 又一类四分类外特例 | 同 A(但只覆盖流式) |
|
||||
| 重试预算 | 对冲不消耗 `max_attempts`——它是"一次尝试的加速形态" | 消耗预算(3 次挂起即 `AllSourcesExhausted`),除非新造免预算类 | 同 A |
|
||||
| 尾部赢面 | 原请求"后发先至"时仍可用其成果;尾部的尾部 = min(两路) | 原请求成果恒被丢弃;延迟恒 = 阈值 + 重试耗时 | 流式同 A;非流式尾部 = 期限(更晚失败,不是更快成功) |
|
||||
| 成本 | 对冲窗口内两路并发,输家可能被上游计费 + 一份 est 预扣滞留 | 取消更早(阈值即取消),已计费浪费**更少** | 最低(覆盖面也最小) |
|
||||
| 实现量 | 大:对冲编排 + 端口加参 + 并发收口 + 遥测区分 | 约为 A 的 1/3:阈值计时器 + 取消 + 失败归类 | 中:同 A 但免非流式分支 |
|
||||
| 解决 issue 现场 | 是(issue 复现即非流式) | 是 | **否**——issue 的现场就是非流式,等于没解决 |
|
||||
|
||||
**关键判断**: B 的性价比看似更高(issue 数据显示挂起峰在 90s+,原请求几乎不可能后发先至),但它要回答一个 A 不用回答的问题——"挂起中的源该不该记失败"。记,则熔断/健康分被一次排队事件污染(双峰窄峰指向源侧固定机制,不是源死亡);不记,则要在四分类外新造语义。A 让输家落进 1.3.6 已有的取消路径,**零新分类语义**,且对冲拿不到配额时自然静默(饱和期不添乱)。C 不解决原问题,仅列为范围收缩的退路。
|
||||
|
||||
**推荐 A**;B 作为"预算敏感且接受熔断语义代价"的降级备选保留在批准项(H1)中由人类定夺。
|
||||
|
||||
## 4. 方案 A 的具体形态
|
||||
|
||||
### 4.1 触发条件(设计问题 1)
|
||||
|
||||
| 调用形态 | 触发判据 | 机制 |
|
||||
| --- | --- | --- |
|
||||
| 流式 | 已过 `hedge_after_s` **且首 token 事件未置位** | `Transport.complete` 加 keyword-only 参数 `first_token_event: asyncio.Event \| None`(必填,不设默认值,与端口既有约定同款);`OpenAICompatTransport` 在 `openai_compat.py:573-575` 首 token 处 `set()`。阈值计时器 = 等待该事件,超时即触发 |
|
||||
| 非流式 | 已过 `hedge_after_s`(纯总时长阈值) | `_complete_once` 物理上无中途信号(`openai_compat.py:646-654`),事件**永不置位**直到完成——同一套"等事件超时"机制自然退化为时间阈值,**零分支** |
|
||||
| 分位数触发 | **v1 不做** | 滚动分位数需要 per-source 状态窗口,跨进程部署还得进 Redis;issue 的双峰形态(主峰 0–10s vs 挂起峰 90s+)用绝对阈值区分度已足够。保留为未来扩展 |
|
||||
|
||||
- "非流式不对冲只做 deadline"已被方案 C 覆盖并否决(不解决 issue 现场);但**配置层面允许只对流式生效**——`stream=False` 的调用方若不接受误对冲成本,可不配阈值。
|
||||
- 误对冲的代价有界:最多 `hedge_max_extra` 次额外请求/逻辑调用,输家记账见 §4.4。
|
||||
- 时钟纪律同 deadline(136 设计 §5.1):只用**相对时长 + 事件循环钟**,不读注入 `now`——测试伪造注入钟跳变不得触发对冲,验收矩阵钉住。
|
||||
|
||||
### 4.2 对冲目标 = 异源(设计问题 2)
|
||||
|
||||
| 决策 | 取法 | 理由 |
|
||||
| --- | --- | --- |
|
||||
| 同源 or 异源 | **异源,且仅异源** | issue 观测的 90–96s 固定窗口窄峰指向源侧机制;同源对冲 = 给同一队列再排一个号,徒增成本 |
|
||||
| 等价源定义 | 同 scope 内 `SourceAdmission.pick` 正常排序选出的下一个候选——等价性由 **scope 语义**承诺(同 scope 源本就可互换,同 model 集合是常态),对冲层不发明新的等价概念 | 复用既有选源排序、冷却备忘、调用内降权(`admission.py:63-127`),不新建"等价类"配置维度 |
|
||||
| 排除当前源 | `pick` 加**私有**排除参数(如 `exclude: frozenset[str]`);对冲任务以在途源名为排除集 | 改动收敛在 middleware 内部,不碰公共端口 |
|
||||
| 无候选可用 | 单源 scope / 其余源全冷却、开路、配额满 → **放弃本次对冲**,继续等原请求 | 对冲是优化不是权利;单源 scope 配了阈值 = 装配期 warning、运行期自然静默(§5) |
|
||||
| 同源对冲开关 | 不做(YAGNI) | 配置面少一个维度;真出现"源内分片排队"形态再立 issue |
|
||||
|
||||
### 4.3 准入不独立:对冲走完整准入(设计问题 3)
|
||||
|
||||
对冲请求**照常走** QuotaGate + BreakerGate + pacer + 冷却备忘(`admission.py:171-213` 的完整 `pick` 路径),**不给旁路**:
|
||||
|
||||
| 情形 | 行为 | 对齐 |
|
||||
| --- | --- | --- |
|
||||
| 配额满 / 被熔断 / pacer 超限 | 放弃本次对冲,原请求继续等(不抛错、不排队硬等) | 对冲若绕闸,源挂起风暴时并发翻倍打进正在排队的网关——正是限流铁律要防的击穿;拿不到配额时自然静默,饱和期不添乱 |
|
||||
| 限流/熔断后端不可用 | `try_acquire`/`try_enter` 抛 `GovernanceBackendError`,照常冒泡 | 铁律"后端不可用 → 报错而非放行",对冲分支不新增降级面 |
|
||||
| permit 持有 | 赢家输家各持各的 permit,各自 `finally` 结算释放(`retry.py:352-353`) | 与两个独立并发调用完全同构,限流契约零改动 |
|
||||
|
||||
### 4.4 成本与取消记账(设计问题 4)
|
||||
|
||||
| 角色 | 结算 | 依据 |
|
||||
| --- | --- | --- |
|
||||
| 赢家(先成功) | 正常成功路径:`settle(实际 usage)`;`usage_source="unavailable"` 时按 est | `retry.py:300-308` 既有分支,零改动 |
|
||||
| 输家(被取消) | 落 1.3.6 取消 S3 格:transport 在途、结算未定 → `settle(est)`(**保留预扣,不退款**) | `retry.py:327-331`;上游可能已对输家计费,est 保留是保守下限——与期限到期同口径,文档明写"对冲掉的那次可能已计费" |
|
||||
| 输家(取消前已真失败) | 走既有失败分支结算(dead=0/瞬时=est) | `retry.py:346-347`,取消落点决定取值,S5 机制已覆盖 |
|
||||
| 对冲轮内 429 | attempt 级免预算 + stall 退还照既有机制 | `retry.py:158-164` |
|
||||
|
||||
**计费对账口径**:一次逻辑调用对冲一次的最大额外成本 = 一份 est 预扣滞留(窗口过期自动释放)+ 输家已被上游计费的不可观测部分。下游要能算出"对冲浪费多少钱"——靠 §4.5 的遥测区分,而不是新记账通道。
|
||||
|
||||
### 4.5 并发安全与遥测区分(设计问题 5)
|
||||
|
||||
| 关注点 | 设计 |
|
||||
| --- | --- |
|
||||
| `_CallContext` 共享 | 两个对冲任务共享同一个 context(同一逻辑调用)。`register_attempt`/`claim_terminal`/`snapshot` 均为**无 await 同步方法**(`types.py:339-360`),事件循环内任务并发调用天然安全,机制零改动;但 `types.py:321-337` docstring 的"单任务对象"承诺须修订为"单逻辑调用、可多任务并发登记"。增补 H8 后 context 再持 `_hedges`/`_generation_ms`/`_hedge_won` 三个计数与 `register_hedge`/`record_generation` 两个同步方法,任务安全性与 `register_attempt` 同款 |
|
||||
| 逻辑调用 ID | 不变:两任务共享 `logical_call_id`;各 attempt 独立 `call_id`(uuid4,`retry.py:279`) |
|
||||
| 快照时点 | 赢家产生 → 输家 `cancel()` 并 **await 收口完毕**(输家 finally 的结算/遥测跑完)→ 才允许 `client.py:442` 的快照返回。`attempts` 因此恒含输家(=2),`total_latency_ms` 含输家清理耗时——与 deadline"返回时刻 = 期限 + 清理耗时"同口径 |
|
||||
| 任务泄漏 | 编排用 `asyncio.wait(FIRST_COMPLETED)` + 显式收口;取消优先铁律不变:外部取消到达时两任务都被取消并穿透,不 shield、不留后台任务(ARCH §6.4) |
|
||||
| 遥测行区分(默认档,零新列零 DDL) | 输家 attempt 行 `error="hedge_cancelled"`(与既有 `"cancelled"` 同通道,`retry.py:327-334` 同款字符串);赢家 attempt 行照常;`CallStats` **只增**三字段(`types.py:296-318` 既有快照对象,经 `LLMResponse.call_stats` 既有通道带出,`types.py:425`;三字段全带默认值,1.3.6 及以前构造的 `CallStats(...)` 位置调用不炸):`hedges: int = 0`(本次调用**实际并发发出**的对冲路数;触发但准入失败静默不计)、`generation_ms: int = 0`(裸生成时间,口径见下行)、`hedge_won: bool = False`(赢家是否对冲路) |
|
||||
| `generation_ms` 口径(增补 H8) | **赢家那次 transport 调用的墙钟时长**(HTTP 发出到响应收完):chat/对冲 = 赢家那次;无对冲 = 成功那次 attempt;结构化重问 = 最后一轮(覆盖语义,每轮成功覆写);embedding = 各批 transport 时长之和(累加语义);OCR = 单次;缓存命中 = 0(未产生 transport 调用,0 是实测而非"未知")。**排除** admission 排队/backoff/对冲触发前等待/清理遥测;计时点收敛在三条链路 `_attempt` 的 transport 调用两侧,用该链路既有注入钟(与 `total_latency_ms` 同钟,差值才有意义);对冲编排裁定赢家后才写入 context,输家(含两路同时完成的竞速落选者)的值一律丢弃 |
|
||||
| 对前端有用的对冲参数(增补 H8 取舍) | **纳入** `hedges` + `hedge_won` + `generation_ms`(经 CallStats 既有通道带出,零遥测新列);**不纳入**每路 attempt 分别耗时/输家身份——那是运维诊断面,遥测 DB attempt 行已有 `hedge_cancelled` 标签与同 `logical_call_id` 可 join 还原,不重复进公开响应。`generation_ms` 与 `total_latency_ms` 的**差值即波动开销**(等待/退避/准入/对冲触发前耗损),前端可直接展示"在等不在生成" |
|
||||
| `hedge_cancelled` 标签机制 | 编排在 `cancel()` **之前**给输家任务置位标记(如 `task._polygateway_hedge_loser = True`);`_attempt` 的 CancelledError 分支读标记选 `"hedge_cancelled"`/`"cancelled"`。外部取消与对冲取消竞速时可能误贴——两任务同消、记账方向一致(est 保留),标签误贴不造成结算或熔断错误,属可接受并明写 |
|
||||
| 遥测行区分(备选调) | attempt 表加 `hedge_role` 列(`NULL/'primary'/'hedge'`,PG/SQLite 各一次 DDL)——遥测列变更代价有 issue #12/#13/#15 教训,v1 不推荐;列进批准项(H3)由人类定夺 |
|
||||
| 熔断/健康信号 | 输家取消**不喂**失败、赢家照常记成功——挂起不是源死亡证据(§3 关键判断) |
|
||||
|
||||
### 4.6 编排形态与失败汇合
|
||||
|
||||
- 对冲轮 = 一次"超级尝试":首个成功即本轮结果;**两任务都失败**才进既有重试循环,且 `fails += 1` 只计一次(对冲是加速形态,不是两次独立尝试;429 的免预算/refund 仍在 attempt 级生效)。
|
||||
- 原 attempt 先失败、对冲在途 → 直接等对冲结果,不重试;对冲先失败、原 attempt 在途 → 继续等原 attempt(等价于未触发对冲)。
|
||||
- `retry` 循环骨架(`retry.py:228-263`)、退避、stall 判定全部不变;变化收敛在"单轮尝试的内部从单任务变任务组"。
|
||||
|
||||
## 5. 配置面(设计问题 6;默认必须关闭)
|
||||
|
||||
| 键 | 值域 | 缺省 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| `{SCOPE}__HEDGE__AFTER_S` | 有限正数秒,复用 `ensure_call_deadline` 同款值域校验(`deadline.py:20-42`) | **未设 = 关闭** | 3 段键天然跳过 `_load_sources`(`config.py:401-407`);`HEDGE` 加进 `_RESERVED_SEGMENTS`(`config.py:57`)防 provider 段撞名 |
|
||||
| `{SCOPE}__HEDGE__MAX_EXTRA` | int ∈ [1,3] | 1 | 每次逻辑调用最多并发对冲几路;>1 仅对冲再挂起时梯次追加 |
|
||||
|
||||
装配路径与期限同款(136 设计 §4.3 形态):`GatewaySettings` 末尾追加两字段 + `__post_init__` 新守卫(`config.py:190-201` 同列);`GatewayClient.__init__` keyword-only 参数**入口即校**(`client.py:239-241` 同列);`from_settings` 透传,`from_env` 无签名变化。
|
||||
|
||||
| 守卫 | 判定 | 理由 |
|
||||
| --- | --- | --- |
|
||||
| `hedge_after_s ≥ min(源 timeout_s)` | `ValueError` | 对冲永不可能触发,配置即错误(与 `_validate_probe` 同款装配期炸掉哲学,`config.py:346-355`) |
|
||||
| `hedge_after_s ≥ min(ttft_timeout_s)`(仅设有该键的源) | 装配期 **warning** | 流式档挂起已被 TTFT 看门狗先行切断(`openai_compat.py:561-566`),对冲形同虚设;非流式仍有效,故不升 ValueError(与 `stall_window_s ≥ max(ttft_timeout_s)` 同型交叉守卫先例,`config.py:336-344`) |
|
||||
| 单源 scope 设了阈值 | 装配期 **warning**,允许 | 源集合可运行期之外的配置演进;运行期拿不到候选自然静默(§4.2) |
|
||||
| `hedge_after_s ≥ call_deadline_s`(两者皆设) | `ValueError` | 期限先于对冲触发,对冲形同虚设(§6) |
|
||||
| chat() per-call 覆盖参数 | **不提供** | 对冲阈值是源/渠道特性,不是任务特性(期限有 per-call 是因为任务耐心不同);需要不同阈值就装配两个 client |
|
||||
|
||||
## 6. 与 `call_deadline_s` 的关系(设计问题 7)
|
||||
|
||||
| 维度 | hedge | deadline |
|
||||
| --- | --- | --- |
|
||||
| 语义 | **提前换路**:提高 deadline 内拿到结果的概率 | **最终保险**:超过耐心即终止(治理等待) |
|
||||
| 层级 | RetryMW 单轮尝试内部 | 公开边界包整棵树(`client.py:419-421`),对冲编排在树内,`deadline.py` 零改动 |
|
||||
| 独立配置 | 可只配 hedge(无期限) | 可只配 deadline(1.3.6 现状) |
|
||||
| 组合 | `hedge_after_s < call_deadline_s`(装配守卫强制);典型:`timeout_s=300, hedge_after_s=8, call_deadline_s=120` | 输家取消的清理耗时不受期限管辖,沿用"返回时刻 = 期限 + 清理耗时"措辞(136 设计 §5.3);**per-call 覆盖** `chat(call_deadline_s=X)` 使 X < hedge_after_s 时,该次调用对冲不触发(deadline 先切整棵树),属合法语义不告警——装配守卫只管默认值,per-call 是调用方的当次选择 |
|
||||
|
||||
两者回答不同问题:deadline 让长尾**更早失败**,hedge 让调用**更快成功**——文档不得混写(136 设计 §11 已立此措辞纪律)。
|
||||
|
||||
## 7. 变更点清单(反 gold-plating;实施前置零)
|
||||
|
||||
| 类别 | 内容 |
|
||||
| --- | --- |
|
||||
| 改动 | `middleware/retry.py`(单轮尝试 → 任务组编排 + 对冲计时 + 输家 `hedge_cancelled` 遥测 + `_attempt` transport 级计时点);`middleware/admission.py`(`pick` 加私有排除参数);`ports.py` + `transports/openai_compat.py`(`complete` 加 `first_token_event` 必填 kw,流式首 token 处置位);`config.py`(两键 + loader + 两守卫 + 保留段);`types.py`(`CallStats` 增三字段 `hedges`/`generation_ms`/`hedge_won` + `_CallContext` docstring 修订与计数方法);`client.py`(构造参数 + 透传);`embedding.py`/`ocr.py`(`_attempt` 加 transport 级计时点——只计时不对冲,非目标 A 不变) |
|
||||
| 新增文件 | 无(编排收敛在 retry.py;若超 150 行可拆 `middleware/hedge.py`,实施期定) |
|
||||
| 直接复用 | 取消结算 S3 格、`settle_and_release` 单一出口、准入全链路、`asyncio.timeout` 范式、假 transport/FakeClock 测试设施、限流契约套件(Lua 不改) |
|
||||
| 明确不做 | 不改四分类/熔断语义/429 分账/限流 Lua/`deadline.py`;不加遥测列(默认档);不做 embedding/OCR/分位数/同源对冲/per-call 参数;不引入 shield/后台任务 |
|
||||
|
||||
## 8. 测试策略(设计问题 8:事件驱动,不 sleep 撞窗口)
|
||||
|
||||
| 原则 | 做法 |
|
||||
| --- | --- |
|
||||
| 事件驱动假 transport | 两个 `asyncio.Event`(`first_token`/`complete`)精确控制 TTFT 与完成时刻;触发判定 = "事件未置位且计时器到期",从不真睡出长尾 |
|
||||
| 真实 loop 钟 + 余量 | 对冲阈值取 0.05s 级、断言容差 4–10×(`tests/unit/test_streaming.py` 既有范式,136 设计 §10 已验证稳定,不标 slow) |
|
||||
| 先失败后通过 | 同一挂起场景:无对冲 → 总时长 = 挂起时长(红);开启 → 总时长 ≈ 阈值 + 快源耗时(绿) |
|
||||
| 注入钟纪律 | 伪造注入 `now` 跳变 10^6 秒不得触发对冲(对冲计时只用 loop 相对时长) |
|
||||
|
||||
验收矩阵(离线、不触网):① 触发两形态(流式 TTFT 未至触发/已至不触发;非流式纯时间触发);② 异源排除(断言第二请求落在另一源;无候选静默);③ 准入失败静默(配额满 → 不对冲,原请求照等);④ 赢输记账(赢家 settle 实际 usage、输家 settle est、`tpm_used` 断言);⑤ 并发安全(`attempts==2`、终态行恰 1 条、`logical_call_id` 一致、无任务泄漏告警);⑥ 默认关闭回归(现有全套件不改一行断言全绿);⑦ 外部取消穿透(两任务同消、`CancelledError` 上抛);⑧ 与 deadline 组合(期限切断含对冲的整棵树);⑨ 配置守卫四路(env/直接构造/replace/client 直传);⑩ 裸生成时间断言(增补 H8):**赢家计时不含等待**(对冲赢家的 `generation_ms` ≈ 赢家路 transport 时长,不含触发前等待/admission/backoff);**对冲赢家取快者**(对冲路赢 → `hedge_won=True` 且为对冲路时长;原路后发先至 → `hedge_won=False` 且为原路时长);**embedding 为批次和**(N 批各自 transport 时长累加);**缓存命中为 0**(第二次同 key 调用 `generation_ms == 0` 且 `attempts == 0`)。
|
||||
|
||||
## 9. 集中人类批准项
|
||||
|
||||
**状态: H1–H7 与增补 H8 全部已于 2026-09-10 获人类批准**(H1 取方案 A;H3 取零新列档;H5 取"v1 只允许 1")。
|
||||
|
||||
| # | 决策 | 推荐 | 备选代价 |
|
||||
| --- | --- | --- | --- |
|
||||
| H1 | 方案选型 | **A(并发对冲)** | B 省 2/3 实现量,但须新造"挂起失败"语义且污染或不污染熔断二选一;C 不解决 issue 现场 |
|
||||
| H2 | `Transport.complete` 加 `first_token_event` 必填 kw(公共端口签名变更) | 批准 | 不加则流式只能用纯时间阈值,误对冲慢生成(issue 明示的反面) |
|
||||
| H3 | 遥测区分档位 | 零新列(`hedge_cancelled` 字符串 + `CallStats` 三字段,含 H8) | `hedge_role` 列更规整但要 PG/SQLite 双 DDL + 迁移纪律 |
|
||||
| H4 | 配置键名/值域/守卫(§5 全表,含交叉守卫 ValueError) | 按 §5 | 交叉守卫降为 warning 则错配静默 |
|
||||
| H5 | `hedge_max_extra > 1` 的梯次对冲 | v1 只允许 1(键存在但上限 1 也接受) | 直接放开到 3 省一次版本,但多路对冲洗掉信号 |
|
||||
| H6 | 两败计一次重试预算 | 批准 | 计两次会让对冲调用更快耗尽预算,语义说不过去 |
|
||||
| H7 | embedding/OCR/分位数/同源对冲/per-call 参数全部不进本版 | 批准 | 任一纳入都是公共面扩大,需单独论证 |
|
||||
| H8(增补) | `CallStats` 再增 `generation_ms`(裸生成时间,口径见 §4.5)与 `hedge_won` 两字段;retry/embedding/ocr 三条链路 `_attempt` 加 transport 级计时点 | 批准(2026-09-10,随 H1–H7 同日) | 不加则前端拿不到"在等不在生成"的量化口径,issue #24 的现场观测(13% 调用吃掉 71% 模型总时间)无法在产品面复现;每路 attempt 分别耗时与输家身份走遥测 DB join 还原,不进公开响应 |
|
||||
|
||||
## 10. 残余风险(诚实标注)
|
||||
|
||||
| 项 | 状态 |
|
||||
| --- | --- |
|
||||
| 输家取消能否止住上游计费 | 无一手证据(与 136 设计 §12 同款):est 保留只是闸内保守记账,**不是**上游真实计费的计量;文档只写"可能已计费",不写"对冲浪费上限 = est" |
|
||||
| 非流式误对冲慢生成 | 物理不可分(无中途信号);只能靠阈值取值(建议 > 源 p50 数倍)与 `hedge_max_extra` 上限控制;分位数触发是未来缓解 |
|
||||
| 对冲流量放大 | 开启后挂起窗口内 in-flight 翻倍;准入全走闸意味着饱和期自然静默,但**配置者须理解**对冲 = 用配额换延迟 |
|
||||
| "挂起不喂熔断"的反向代价 | 一个持续挂起的源不会因对冲输家而被熔断标记;源级淘汰仍靠既有失败/超时路径——这是有意选择(§3),但运维上"挂起率"只能靠 `hedge_cancelled` 遥测行统计 |
|
||||
| 两任务共享 `_CallContext` 的承诺修订 | docstring 级变更;若未来给 context 加带 await 的方法,须重审任务安全 |
|
||||
| 多路对冲(H5 若放开) | 信号冲刷与成本上界均未论证,v1 不碰 |
|
||||
@@ -0,0 +1,293 @@
|
||||
---
|
||||
type: finding
|
||||
node_id: finding:2026-09-09-135-call-observability-validation
|
||||
title: "1.3.5 T2/T3/T4 验收证据:36 列遥测、失败终态、PG 存储兼容与变异矩阵"
|
||||
date: 2026-09-09
|
||||
---
|
||||
|
||||
# 1.3.5 T2/T3 验收证据
|
||||
|
||||
> 最新状态(2026-09-09):**1.3.5 已发布**,main 与 `v1.3.5` 同指 `ab00aa4`;§8.6 所列「尚未执行」已由
|
||||
> [发布完成与外部验收](2026-09-09-135-release-completion.md)逐项关闭。以下为分阶段历史,不追改当时结论。
|
||||
>
|
||||
> 范围:**仅 T2(遥测 10 列 / 诊断保真 / scope+operation / 装配闸)与 T3(终态行 / 取消 / 统一出口)**,
|
||||
> 外加计划 T4 中"签名与列数机械迁移"那一片(与 schema 同批完成,避免先提交 schema 却留写入缺键)。
|
||||
> **不含** T4 的 PG 集成、变异矩阵与文档同步——另任务承接,缺口见 §5。
|
||||
> 设计:`designs/2026-09-09-135-call-observability-design.md`;计划:`plans/2026-09-09-135-call-observability.md`。
|
||||
> 基线 HEAD `87c261b`(T1 已提交,1419 unit 全绿)。全部命令在 `PolyGateway` conda 环境执行。
|
||||
|
||||
## 1. 红绿证据链
|
||||
|
||||
TDD 纪律要求"先失败后通过",且红必须是**行为红**而非 import 红。下表每行都对应本会话的真实工具输出。
|
||||
|
||||
| # | 阶段 | 命令 | 结果 |
|
||||
| --- | --- | --- | --- |
|
||||
| 0 | 基线 | `pytest tests/unit -q` | **1419 passed**(对照底) |
|
||||
| 1 | 红(T2/T3 目标行为) | `pytest tests/unit/test_telemetry.py -k "RowLevelObservability or RecorderShapeGate"` | **26 failed**,证据 `tests/outputs/135/red-01-emitter-rows.txt` |
|
||||
| 2 | 实现后全量红面 | `pytest tests/unit -q` | **137 failed / 1308 passed**(机械迁移面暴露),`red-02-after-impl.txt` |
|
||||
| 3 | 迁移中 | 同上 | 60 → 25 → 8 → 4 failed(`red-03`/`red-04`/`red-05`) |
|
||||
| 4 | 绿 | `pytest tests/unit -q` | **1467 passed, 0 failed** |
|
||||
| 5 | 静态门 | `make check`(ruff lint+format + import-linter) | **Contracts: 1 kept, 0 broken** |
|
||||
|
||||
第 1 步的红是行为红而非 import 红:新用例调用的是**已存在**的 `TelemetryEmitter`,
|
||||
失败形态是"缺 `scope=`/`operation=`/`stats=` 关键字"与"断言的列不存在",不是模块导不进来。
|
||||
|
||||
补测阶段另有两次真实红(均由 conda 实跑暴露、当场修正,非噪音):
|
||||
`test_client.py` 的 `NameError: ResultInvalidError / ChatRequest`(漏 import)、
|
||||
`TestTerminalRowSqlSemantics` 的 `'coroutine' object has no attribute 'execute'`(async helper 漏 await)。
|
||||
|
||||
## 2. 核心要求逐条对应
|
||||
|
||||
### 2.1 36 列与 Emitter 完整一致(不留"schema 有列、写入缺键")
|
||||
|
||||
一次性同批改完五处列定义 + 端口签名 + Emitter 写入,故不存在中间态。
|
||||
|
||||
| 判据 | 证据 |
|
||||
| --- | --- |
|
||||
| `len(COLUMNS) == 36`、物理列 37 | `test_telemetry.py::TestSchemaModule::test_columns_and_ddl_are_frozen`;`TestSQLiteSchemaMode` 断言 23 → 37 |
|
||||
| 五处列序一致(DDL/BACKFILL×2/COLUMNS) | 新建库与 ALTER 追加列序同为 `_EXPECTED_COLUMNS` |
|
||||
| 端口实测 36 字段、10 新参 keyword-only 且无默认值 | `test_ports.py::TestTelemetryRecorderSignature`(`inspect.signature` 实测,不凭记忆) |
|
||||
| **Emitter 实参键集合 == `schema.COLUMNS`** | `TestEmitterRecorderContract` 三入口逐个断言 `set(rows[0]) == set(COLUMNS)` |
|
||||
| 1.2.1 冻结 INSERT 未被改写 | 按 `_PRE_135_COLUMNS`(26 列)重现原文;全量 36 列另按占位符个数断言 |
|
||||
|
||||
装配闸(C3):`_assert_recorder_shape` 在 `TelemetryEmitter.__init__` 做一次 `signature.bind`,
|
||||
参数名**从协议签名派生**而非手抄第四份清单——`test_gate_derives_parameter_names_from_the_protocol`
|
||||
用 monkeypatch 换掉协议后闸自动跟随,证明没有硬编码。旧签名 recorder / 无该方法 / 不可 inspect
|
||||
一律装配期 `ValueError`(不是 warning:降级铁律管的是运行期写失败,不是配置错误)。
|
||||
|
||||
### 2.2 三个 client 的**真实链路**失败终态
|
||||
|
||||
不是只测 Emitter,而是驱动真实 client + MockTransport 到落库。
|
||||
|
||||
| 链路 | 用例 | 断言 |
|
||||
| --- | --- | --- |
|
||||
| chat 重试耗尽 | `TestChatTerminalFailureRows::test_retry_exhaustion_writes_exactly_one_terminal_row` | 3 条 attempt + **恰 1 条**终态;终态 `attempts == 3` |
|
||||
| chat 结构化耗尽 | `test_structured_exhaustion_writes_the_only_failure_row` | attempt 行**全是成功行**,终态是唯一失败记录;error 含有界 `validation=`/`repair=` 且 < 1200 字符、不含 raw_text |
|
||||
| chat 400 直拒 | `test_request_rejected_now_has_both_an_attempt_and_a_terminal_row` | 尝试行 + 终态行**各 1**(已批准的行数翻倍);两行共享同一 `logical_call_id` |
|
||||
| embedding | `test_embedding.py::TestReasonlessTelemetryContract::test_failed_attempts_still_have_null_effort` | attempt 数 == 脚本长度,终态恰 1 |
|
||||
| OCR 两方法 | `test_ocr_client.py` 同名用例(参数化 `recognize_text`/`parse_layout`) | 终态恰 1,且 `operation` 为**公开方法名** |
|
||||
|
||||
去重:`test_terminal_row_is_written_once_per_logical_call` 连调出口 3 次,SQL 可见仍 1 条(`claim_terminal`)。
|
||||
非领域异常:`test_non_domain_exception_writes_no_terminal_row` —— `KeyError` 原样传播、**0 条**终态、分类不被改写。
|
||||
|
||||
**双写已消除**:`TelemetryMW` 的两个终态分支删除,改由三个 client 的公开边界经
|
||||
`emit_terminal_once` 统一写出;`TestTelemetryMW::test_scope_level_failure_is_not_written_here_anymore`
|
||||
与 `test_cancellation_is_not_written_here_anymore` 锁死"本层不再写终态",防回归双计。
|
||||
|
||||
### 2.3 取消口径
|
||||
|
||||
- chat 取消:`test_cancellation_writes_at_most_one_terminal_row` —— 恰 1 条,`error == "cancelled"`,
|
||||
且 `error_type` 为 NULL(**字符串不解析猜诊断**)。
|
||||
- 三链路同策略"尽力写一条、允许 0";`emit_terminal_once` **不 shield、不开后台任务**,
|
||||
快照冻结是同步动作;写入 await 上再被取消则 `CancelledError` 原样传播(与 TelemetryMW 历史行为同款)。
|
||||
|
||||
### 2.4 SQL 不双计(按真实 SQLite 落库断言)
|
||||
|
||||
`TestTerminalRowSqlSemantics` 用真实 `SQLiteRecorder` 驱动一次失败 chat 后直接查表:
|
||||
|
||||
| 迁移影响 | 断言 |
|
||||
| --- | --- |
|
||||
| 失败计数判据 | `error IS NOT NULL` 得 **3**(2 尝试 + 1 终态),`event_kind='terminal_failure'` 得 **1** |
|
||||
| 费用不双计 | 终态行 `cost IS NOT NULL` 计数为 **0**;且 `usage_source='unavailable'`、token 全 0 |
|
||||
| 时延分组 | 终态 `latency_ms` ≥ 任何单次尝试(含退避),故看板必须按 `event_kind` 分组 |
|
||||
| 双时钟微差 | 终态 `latency_ms == total_latency_ms`(同一份冻结快照) |
|
||||
| 逻辑两列归属 | 尝试行 `attempts`/`total_latency_ms` 恒 NULL |
|
||||
| §5 归因查询 | 同一 `logical_call_id` 同时给出整池 reason 文案与逐源 503 现场;终态 `http_status_code` 为 NULL(C1 不冒充) |
|
||||
|
||||
### 2.5 诊断保真与 operation 修正
|
||||
|
||||
- 中转把 529 改写成 503 → **记 503 不猜回 529**;直接 529 记 529。
|
||||
- 空 `str()` 的 Connect/Read/Write/PoolTimeout → `cause_type` 落对应 httpx 类名,`error` 退回类名。
|
||||
- 成功行五列全 NULL(**不统一填 200**)。
|
||||
- `_status_to_error` 增 keyword `operation`:`embed()` 非 200 改传 `"embedding"`(修正历史误标),
|
||||
流式与非流式 chat 两处仍 `"chat"`(按实施计划 §2 归属表,未按设计行号误标)。
|
||||
- 新列 `operation` 恒为公开方法四值,**绝不读 `exc.operation`**:
|
||||
`test_operation_is_given_by_the_call_site_not_the_exception` 用 `exc.operation="download_result"` 反证。
|
||||
- OCR `"类名: msg"` 前缀由出口的显式策略参数 `class_prefixed_error` 承载,不再三处各拼一遍。
|
||||
|
||||
## 3. 有界 validation 说明的单一所有者
|
||||
|
||||
`structured.py` 的 `_MAX_FEEDBACK_ERRORS`/`_MAX_ERROR_CHARS`/`_format_errors` 改名为
|
||||
公开的 `MAX_FEEDBACK_ERRORS`/`MAX_ERROR_CHARS`/`format_bounded_errors`,
|
||||
由"重问反馈"与"终态结构化说明"两个消费者共同引用,**数值只有一份**。
|
||||
行为逐字不变(`test_structured.py` 22 项全绿,含反馈文案用例)。
|
||||
|
||||
## 4. 未改动确认(防越界)
|
||||
|
||||
`errors.py`、`admission.py`、`ratelimit.py`、`breaker.py`、`sources.py`、`thinking.py`、
|
||||
`providers.py`、`telemetry/sqlite.py`、`telemetry/postgres.py`、`transports/monkey_ocr.py`
|
||||
一字未动——两个 recorder 靠 `**fields` + `schema.COLUMNS` 自动吃到新列。
|
||||
缓存 key 公式、重试预算与退避、429 免预算、stall 算法、取消结算、推理能力表均未触碰。
|
||||
唯一顺带修正:`RetryMW.__init__` 的 `emitter` 注解由 `object | None` 收紧为
|
||||
`TelemetryEmitter | None`(TYPE_CHECKING 导入,同层不破分层契约),因本轮改了它的 `_emit`。
|
||||
|
||||
## 5. T4 验收(PG 存储兼容、变异矩阵、文档同步)
|
||||
|
||||
基线 HEAD `393f2bf`(1467 unit 全绿),全部命令在 `PolyGateway` conda 环境执行。
|
||||
|
||||
### 5.1 PG 存储兼容(真实实验室 Postgres,复用 `pg_sandbox`)
|
||||
|
||||
机械迁移:`_EXPECTED_COLUMNS` 27 → 37 列、`_record_minimal` 字段字典补 10 键(与单测同款)、
|
||||
`_PRE_TENANT_COLUMNS` 的缺列集由4 列扩到 14 列。新十列一律由 `_CALL_OBSERVABILITY_COLUMNS` 派生,
|
||||
两处 manual 档告警的逐字断言改成按 `COLUMNS` 序派生的 `_PRE_TENANT_MISSING_NOTICE`——
|
||||
另抄一份列名必然漂移,而漂移的表现是“manual 档没补列”这条断言假绿。
|
||||
|
||||
| 阶段 | 命令 | 结果 |
|
||||
| --- | --- | --- |
|
||||
| 机械迁移前(已知必红) | `pytest tests/integration/test_postgres_telemetry.py -q -x` | **1 failed / 18 passed**:manual 档列序断言 `Right contains 10 more items, first extra item: 'scope'` |
|
||||
| 机械迁移后 | 同上(无 `-x`) | **27 passed**,EXIT=0,`tests/outputs/135/pg-telemetry-after-mechanical.log` |
|
||||
| 新增四项验收后 | 同上 | **30 passed**,EXIT=0,`tests/outputs/135/pg-telemetry-new-cases.log` |
|
||||
|
||||
新增 `TestCallObservabilityColumnsAcceptance`(对应计划 T4 的 PG 四项),共用一张 27 列的 1.3.4 形态旧表(`pre_135_schema`):
|
||||
|
||||
| 验收项 | 用例与关键断言 |
|
||||
| --- | --- |
|
||||
| auto 追加 10 列 + 旧行 NULL | `test_pre_135_table_gains_the_ten_columns_and_old_rows_stay_null`:物理列 27 → 37 且列序 == `_EXPECTED_COLUMNS`;attempt 行与 terminal 行十列取值**整体比对**(不是逐条 in);历史行 `dict.fromkeys(...)` 十列全 NULL |
|
||||
| manual 缺列裁剪 | `test_manual_trims_the_insert_on_a_pre_135_table`:表结构逐字不动(== `_PRE_135_COLUMNS`);裁剪后 26 列逐列等于提交值(防整体错位);恰 1 条告警且含可直接粘贴的首/末列 ALTER;无“写入失败”/“补列失败”,`degraded is False` |
|
||||
| 新旧进程混写 | `test_old_and_new_writers_share_one_table`:新版建表写 36 列 → 旧版进程用 `insert_sql("postgres", 26 列集)` 写入 → 新版再写;三行共存、表结构不变、旧行新列全 NULL、无写入失败 warning |
|
||||
| 下游口径验收 | 同上用例尾部:`WHERE event_kind = 'terminal_failure'` 计得 1;`event_kind IS NULL` 计得 1(混写期旧行既不误计成失败也不误计成成功) |
|
||||
|
||||
纪律:未引用 `assert_no_leftovers`(它是 `test_pg_sandbox.py` 的模块级 fixture,对本文件不可见,上提它要改 `conftest.py`);
|
||||
未新建沙箱设施、未碰共享表 `llm_calls`、未读或打印 DSN。新增的只有一个 `_execute_args`(带参数单语句)与
|
||||
`_minimal_fields`(从 `_record_minimal` 拆出的字段字典,供“旧进程”复用同一份取值)。
|
||||
|
||||
### 5.2 变异矩阵(仓库外副本,主工作区生产代码零改动)
|
||||
|
||||
驱动脚本 `/tmp/pgw-135-mut/run_mutations.py`;每项“还原副本 → 施加单点变异(断言替换命中)→ 跑指定节点 → 还原”。
|
||||
完整日志 `tests/outputs/135/mutations.log`,脚本总退出码 **EXIT=0**。
|
||||
|
||||
| # | 变异 | exit | 被杀断言(节选) |
|
||||
| --- | --- | --- | --- |
|
||||
| M1 | `register_attempt()` 挪到 transport 成功之后 | 1 | `test_failed_retries_are_counted`、`test_budget_free_429_still_counts_as_an_attempt`、`test_retry_exhausted_counts_every_attempt` |
|
||||
| M2 | `ChatRequest` 每次 `replace` 复制出新上下文 | 1 | `TestLogicalCallStats` 4 项 + `test_retry_exhaustion_writes_exactly_one_terminal_row` |
|
||||
| M3 | `_rehydrate` 去掉 `call_stats=None` 覆盖 | 1 | `test_historic_dict_never_impersonates_call_stats` |
|
||||
| M4 | Emitter 入口提前 `str(exc)` 压平 | 1 | 529/503 保真、4 个空超时文案 `cause_type`、类名前缀策略、结构化有界说明等 9 项 |
|
||||
| M5 | 终态行复制最后一次 attempt 的 token/cost | 1 | `test_terminal_rows_never_contribute_to_cost`、`test_terminal_row_costs_nothing` |
|
||||
| M6 | 去掉 `claim_terminal` 去重 | 1 | `test_terminal_row_is_written_once_per_logical_call`、`test_claim_terminal_is_true_once` |
|
||||
| M7 | 装配闸改为捕获 `TypeError` 后 warning | 1 | `test_old_signature_recorder_is_refused_at_assembly`、`test_uninspectable_recorder_is_a_configuration_error` |
|
||||
|
||||
还原校验:副本 `.py` 文件集合散列与纯净态**逐字一致**(`66b58b9a…`),还原后同一批节点 **172 passed**。
|
||||
|
||||
> **方法论陷阱(影响本仓所有变异证据的有效性)**:只设 `PYTHONPATH=<副本>/src` 是**无效的**——
|
||||
> `pyproject.toml` 的 `[tool.pytest.ini_options] pythonpath = ["src"]` 会把**仓库内**的 `src` 抢先塞进 `sys.path[0]`,
|
||||
> 于是测的仍是原代码。首轮实跑七项变异**全部“存活”(exit=0)**就是这个坑;
|
||||
> 改用 `-o pythonpath=<副本>` 覆盖后七项全部被杀。今后做变异必须先证“副本真的被导入”,
|
||||
> 否则“变异存活”会被误读成“测试不够强”,而真相是变异根本没生效。
|
||||
|
||||
### 5.3 文档同步(字段数一律 `inspect` 实测,不凭记忆)
|
||||
|
||||
实测值:`len(inspect.signature(TelemetryRecorder.record_llm_call).parameters) - 1 == 36`,`len(COLUMNS) == 36`,物理列 37。
|
||||
|
||||
| 位置 | 改了什么 |
|
||||
| --- | --- |
|
||||
| `README.md` 能力表 | “必录 26 字段” → 36 字段 + 三类行与诊断列;新增“逻辑调用统计”一行 |
|
||||
| `README.md` 新小节 | 《1.3.5 逻辑调用统计与失败诊断》:`call_stats` 读法、**SQL 迁移五项**、归因查询、存储侧升级 |
|
||||
| `README.md` cap 覆盖面 | 补“`error_body` 沿用 `summarize_body` 上限、结构化说明自带限长,**两者都不在 cap 覆盖内**” |
|
||||
| `ARCHITECTURE.md` §7.8 | 必录字段 26 → 36(十列逐个列出);新增“逻辑调用十列”段(列语义表 + 不变量 I3/I4 + `operation` vs `exc.operation` 两个语义 + 装配闸);cap 段补两处诊断文本不在覆盖面 |
|
||||
| `CHANGELOG.md` | 新建《未发布》:公共面四项变更表 + 下游必须做的事(点名 `WHERE event_kind` 与装配期报错) |
|
||||
| `.env.example` | `PGW_TELEMETRY_TEXT_CAP` 块补 1.3.5 覆盖面例外 |
|
||||
| `schemas/llm-calls.md` | 标题 26 → 36 字段;十列逐行登记;新增《三类行与失败归因口径》(含归因 SQL 与迁移四条) |
|
||||
| `metrics/call-telemetry-coverage.md` | 新增《1.3.5 三类行与逻辑调用覆盖》;live 基线列**不写伪百分比** |
|
||||
|
||||
版本号与发布步骤**未动**(任务边界);`pyproject.toml` / `__init__.py` 仍为 1.3.4。
|
||||
|
||||
### 5.4 本轮收口验证
|
||||
|
||||
| 检查 | 命令 | 结果 |
|
||||
| --- | --- | --- |
|
||||
| 静态 | `make check`(ruff format+lint + import-linter) | 见 §7 实跑记录 |
|
||||
| 全量单测 | `pytest tests/unit -q` | 见 §7 |
|
||||
| 目标集成 | `pytest tests/integration/test_postgres_telemetry.py -q` | **30 passed** |
|
||||
|
||||
## 6. 剩余缺口(不归本任务,不自称已关)
|
||||
|
||||
| 缺口 | 说明 |
|
||||
| --- | --- |
|
||||
| 独立验证 | 未派全新上下文 verifier(合并前硬门,由父会话前台派) |
|
||||
| slow / e2e | 未跑,属发布清单第 4 步(CLAUDE §4.4.1) |
|
||||
| live 覆盖基线 | `metrics/call-telemetry-coverage.md` 的实际基线列仍待首次生产运行填入 |
|
||||
| 下游自建 recorder | 装配闸只能证明形状可被接受,证不了函数体真的落这些列(已写进 README/CHANGELOG) |
|
||||
|
||||
## 7. 环境噪音记录(不是缺陷)
|
||||
|
||||
自动检查器用**系统解释器 Python 3.13.9**(无 `redis`/`pydantic`/`httpx`/`loguru`/`asyncpg` 等依赖),
|
||||
持续报 `test_client.py` 2 项失败与大量 "Import could not be resolved"、`StrEnum is unknown import symbol`。
|
||||
已核实为环境问题、非本轮引入:
|
||||
|
||||
- 失败根因是 `ModuleNotFoundError: No module named 'redis'`(optional extra),本轮 diff 对 redis 零改动;
|
||||
- 在**未改动的 HEAD** 上用同一系统解释器复跑,同样 2 failed / 90 passed;
|
||||
- 项目强制环境 `conda run -n PolyGateway`(Python 3.12.13)下:`test_client.py` 98 passed、全量 1467 passed。
|
||||
|
||||
判据以 CLAUDE.md §2 规定的 conda 环境与 `make check` 为准。
|
||||
|
||||
**T4 轮次同样现象(已逐项复现并关闭)**:检查器报 `tests/integration/test_postgres_telemetry.py` “2/3 failed”
|
||||
与 6 处 `Import "asyncpg" could not be resolved`。根因与上同:`asyncpg>=0.29` 是 optional extra
|
||||
(`pyproject.toml:25` 的 `postgres`),检查器解释器里没装。已做确定性复现:
|
||||
|
||||
- 那 3 条恰是本文件**仅有的不依赖真实 PG 的用例**,其中 2 条要造 `PostgresRecorder`;
|
||||
- 用 `/home/iomgaa/miniconda3/bin/python`(无 asyncpg)跑这 3 条:**2 failed / 1 passed / 0.09s**,与检查器报告逐字吹合;
|
||||
- 同 3 条在 `conda run -n PolyGateway` 下:**3 passed**;整文件 **30 passed**(真实 PG);
|
||||
- 被标记的 6 行均为**本轮未触及的旧行**,本轮只新增 1 处同款函数内 `import asyncpg`。
|
||||
|
||||
未为此修改代码:给旧行加 type-ignore 属任务外改动(反 gold-plating),且并非真修复。
|
||||
|
||||
## 8. 1.3.5 发布准备(2026-09-09,唯一 writer 会话)
|
||||
|
||||
范围仅**发布准备文档与版本号**:README 安装下界、CHANGELOG 定版、两处版本、本节记录。生产代码与测试**零改动**(`git diff` 只有 `CHANGELOG.md` / `README.md` / `pyproject.toml` / `__init__.py` 各 1 行,外加本文件追加的这一节)。未 merge/push/tag/构建/上传。
|
||||
|
||||
### 8.1 先查远端占用(改文件之前)
|
||||
|
||||
| 检查 | 结果 |
|
||||
| --- | --- |
|
||||
| `git ls-remote --tags origin` | 最高 `refs/tags/v1.3.4`,**无 v1.3.5**;远端分支只有 `main`(`af57f93`)与 `docs/1.3.4-release-evidence` |
|
||||
| 本地 tag | 同样止于 `v1.3.4` |
|
||||
|
||||
远端未被占用是**本次时点**的事实,父会话真正 push/tag 前仍须复查以防竞态。
|
||||
|
||||
### 8.2 版本与文档数字(数字一律实测)
|
||||
|
||||
| 项 | 结果 |
|
||||
| --- | --- |
|
||||
| 两处版本 | `pyproject.toml` 与 `src/polygateway/__init__.py` 同为 `1.3.5`;由 `tests/unit/test_package.py` 机械断言一致(**6 passed**) |
|
||||
| README 安装下界 | `>=1.3.4,<2` → `>=1.3.5,<2`(本版含装配期形状闸与 SQL 口径迁移,旧下界会让下游装到不含新列的包) |
|
||||
| README 迁移醒目度 | 《1.3.5 逻辑调用统计与失败诊断》为顶层小节且带 `[!WARNING]`(点名"失败行变多、旧失败 SQL 会多数、自建 recorder 装配期报错"),能力表"逻辑调用统计"一行直链该锚点;沿用 1.3.4 的同款版式,未重排既有 1.3.4 迁移节 |
|
||||
| CHANGELOG 定版 | `## 未发布` → `## 1.3.5(2026-09-09)`,日期取自本机 `date +%F` 实际值,正文未改 |
|
||||
| 字段数复核 | `inspect.signature(TelemetryRecorder.record_llm_call)` 去 self **36 参**、`len(schema.COLUMNS) == 36`、探针实测物理列 **37**——README/CHANGELOG 的 36/37 与实测吻合,非凭记忆 |
|
||||
|
||||
README 静态数字未发现错误,故未做任何顺带修改。
|
||||
|
||||
### 8.3 复用既有证据的适用边界(不拿旧证据顶替本版统计)
|
||||
|
||||
| 证据 | 是否复用 | 边界 |
|
||||
| --- | --- | --- |
|
||||
| 独立验证(全新上下文 verifier,父会话前台派) | 复用结论:**0 阻塞** | 审的是本分支 `b4812e1` 的 1.3.5 变更面;本会话未重复派子代理,也未把它当作 live 覆盖或发布后检查的替代 |
|
||||
| 本版最终全套件 `make test` | **本版实跑,不复用** | `final-gates/make-test.log/.exit`:**1605 passed / 23 skipped / 108 deselected / 95%,exit 0** |
|
||||
| 本版真实 PG 集成 | **本版实跑,不复用** | `final-gates/pg-telemetry-verbose.log/.exit`:**30 passed,exit 0**(含新增四项列兼容验收) |
|
||||
| 本版 unit | **本版实跑** | 本轮版本号改动后复跑 `pytest tests/unit -q`:**1469 passed,exit 0**(`release/unit.log/.exit`) |
|
||||
| 1.3.4 模型矩阵/逐型号推理证据 | **按适用条件复用** | 1.3.5 对 `thinking.py`、能力表、wire 片段、缓存 key 公式、重试/限流/熔断语义**一字未改**(§4 已逐文件确认),故 1.3.4 的型号级结论在其原有边界内继续成立——连同它的 FAIL/UNKNOWN/不可达/缺轮**一并继承**,不因本版而升格。它**不能**充当 1.3.5 新增 API/schema 的证据,也不提供本版的测试统计数字 |
|
||||
| 1.3.4 的全模型矩阵重跑 | **不跑** | 本版未改推理路径;用户已批准不再补全模型矩阵。本会话不发全型号请求 |
|
||||
|
||||
本版**改变的公共面**(`CallStats` 与四类响应新字段、端口 10 新参、遥测 10 列与 `terminal_failure` 行)全部由新写的离线真链路用例(三个 client + MockTransport 直到落库)、真实 SQLite 断言、真实 PG 30 项承担,见 §2、§2.4、§5.1。
|
||||
|
||||
### 8.4 本轮受影响路径的真实现场(并发 1、超时不压、无全模型请求)
|
||||
|
||||
| 亲跑 | 命令与证据 | 结果 |
|
||||
| --- | --- | --- |
|
||||
| 真实网关冒烟(复用既有 e2e,不另造工具) | `pytest tests/e2e/test_smoke_gateway.py -m slow -v`,tmux `pgw135-smoke`,`release/smoke-gateway.log/.exit` | **4 passed,exit 0**(流式/非流式/结构化 json/结构化模型四节点,9.55s) |
|
||||
| 有界现场探针(**恰 1 次**真实调用,全程走库) | `release/probe_call_stats.py` → `release/probe-call-stats.log/.exit`,tmux `pgw135-probe` | **exit 0**:`MiniMax-M3`/源 `minimax_1`,`call_stats.attempts=1`、`total_latency_ms=2581` ≥ 单次尝试 `2562`、`logical_call_id` 与落库行逐字一致;临时 SQLite 物理列 **37**、写入面 **36**、恰 1 条 `event_kind='attempt'` 行、`operation='chat'`、`scope='LLM'`、逻辑两列与四个诊断列全 NULL |
|
||||
|
||||
探针脚本只落在 `tests/outputs/135/release/`(该目录 gitignore,不入库),不新增生产代码、测试或平台设施,未发裸 HTTP,沿用 `.env` 现有超时与单源配置。
|
||||
|
||||
### 8.5 提交前收口门
|
||||
|
||||
| 门 | 证据 | 结果 |
|
||||
| --- | --- | --- |
|
||||
| `make check` | `release/check.log/.exit` | **exit 0**:94 文件格式通过、ruff 通过、import-linter **1 kept / 0 broken** |
|
||||
| 包单测(两处版本一致) | `release/package.log/.exit` | **6 passed,exit 0** |
|
||||
| 全量 unit | `release/unit.log/.exit` | **1469 passed,exit 0** |
|
||||
|
||||
23 项 skip 的理由已逐条留档(`final-gates/skip-reasons.log`):17 项 redis 时间语义由 integration 变体覆盖、6 项需实验室语料 `data/soak/chs_images`;**skip 不计通过**。conda 启动器既有 `RequestsDependencyWarning` 保留,不宣称零告警。
|
||||
|
||||
### 8.6 尚未执行(不得当成已完成)
|
||||
|
||||
merge、push、`git tag -a v1.3.5`、`python -m build`、`twine check`/`upload`、`pip download` 解包验证、Gitea Release 与包页面/仓库关联检查**全部未执行**;`pytest -m slow` 的其余 e2e(本轮只跑了 `test_smoke_gateway.py` 四节点)与 live 覆盖基线同样未跑。§6 的四项缺口除"独立验证"已由父会话关闭外,其余保持开启。
|
||||
@@ -0,0 +1,98 @@
|
||||
---
|
||||
type: finding
|
||||
node_id: finding:2026-09-09-135-release-completion
|
||||
title: "1.3.5 发布完成与外部验收:合并后门、registry 产物与页面亲查"
|
||||
date: 2026-09-09
|
||||
---
|
||||
|
||||
# 1.3.5 发布完成与外部验收
|
||||
|
||||
> 状态:**1.3.5 已发布并完成外部验收,#19 / #23 已评论关闭**。main 与 `v1.3.5` 同指
|
||||
> `ab00aa44576bd4cd47d0680d107d5191679dc1ee`。发布准备阶段的证据见
|
||||
> [1.3.5 验收证据](2026-09-09-135-call-observability-validation.md)(其 §8.6 列的"尚未执行"由本文逐项关闭,
|
||||
> 该文当时结论不追改)。本轮无生产/测试代码改动、未派子代理;原始日志在
|
||||
> `tests/outputs/135/publish-20260909/`(gitignore,不入库)。
|
||||
|
||||
## 1. 合并与合并后门
|
||||
|
||||
长跑全部在 tmux `pgw135-publish` 串行执行,`PYTHONUNBUFFERED=1`,命令末尾**不接管道**,每门独立 `.exit`。
|
||||
|
||||
| 步骤 | 结果与证据 |
|
||||
| --- | --- |
|
||||
| 远端占用复查(改动前) | `git ls-remote` 最高 `v1.3.4`;registry simple 索引按完整字面量 `polygateway-1.3.5` 无命中;Release API `/releases/tags/v1.3.5` 与包页面 1.3.5 均 **404**。origin/main = 本地 main = `af57f93`,无未审变更 |
|
||||
| 合并 | `--no-ff` 得 `ab00aa4`;`git diff HEAD feature/1.3.5-call-observability` **空**,即合并树与父会话已审的 `433039b` 逐字一致;`merge.log/.exit` |
|
||||
| CLAUDE.md 纯格式 diff | 合并前仅对该文件 `git stash push`(未 stash 全部),合并后 `stash pop` 回原样;全程未提交、未覆盖 |
|
||||
| `make lint` | **exit 0**,ruff 无自动改动,import-linter **1 kept / 0 broken**;`lint.log/.exit` |
|
||||
| `make test` | **1605 passed / 23 skipped / 108 deselected,95% 覆盖,282.48s,exit 0**;`daily.log/.exit`。skip 与 deselected **不计通过** |
|
||||
| 真实网关 e2e 冒烟(显式 slow) | `pytest tests/e2e/test_smoke_gateway.py -m slow -v`:**4 passed,98.00s,exit 0**(流式/非流式/结构化 json/结构化模型阶梯);`smoke-gateway.log/.exit` |
|
||||
| Redis 时间语义(显式 slow) | `pytest tests/contracts tests/integration -m slow -q`:**18 passed / 159 deselected,1138.67s,exit 0**;`redis-time.log/.exit` |
|
||||
|
||||
未补全模型矩阵(用户已批准):本版对 `thinking.py`、能力表、wire 片段、缓存 key 公式、重试/限流/熔断语义一字未改,
|
||||
故 1.3.4 的型号级结论在其**原有边界**内继续成立,连同其 FAIL/UNKNOWN/不可达/缺轮一并继承,不因本版升格。
|
||||
本版**已变更**的统计/SQL/PG 口径不复用旧统计,全部由 1.3.5 新测试承担(36 列契约、终态行、真实 PG 30 项)。
|
||||
|
||||
## 2. push / tag / 构建 / 上传
|
||||
|
||||
| 步骤 | 结果 |
|
||||
| --- | --- |
|
||||
| push main | `af57f93..ab00aa4`,exit 0;`push-main.log/.exit` |
|
||||
| tag | `git tag -a v1.3.5`,注释 tag 对象 `ab780cf36a8c6b2de316c0b194dcaa9485e8172f`,解引用 = `ab00aa4`;push exit 0;`remote-refs-after.log` |
|
||||
| 构建 | 确认 `dist` 是项目内真实目录(非符号链接)后清掉旧 1.3.3/1.3.4 产物;`python -m build` exit 0,wheel + sdist 各一份 |
|
||||
| `twine check` | 两份均 **PASSED**,exit 0 |
|
||||
| 上传 | exit 0。凭据只从 tea 配置读入 `TWINE_PASSWORD` 环境变量,不进 argv/日志;**未重传任何已存在版本的字节** |
|
||||
|
||||
产物 SHA-256:
|
||||
wheel `c017599ea6bef69a485e1d35eab2ca89154cd33180ebd6dc759647097221a6b4`;
|
||||
sdist `87fd4151b77f55829b5d3206c5b75ac6c1c6bb8c94faa194955bf0db33ad0211`。
|
||||
|
||||
## 3. 独立下载、解包与安装后行为
|
||||
|
||||
全部在仓库外临时目录(`/tmp/pgw135-registry-*`)进行,不引入新依赖。
|
||||
|
||||
| 检查 | 结果 |
|
||||
| --- | --- |
|
||||
| registry 下载 | wheel exit 0;sdist 因私有索引不镜像 setuptools 需 `--no-build-isolation`(沿用 1.3.4 已记录的处置)exit 0 |
|
||||
| 字节一致 | 下载的两份 SHA-256 与本地构建**逐字相同** |
|
||||
| 包内源码 | `__init__.py` / `types.py` / `ports.py` / `telemetry/schema.py` / `middleware/telemetry.py` / `client.py` / `embedding.py` / `ocr.py` 八个文件 wheel、sdist 与已发布提交**散列全等**;sdist 内 README == 仓库 README |
|
||||
| 元数据 | wheel METADATA `Version: 1.3.5`、`Description-Content-Type: text/markdown`、正文 30465 字符,含《1.3.5 逻辑调用统计与失败诊断》与安装下界 `>=1.3.5,<2`;Project-URL 三项齐全 |
|
||||
| 安装后公共面(`pip install --target` + 仅从该目录导入) | `CallStats` 在 `__all__`、frozen、字段 `logical_call_id/attempts/total_latency_ms`;四类响应 `call_stats` 均为**末尾**字段且默认 `None`;`record_llm_call` 实测 **36 参**,10 个新参全为 keyword-only 且**无默认值**;`len(schema.COLUMNS) == 36` |
|
||||
| 安装后终态行行为(离线,httpx MockTransport 固定 503,不发真实请求) | 3 条 `attempt` + **恰 1 条** `terminal_failure`;终态 `attempts=3`、`total_latency_ms` 非空、`cost IS NULL`、`usage_source='unavailable'`、`http_status_code` 为 NULL;尝试行三条均记真实 **503** 与 `error_body`,`error_type='TransientError'`;四行共享同一 `logical_call_id`;物理列 **37**(36 写入列 + `created_at`) |
|
||||
|
||||
## 4. 外部产物亲查(匿名,看到什么算什么)
|
||||
|
||||
| 目标 | 结果 |
|
||||
| --- | --- |
|
||||
| Release 创建 | POST **201**,id 22,正文 1898 字符逐字取自 CHANGELOG 1.3.5 段 |
|
||||
| [包页面](https://gitea.iomgaa.online/iomgaa/-/packages/pypi/polygateway/1.3.5) | 匿名 **200**;可见正文含安装命令与《1.3.5 逻辑调用统计与失败诊断》;页面有指向 `iomgaa/PolyGateway` 的仓库锚点 |
|
||||
| [v1.3.5 Release](https://gitea.iomgaa.online/iomgaa/PolyGateway/releases/tag/v1.3.5) | 匿名 **200**;可见文本含 `WHERE event_kind = 'terminal_failure'` 与「26 → 36」迁移说明 |
|
||||
| [Releases 列表](https://gitea.iomgaa.online/iomgaa/PolyGateway/releases) | 匿名 **200**,含 v1.3.5 |
|
||||
| simple 索引 | 匿名 **200**,1.3.5 的 wheel 与 sdist 均列出 |
|
||||
|
||||
## 5. issue 收尾
|
||||
|
||||
仅动 **#19 / #23**:各评论一条后关闭,随后**以 GET 读到的 state 为准**复核(不拿 PATCH 返回码推断),两者均 `closed`。
|
||||
评论内容点名:自建 recorder 需补 10 参或改 `**fields`(**装配期报错**)、PG manual 档补列、
|
||||
计失败改 `WHERE event_kind = 'terminal_failure'`、`AVG(latency_ms)` 按 `event_kind` 分组;
|
||||
并明确「新增失败统计列与终态行是**常规口径变更、不是异常字段**」,`http_status_code` 不可当成败判据;
|
||||
同时声明**未做全模型矩阵、不宣称所有模型或渠道可用**。#22 / #24 未触碰(复核仍 `open`、评论数 0)。
|
||||
|
||||
## 6. 真实失败与处置(原件保留,不改写成功)
|
||||
|
||||
| 失败 | 根因与处置 |
|
||||
| --- | --- |
|
||||
| 首次安装后终态验证 exit 1 | `sqlite3.OperationalError: no such column: id` —— 是**我的验证脚本**臆断了主键列名,实际第 37 个物理列是 `created_at`。改脚本 `ORDER BY rowid` 并把多出的物理列打印出来自证,**未改库代码**;失败日志保留 |
|
||||
| 首次 Release 创建 exit 1 | `KeyError: 'TOKEN'` —— 变量未 export 到子进程,**未发出任何 API 请求**(不是服务端拒绝)。改为 export 后 201 |
|
||||
| 包关联 POST **400** | 返回 `invalid argument`,与 1.3.4 同款。**不把 400 当成功、也不据此猜已关联**:另以认证 GET 包 API 读到 `repository.full_name = iomgaa/PolyGateway`,并匿名读到包页面上的仓库链接,两条实测证据才是关联成立的依据;既有正确关系不为重试而解绑 |
|
||||
|
||||
## 7. 残余缺口(不自称已关)
|
||||
|
||||
| 缺口 | 说明 |
|
||||
| --- | --- |
|
||||
| live 覆盖基线 | `metrics/call-telemetry-coverage.md` 的实际基线列仍待首次生产运行填入 |
|
||||
| 下游自建 recorder | 装配闸只证形状可被接受,证不了函数体真的落这些列(已写进 README/CHANGELOG/issue 评论) |
|
||||
| 模型矩阵 | 未重跑;1.3.4 的 FAIL/UNKNOWN/不可达/缺轮例外原样继承,不外推为「全模型可用」 |
|
||||
| 其余 e2e | 本轮 slow 只跑了 `test_smoke_gateway.py` 与 contracts/integration 的 Redis 时间语义,`tests/e2e/` 其余文件未跑 |
|
||||
| 用户文档站(Gitea Wiki) | 本轮未同步;`docs-convention.md` §2 清单待另行执行 |
|
||||
|
||||
conda 启动器既有 `RequestsDependencyWarning` 保留,不宣称零告警。本记录落在独立
|
||||
`docs/1.3.5-release-evidence` 分支,**不改动已发布的 main/tag/registry 产物**。
|
||||
@@ -0,0 +1,109 @@
|
||||
---
|
||||
type: finding
|
||||
node_id: finding:2026-09-10-136-call-deadline-validation
|
||||
title: "1.3.6 可选调用期限与取消结算验证记录"
|
||||
date: 2026-09-10
|
||||
---
|
||||
|
||||
# 1.3.6 可选调用期限与取消结算验证记录
|
||||
|
||||
> 范围:分支 `feature/1.3.6-call-budgets` 上的 T1–T3 三个行为提交(`1ff83bb` / `9474c76` / `da77b12`)与 T4 文档提交。设计 `research-wiki/designs/2026-09-09-136-call-budgets-design.md`(人类已批准),计划 `research-wiki/plans/2026-09-10-136-call-deadline.md`。
|
||||
> 本文件是**证据索引**:原始输出在 `tests/outputs/136/`(按纪律**不提交**),此处只记路径、命令、退出码与结论。
|
||||
> 环境:conda 环境 `PolyGateway`,**Python 3.12.13**(`conda run -n PolyGateway python -V` 实测)。所有 pytest/lint 命令均**不接管道**,退出码直取。
|
||||
> 版本号未 bump、未 tag、未发布——发布清单(CLAUDE.md §4.4.1)不在本轮范围。
|
||||
|
||||
## 1. 红绿证据索引
|
||||
|
||||
| 任务 | 阶段 | 证据文件 | 结果 |
|
||||
| --- | --- | --- | --- |
|
||||
| T1 取消结算 | 红/绿 | **未落盘**(见 §1.1 诚实说明) | 见 §1.1 |
|
||||
| T1 真实 Redis | 绿(本轮在 HEAD `da77b12` 上复跑) | `tests/outputs/136/t4/redis_cross_connection.txt` | `8 passed`,`exit=0` |
|
||||
| T2 期限 | 红(值域+形态) | `tests/outputs/136/t2/red_deadline.txt` | 收集期 `1 error`(`deadline.py` 缺席),`exit=2` |
|
||||
| T2 期限 | 绿(`test_deadline.py`) | `tests/outputs/136/t2/green_deadline.txt` | `22 passed`,`exit=0` |
|
||||
| T2 接线中途 | 绿 | `tests/outputs/136/t2/unit_contracts_midway.txt`、`unit_contracts_after_wiring.txt` | 各 `1560 passed, 17 skipped` |
|
||||
| T2 入口冒烟 | 绿 | `tests/outputs/136/t2/entry_smoke.txt` | 四个方法签名含 `call_deadline_s`;到期异常 `has retry_after_s: False`,`exit=0` |
|
||||
| T2 回归门 | 绿 | `tests/outputs/136/t2/final_unit_contracts.txt` | `1572 passed, 17 skipped`,`pytest_exit=0` |
|
||||
| T2 lint | 红→绿 | `tests/outputs/136/t2/lint.txt`(`Found 3 errors`,`lint_exit=2`)→ `lint_final.txt`(`All checks passed!` + `Contracts: 1 kept, 0 broken.`,`lint_exit=0`) | 修后绿 |
|
||||
| T2c 补测 | 红 pass1 | `tests/outputs/136/t2c/red_pass1_import_absent.txt` | `3 errors in 0.29s`(三个模块收集期 ImportError) |
|
||||
| T2c 补测 | 红 pass2 | `tests/outputs/136/t2c/red_pass2_real_reasons.txt` + `red_method_and_reasons.txt` | `32 failed`;分布见 §1.2 |
|
||||
| T2c 补测 | 绿 | `green_unit_after_hardening.txt`(`1530 passed`)、`green_client_recheck.txt`(`110 passed`,`EXIT=0`)、`green_recheck_client_embedding.txt`(`165 passed`)、`green_unit_contracts.txt`(`1592 passed, 17 skipped`)、`final_unit_contracts.txt`(`1606 passed, 17 skipped`) | 全绿 |
|
||||
| T2c lint | 绿 | `tests/outputs/136/t2c/lint.txt` | `All checks passed!` + `Contracts: 1 kept, 0 broken.` |
|
||||
| T3 `Retry-After` | 红 | `tests/outputs/136/t3/red.txt` | `6 failed, 8 passed, 129 deselected` |
|
||||
| T3 `Retry-After` | 绿 | `green_file.txt`(`143 passed`)、`green_unit_contracts.txt`(`1606 passed, 17 skipped`) | 全绿 |
|
||||
| T3 lint | 绿 | `tests/outputs/136/t3/lint.txt` | `All checks passed!` + `Contracts: 1 kept, 0 broken.` |
|
||||
|
||||
`tests/outputs/136/t2/lsp_noise_refutation.txt` 与 `t2c/lsp_noise_refutation.txt` 记录编辑器 LSP 报的 import/属性告警属环境噪声(`pydantic` 在 conda 环境可解析),不是代码缺陷。
|
||||
|
||||
### 1.1 T1 的证据形态(诚实说明)
|
||||
|
||||
T1(`1ff83bb`)的**先红后通过证据产生于当时的会话工具输出,未落盘为 `tests/outputs/136/t1/` 文件**。本文件不追认那次输出,只登记两项**当下可复核**的替代证据:
|
||||
|
||||
| 替代证据 | 内容 |
|
||||
| --- | --- |
|
||||
| 提交 `1ff83bb` 的 diff | 三个源文件 + 四个测试文件共 213 插入;测试侧含 S3/S7(取消结算按 `est`)、S5-dead(仍 `0`)、S8(`RuntimeError` 逃逸仍 `0`)、真实 usage 恰为 0 的成功仍 `0` 四组断言 |
|
||||
| 本轮在 HEAD `da77b12` 上重跑真实 Redis | `pytest tests/integration/test_redis_cross_connection.py -q` → `8 passed`,`exit=0`(`tests/outputs/136/t4/redis_cross_connection.txt`) |
|
||||
|
||||
结论口径:**T1 的“红”只有会话内证据、无归档文件**;T1 的“绿”在当前 HEAD 上已被真实 Redis 复现证实。
|
||||
|
||||
### 1.2 T2c 两趟红证据的方法说明(诚实说明)
|
||||
|
||||
T2c 的红证据是在**基线 `1ff83bb`(T2 之前)**的 `git worktree --detach` 检出上取的,用 `PYTHONPATH=<worktree>/src` 覆盖 editable `.pth`(已实测 `polygateway` 加载自 worktree 且 `deadline.py` 缺席):
|
||||
|
||||
| 趟次 | 做法 | 结果 |
|
||||
| --- | --- | --- |
|
||||
| pass1 | 用例原样跑 | 三个测试模块**收集期** ImportError(`CallDeadlineExceeded` 不存在)→ `3 errors`。只证明符号缺席,**没有执行到函数体** |
|
||||
| pass2 | 仅把缺失的**导入符号**替换成**本地占位异常类**(shim),让函数体真正跑起来 | `32 failed`:**27 条“参数/属性不存在”**(`chat()` 9、`embed()` 4、`GatewaySettings.__init__()` 3、`GatewaySettings.call_deadline_s` 属性 3、`recognize_text()` 2、`GatewayClient.__init__()` 2、`parse_layout()`/`OcrClient.__init__()`/`EmbeddingClient.__init__()`/`GatewayClient._call_deadline_s` 各 1)+ **5 条 `Failed: DID NOT RAISE ValueError`** |
|
||||
|
||||
**该 shim 是一次性本地脚手架,未提交、不在任何分支上**;它只替换导入符号,不改被测源码。故 pass2 的红**是针对预 T2 源码的真实失败原因分布**,而非构造错误——但读者需知这份红**无法从仓库检出复现**,只能从上表与 `red_method_and_reasons.txt` 复核。
|
||||
|
||||
## 2. 命令与退出码
|
||||
|
||||
| 命令(前缀均为 `conda run -n PolyGateway python -m`,lint 为 `make lint`) | 何时 | 结果 |
|
||||
| --- | --- | --- |
|
||||
| `pytest tests/unit/test_deadline.py -q` | T2 红 | `exit=2`(collection error,符合预期) |
|
||||
| `pytest tests/unit/test_deadline.py -q` | T2 绿 | `22 passed`,`exit=0` |
|
||||
| `pytest tests/unit tests/contracts -q` | T2 门 | `1572 passed, 17 skipped`,`exit=0` |
|
||||
| `pytest tests/unit/test_client.py tests/unit/test_embedding.py tests/unit/test_ocr_client.py tests/unit/test_config.py -q -rf -k "…deadline…"` | T2c 红 | `32 failed`(基线 worktree,见 §1.2) |
|
||||
| `pytest tests/unit tests/contracts -q` | T2c 门 | `1606 passed, 17 skipped`,`exit=0` |
|
||||
| `pytest tests/unit/test_openai_compat.py -q -rf -k RetryAfterNonFinite` | T3 红 | `6 failed, 8 passed, 129 deselected` |
|
||||
| `pytest tests/unit/test_openai_compat.py -q` | T3 绿 | `143 passed` |
|
||||
| `pytest tests/unit tests/contracts -q` | T3 门 | `1606 passed, 17 skipped`,`exit=0` |
|
||||
| `pytest tests/integration/test_redis_cross_connection.py -q` | T1/T4 复跑 | `8 passed`,`exit=0` |
|
||||
| `make lint`(ruff + import-linter) | T2/T2c/T3 收尾 | `All checks passed!`;`Contracts: 1 kept, 0 broken.`(新 `polygateway.deadline` 层在内) |
|
||||
|
||||
T4(本次文档提交)**不含行为变更**,故未跑测试;仅新增上表最后一行的真实 Redis 复跑作为 §1.1 的替代证据。
|
||||
|
||||
## 3. 豁免索引(哪些门没跑,为什么,谁来兜)
|
||||
|
||||
| 未执行项 | 原因 | 兜底责任 |
|
||||
| --- | --- | --- |
|
||||
| `pytest -m slow`(真实网关 e2e、Redis 时间语义变体) | 成败取决于外部服务当下状态,默认被 `addopts = "-m 'not slow'"` 排除;计划 §5 明确本轮不跑 | **发布清单(CLAUDE.md §4.4.1)第 4 步**,合并 main 后统一执行 |
|
||||
| `tests/e2e/` 四个文件 | 同上,本版零付费调用 | 同上 |
|
||||
| 真实网关的期限行为实测 | 期限用例用真实事件循环时钟+假 transport 构造,余量 4–10 倍,不依赖网关 | 发布清单第 4 步的 e2e 顺带覆盖;**本轮无真实网关证据** |
|
||||
| 模型能力矩阵复验 | 本版未触碰推理/能力表 | 不适用 |
|
||||
| 跨 Python 版本验证 | 见 §4 残余三 | 未兜底,登记为残余 |
|
||||
|
||||
真实 Redis **不在豁免之列**:T1 已证、本轮在 HEAD 上复跑(`8 passed`),未以 memory 后端冒充。
|
||||
|
||||
## 4. 残余风险(三条,逐条复述设计 §12)
|
||||
|
||||
| # | 残余 | 诚实口径 |
|
||||
| --- | --- | --- |
|
||||
| 1 | 取消结算按 `est` 保留预扣 | 这是**保守选择,不是“上游已计费”的证明**。库无法知道端口已开始的那次调用是否真的产生了计费用量;方向定为宁多扣不空退(多扣只损失本窗口一点额度,空退会让已计费用量绕过闸门)。真实 usage 已知(含恰为 0)与已判 `SourceDead` 的 `0` 不被覆写;未分类异常逃逸仍按 `0`,属**已知残留,本版不动** |
|
||||
| 2 | 清理期自抛 `TimeoutError` 时**无终态遥测行** | 与 1.3.5 的裸 `TimeoutError` 穿透**同一口径**(这条路径一直存在、一直没有终态行),本版没有让它变坏;**但期限把这条路径常态化了**——启用期限后触发清理的频率上升,其可达性随之上升。`with_call_deadline` 的局部变量身份比较保证这种 `TimeoutError` **不会**被误标成 `CallDeadlineExceeded`(`test_deadline.py` 有断言钉住) |
|
||||
| 3 | 跨 Python 版本仅 3.12.13 有实证 | `asyncio.timeout` 的 `cm.expired()`/`uncancel()` 行为与清理期异常传播是探针在 **3.12.13 单一版本**上实测的;3.13+的行为未验证。库声明 3.12+,故这是**真实的验证缺口**,不是理论担忧 |
|
||||
|
||||
## 5. 发布清单第 4 步结果(2026-09-10,main `b0ab39e`)
|
||||
|
||||
| 门 | 结果 | 证据 |
|
||||
| --- | --- | --- |
|
||||
| `make lint`(合并后 main) | 通过(ruff + import-linter 1 kept 0 broken) | 会话内输出 |
|
||||
| 全套件(unit+contracts+integration) | **1683 passed, 23 skipped, exit=0** | `tests/outputs/136/release/full-gate.log` + `.exit` |
|
||||
| slow 交集子集(Redis 时间语义变体 + 真实网关冒烟) | **22 passed, exit=0**(21 分钟) | `tests/outputs/136/release/slow-scoped.log` + `.exit` |
|
||||
| `test_thinking_live.py` 全模型能力矩阵 | **未跑——按 2026-09-10 人类批准的新规则豁免**:本版 diff 零触碰 `thinking.py`/能力注册表/相关 e2e 设施,复用 1.3.4/1.3.5 已登记矩阵证据;当晚多渠道额度耗尽,全矩阵只会产出超时链噪声。规则变更已写入 CLAUDE.md §4.4.1 第 4 步与 §4.6(提交 `b0ab39e`);被中途终止的全量尝试日志留存于 `slow-gate.log` 备查 | 本表 |
|
||||
|
||||
## 6. 未在本轮做的事
|
||||
|
||||
- 不实现 issue #24(长尾对冲):未获批准,代码与文档均无 hedge 机制,本版**不缓解 #24**。
|
||||
- 不修 `ResultInvalid`/`RequestRejected` 已计费坏结果仍退全款:属另一族记账语义,未批准,登记待立 issue。
|
||||
- 不 bump 版本号、不打 tag、不构建、不上传 registry、不同步 wiki——全部留给发布清单。
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
type: finding
|
||||
node_id: finding:2026-09-10-24-hedged-requests-validation
|
||||
title: "issue #24 长尾对冲请求与裸生成时间验证报告"
|
||||
date: 2026-09-10
|
||||
---
|
||||
|
||||
# issue #24 长尾对冲请求与裸生成时间验证报告
|
||||
|
||||
> 计划:`research-wiki/plans/2026-09-10-24-hedged-requests.md`;设计:`research-wiki/designs/2026-09-10-24-hedged-requests-design.md`(H1–H8 全数获批)。
|
||||
> 分支 `feature/1.3.7-hedged-requests`;基线 main `166b286`(1.3.6);代码提交 `0a6d622`(T1)/`463eca3`(T2)/`adc0694`(T3),文档提交见本文件 git 历史。
|
||||
> 所有命令在 `PolyGateway` conda 环境执行,未接管道(退出码不失真)。
|
||||
|
||||
## 1. 红绿证据索引
|
||||
|
||||
证据目录 `tests/outputs/137/{t1,t2,t3}`(不提交,本地留存);每份日志末尾带 `EXIT_CODE=` 行。
|
||||
|
||||
| 任务 | 相位 | 证据文件 | 退出码 | 结果与失败形态 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| T1 端口事件 | 红(批次 A) | `t1/red-batch-a.log` | 1 | 4 failed,143 deselected——失败均为 `TypeError`(端口签名无 `first_token_event` 必填 kw),非断言值不符 |
|
||||
| T1 | 绿(批次 A) | `t1/green-batch-a.log` | 0 | 147 passed |
|
||||
| T1 | 绿(unit 全套) | `t1/green-unit-full.log` | 0 | 1550 passed,既有断言一行未改 |
|
||||
| T2 CallStats 三字段 | 红(批次 B–F) | `t2/red-batch-b-f.log` | 1 | 10 failed,373 deselected——字段不存在(`TypeError`/`AttributeError`)与计时字段恒 0 |
|
||||
| T2 | 绿(批次 B–F) | `t2/green-batch-b-f.log` | 0 | 10 passed,373 deselected |
|
||||
| T2 | 绿(unit+contracts) | `t2/green-unit-contracts.log` | 0 | 1621 passed,17 skipped |
|
||||
| T3 对冲编排 | 红(编排+配置守卫,计划批次 G/H) | `t3/red_batches_d_h.txt` | 1 | 25 failed——14×RetryMW 缺 `hedge_after_s`、2×GatewayClient 缺参、4×GatewaySettings 缺属性、4×守卫未抛 ValueError、1×client 缺属性,均为未实现形态 |
|
||||
| T3 | 绿(同上 25 用例) | `t3/green_batches_d_h_run1.txt` | 0 | 25 passed,2.55s |
|
||||
| T3 | 绿(批次 I 默认关闭回归) | `t3/green_full_unit_contracts.txt` | 0 | **1646 passed** = 基线 1621 + 新增 25,17 skipped,48s;既有断言一行未改 |
|
||||
| T3 | lint | `t3/lint.txt` | 0 | `make lint`(ruff --fix + import-linter)通过,Contracts: 1 kept,0 broken |
|
||||
|
||||
T3 各相位命令原文与退出码另见 `t3/commands.md`(该文件表头"批次 D–H"为执行批次流水号,对应计划 §5 的批次 G 对冲编排 + H 配置守卫,25 用例 = 15 编排 + 8 配置 + 2 client 入口校验)。
|
||||
|
||||
## 2. 批次与提交映射
|
||||
|
||||
| 提交 | 计划任务 | 覆盖批次(计划 §5) | 关键断言 |
|
||||
| --- | --- | --- | --- |
|
||||
| `0a6d622` | T1 端口事件(H2) | A | 流式首 token 置位事件;非流式永不置位;`None` 不观测行为不变;漏传必填 kw 即 `TypeError` |
|
||||
| `463eca3` | T2 三字段与计时(H3+H8) | B–F | 三字段默认 0/0/False;chat 计时排除退避与准入;结构化重问取最后一轮;embedding 批次累加;OCR 单次;缓存命中恒 0 |
|
||||
| `adc0694` | T3 对冲编排(H1/H4/H5/H6) | G、H、I | 触发两形态;异源排除;准入失败静默;赢家 settle 实际/输家 settle est;`hedge_cancelled` 标签;输家不喂熔断;`attempts==2` 无任务泄漏;外部取消两路穿透;deadline 切断对冲树;原路后发先至;两败计一次预算;两 429 免预算退 stall;混合失败计一次;配置守卫四路 |
|
||||
|
||||
## 3. 实测核对(文档承诺 vs 运行实测,本文件交付时复核)
|
||||
|
||||
| 承诺 | 核对方式 | 实测结果 |
|
||||
| --- | --- | --- |
|
||||
| `CallStats` 三字段默认值 | `CallStats(logical_call_id='x', attempts=1, total_latency_ms=5)` 仅旧三参构造 | `hedges=0`、`generation_ms=0`、`hedge_won=False`,构造不炸 |
|
||||
| `hedge_won` 语义 | 现读 `middleware/retry.py:498-578` | 赢家裁定后 `hedge_won=winner is hedge`;两败轮次照登 `hedge_won=False` 且裸生成时间无归属不记;触发但准入失败不计 `hedges` |
|
||||
| 两键 env 解析 | `GatewaySettings.from_env(env=...)` 注入两源 + `LLM__HEDGE__AFTER_S=8`/`MAX_EXTRA=2` | 解析出 `hedge_after_s=8.0`/`hedge_max_extra=2`,两源照常加载(3 段键未被当源字段);`MAX_EXTRA=2` 触发装配期 warning「v1 仅单路对冲生效」;不设两键时 `None`/`1` |
|
||||
| 版本号不动 | `pyproject.toml` 与 `src/polygateway/__init__.py` | 两处均 `1.3.6`,本计划不 bump、不 tag、不发布 |
|
||||
| 文档行号 | README/CHANGELOG/.env.example 接入点 | 均按交付时现读行号接入,未沿用计划旧行号 |
|
||||
|
||||
## 4. 豁免索引(未跑项与去向)
|
||||
|
||||
| 未跑项 | 理由 | 去向 |
|
||||
| --- | --- | --- |
|
||||
| `tests/e2e/`(slow) | 本计划不新增、不跑真实网关用例;`tests/e2e/conftest.py` 包装 transport 已同步转发 `first_token_event` | 发布清单第 4 步按 diff 交集选子集(本 diff 触及公开入口与 e2e 设施,e2e 冒烟届时在交集内) |
|
||||
| Redis 契约/integration(slow) | T1–T3 未触碰限流 Lua、`Permit` 端口、`backends/**` 与 `tests/contracts/**`(计划 §2 不改清单);对冲结算复用既有 `settle_and_release` 路径,无新 Lua 行为可测 | 同按发布交集规则判断;diff 已触及 retry/限流结算路径,**Redis 时间语义变体届时在交集内**(计划 §5 末句已登记) |
|
||||
| `test_thinking_live.py` 全模型矩阵 | 未动 `thinking.py`/能力注册表 | 不在交集,复用最近一次有效矩阵证据 |
|
||||
|
||||
## 5. 残余复述(设计 §10 与计划 §6 的已批准口径)
|
||||
|
||||
| 项 | 口径 |
|
||||
| --- | --- |
|
||||
| 非流式「挂起 vs 慢生成」 | 物理不可分(无中途信号),只能靠阈值取值(建议源 p50 数倍)与 `hedge_max_extra` 上限控制误对冲;分位数触发为未来扩展 |
|
||||
| 输家取消止不住上游计费 | est 保留只是闸内保守记账,不是上游真实计量的计量;文档只写「可能已计费」,不写「浪费上限 = est」 |
|
||||
| 竞速误贴标签 | 外部取消与对冲取消同时到达时,输家行可能误贴 `cancelled`/`hedge_cancelled`;两任务同消、记账方向一致(est 保留),不造成结算或熔断错误,可接受 |
|
||||
| `hedge_max_extra` v1 单路 | 值域 [1,3] 接受,>1 仅装配期 warning,运行期恒单路(对冲任务恒传 `first_token_event=None`,不再触发梯次);若未来审定应为 `ValueError`,改 `check_hedge_assembly` 一处 + 批次 H 一条断言 |
|
||||
| 挂起源不喂熔断的反向代价 | 持续挂起的源不会因对冲输家被熔断标记;「挂起率」只能靠 `hedge_cancelled` 遥测行统计(同 `logical_call_id` join 还原) |
|
||||
|
||||
## 6. 结论
|
||||
|
||||
## 7. 发布清单第 4 步结果(2026-09-10,main `a04d87e`)
|
||||
|
||||
| 门 | 结果 | 证据 |
|
||||
| --- | --- | --- |
|
||||
| `make lint`(合并后 main) | 通过(ruff + import-linter 1 kept 0 broken) | 会话内输出 |
|
||||
| 全套件(unit+contracts+integration) | **1727 passed, 23 skipped, exit=0** | `tests/outputs/137/release/full-gate.log` + `.exit` |
|
||||
| slow 交集子集(Redis 时间语义变体 + 真实网关冒烟;交集判据:本版动了重试/退避/取消路径与公开入口) | **22 passed, exit=0**(21 分钟) | `tests/outputs/137/release/slow-scoped.log` + `.exit` |
|
||||
| `test_thinking_live.py` 全模型能力矩阵 | **未跑——按 2026-09-10 生效的交集规则豁免**:本版 diff 零触碰 `thinking.py`/能力注册表/相关 e2e 设施,复用 1.3.4/1.3.5 已登记矩阵证据 | 本表 |
|
||||
|
||||
T1–T3 全部行为变更具备先红后绿证据(§1),默认关闭回归门成立(1646 passed 且既有断言一行未改),lint 与 import-linter 契约通过。文档承诺经运行实测核对(§3)。版本 bump、tag、发布与 slow/e2e 交集子集**不在本计划内**,按 CLAUDE.md §4.4.1 另行执行。
|
||||
@@ -465,6 +465,13 @@
|
||||
"relation": "implements",
|
||||
"evidence": "复用既有26字段,四种行来源与无推理成败NULL;不增加生产数据面",
|
||||
"added": "2026-09-09T06:32:06.527808+00:00"
|
||||
},
|
||||
{
|
||||
"source": "plan:2026-09-09-135-call-observability",
|
||||
"target": "design:2026-09-09-135-call-observability-design",
|
||||
"relation": "implements",
|
||||
"evidence": "1.3.5 逻辑调用统计与结构化失败诊断实施计划落地已批准设计",
|
||||
"added": "2026-09-09T13:42:00.298915+00:00"
|
||||
}
|
||||
]
|
||||
}
|
||||
+10
-6
@@ -1,8 +1,8 @@
|
||||
# Research Wiki 索引
|
||||
|
||||
> 自动生成,更新时间:2026-09-09 06:32 UTC
|
||||
> 自动生成,更新时间:2026-09-09 18:01 UTC
|
||||
|
||||
## design (42)
|
||||
## design (43)
|
||||
|
||||
- [1.3.4 推理意图与测试证据设计](designs/2026-09-09-134-thinking-contracts-design.md) `design:2026-09-09-134-thinking-contracts-design`
|
||||
- [2026-07-20-m1-core-design](designs/2026-07-20-m1-core-design.md) `design:2026-07-20-m1-core-design`
|
||||
@@ -24,6 +24,7 @@
|
||||
- [2026-08-19-issue14-admission-wait-policy-design](designs/2026-08-19-issue14-admission-wait-policy-design.md) `design:2026-08-19-issue14-admission-wait-policy-design`
|
||||
- [2026-08-24-issue15-telemetry-pool-lifecycle-design](designs/2026-08-24-issue15-telemetry-pool-lifecycle-design.md) `design:2026-08-24-issue15-telemetry-pool-lifecycle-design`
|
||||
- [2026-09-04-reasoning-effort-design](designs/2026-09-04-reasoning-effort-design.md) `design:2026-09-04-reasoning-effort-design`
|
||||
- [2026-09-09-135-call-observability-design](designs/2026-09-09-135-call-observability-design.md) `design:2026-09-09-135-call-observability-design`
|
||||
- [est_tokens 解耦: 拆分限流预扣与遥测用量兜底(issue #2)](designs/est-tokens-decoupling.md) `design:est-tokens-decoupling`
|
||||
- [GatewaySettings 装配校验补齐(第二轮)](designs/settings-invariants-round-2.md) `design:settings-invariants-round-2`
|
||||
- [GatewaySettings 跨字段不变量守卫的生效范围](designs/settings-invariant-guards.md) `design:settings-invariant-guards`
|
||||
@@ -47,9 +48,11 @@
|
||||
- [调用方自定义维度设计(issue #11)](designs/issue11-caller-dimensions.md) `design:issue11-caller-dimensions`
|
||||
- [采样参数透传设计(issue #4)](designs/sampling-params.md) `design:sampling-params`
|
||||
|
||||
## finding (15)
|
||||
## finding (17)
|
||||
|
||||
- [1.3.4 T0–T4 与 T7 确定性验证](findings/2026-09-09-134-thinking-contracts-validation.md) `finding:2026-09-09-134-thinking-contracts-validation`
|
||||
- [1.3.4 推理契约验证与发布准备](findings/2026-09-09-134-thinking-contracts-validation.md) `finding:2026-09-09-134-thinking-contracts-validation`
|
||||
- [1.3.5 T2/T3/T4 验收证据:36 列遥测、失败终态、PG 存储兼容与变异矩阵](findings/2026-09-09-135-call-observability-validation.md) `finding:2026-09-09-135-call-observability-validation`
|
||||
- [1.3.5 发布完成与外部验收:合并后门、registry 产物与页面亲查](findings/2026-09-09-135-release-completion.md) `finding:2026-09-09-135-release-completion`
|
||||
- [2026-07-20-m2-soak-workload](findings/2026-07-20-m2-soak-workload.md) `finding:2026-07-20-m2-soak-workload`
|
||||
- [2026-07-21-m25-acceptance](findings/2026-07-21-m25-acceptance.md) `finding:2026-07-21-m25-acceptance`
|
||||
- [2026-07-21-p6-soak-baseline](findings/2026-07-21-p6-soak-baseline.md) `finding:2026-07-21-p6-soak-baseline`
|
||||
@@ -65,9 +68,10 @@
|
||||
- [P7 OCR soak 验收: 99.73% 与 13 不变量全 PASS](findings/p7-ocr-soak.md) `finding:p7-ocr-soak`
|
||||
- [推理开关与 reasoning_tokens: 供应商实测与业界做法](findings/2026-08-02-thinking-switch-and-reasoning-tokens.md) `finding:2026-08-02-thinking-switch-and-reasoning-tokens`
|
||||
|
||||
## plan (37)
|
||||
## plan (38)
|
||||
|
||||
- [1.3.4 推理契约实施计划](plans/2026-09-09-134-thinking-contracts.md) `plan:2026-09-09-134-thinking-contracts`
|
||||
- [1.3.5 逻辑调用统计与结构化失败诊断实施计划](plans/2026-09-09-135-call-observability.md) `plan:2026-09-09-135-call-observability`
|
||||
- [2026-07-20-m1-core-plan](plans/2026-07-20-m1-core-plan.md) `plan:2026-07-20-m1-core-plan`
|
||||
- [2026-07-20-m2-distributed-plan](plans/2026-07-20-m2-distributed-plan.md) `plan:2026-07-20-m2-distributed-plan`
|
||||
- [2026-07-21-m25-resilience-plan](plans/2026-07-21-m25-resilience-plan.md) `plan:2026-07-21-m25-resilience-plan`
|
||||
@@ -111,7 +115,7 @@
|
||||
|
||||
## schema (1)
|
||||
|
||||
- [表结构: llm_calls(遥测 26 字段)](schemas/llm-calls.md) `schema:llm-calls`
|
||||
- [表结构: llm_calls(遥测 36 字段)](schemas/llm-calls.md) `schema:llm-calls`
|
||||
|
||||
## metric (2)
|
||||
|
||||
|
||||
@@ -156,3 +156,7 @@
|
||||
- [2026-09-09 05:39 UTC] 重建索引: 98 篇页面
|
||||
- [2026-09-09 06:32 UTC] 新增边: schema:llm-calls --implements--> design:2026-09-09-134-thinking-contracts-design
|
||||
- [2026-09-09 06:32 UTC] 重建索引: 98 篇页面
|
||||
- [2026-09-09 13:42 UTC] 新增边: plan:2026-09-09-135-call-observability --implements--> design:2026-09-09-135-call-observability-design
|
||||
- [2026-09-09 13:42 UTC] 重建索引: 100 篇页面
|
||||
- [2026-09-09 16:49 UTC] 重建索引: 101 篇页面
|
||||
- [2026-09-09 18:01 UTC] 重建索引: 102 篇页面
|
||||
|
||||
@@ -17,3 +17,17 @@ date: 2026-07-20
|
||||
| live 覆盖 | 逐轮 PASS/FAIL/UNCOVERED、计划轮数与缺轮分别统计;必需单元不因 pytest exit 0 自动放行 | 未执行;UNKNOWN/缺轮/skip 不能记PASS |
|
||||
|
||||
复用 schema:llm-calls(无新字段/DDL),证据索引见 `findings/2026-09-09-134-thinking-contracts-validation.md`。独立错误取证只在 tests 内存,Markdown 只记录白名单安全摘要与布尔校验,不使用生产遥测旁路补失踪尝试。生产埋点仍是 TelemetryEmitter 单点出口。
|
||||
|
||||
## 1.3.5 三类行与逻辑调用覆盖
|
||||
|
||||
本版把覆盖度的计量单位从"一次尝试"改成"一次逻辑调用"——此前结构化耗尽、embedding/OCR 的无源与准入拒绝根本没有任何行,"每次调用必录"在这几条路径上是不成立的。
|
||||
|
||||
| 指标 | 确定性阈值/证据 | 实际 live 基线 |
|
||||
| --- | --- | --- |
|
||||
| 领域失败的终态行覆盖 | chat 结构化耗尽、embedding/OCR 的 `no_sources`、准入拒绝、`retry_exhausted`、尝试外取消各恰有 **1** 条;单测以真实临时 SQLite 抓实 | 待首次实际运行,不填伪百分比 |
|
||||
| 至多一条(I3) | 同一逻辑调用重复进出口不产生第二条(`claim_terminal`);recorder 写失败仅 warning,故 SQL 可见 ≤ 1 | 待首次实际运行 |
|
||||
| 非领域异常 0 条(I4) | 编程错(如 `KeyError`)不写终态行、原样传播、分类不被改写 | 待首次实际运行 |
|
||||
| 诊断列保真 | 中转把 529 改写成 503 则记 503(**不猜回**);空 `str()` 的 httpx 超时类落 `cause_type`;成功行五列 NULL | 待首次实际运行 |
|
||||
| 存储兼容 | 真实 PG 沙箱: auto 追加十列、manual 裁剪写入、旧行新列恒 NULL、新旧进程混写共存 | 2026-09-09 实跑 30 passed(见下方证据索引) |
|
||||
|
||||
本版**新增十列**(见 schema:llm-calls),不新建表、不新增依赖。证据索引见 `findings/2026-09-09-135-call-observability-validation.md`;覆盖度真实基线仍待首次生产运行填入,不写伪百分比。
|
||||
|
||||
@@ -0,0 +1,455 @@
|
||||
---
|
||||
type: plan
|
||||
node_id: plan:2026-09-09-135-call-observability
|
||||
title: "1.3.5 逻辑调用统计与结构化失败诊断实施计划"
|
||||
date: 2026-09-09
|
||||
---
|
||||
|
||||
# 1.3.5 逻辑调用统计与结构化失败诊断实施计划
|
||||
|
||||
> 设计:`research-wiki/designs/2026-09-09-135-call-observability-design.md`,**人类于 2026-09-09 正式批准**(§10 六项批准项全数获批)。
|
||||
> 计划审核门:Claude 自审 + **独立模型替代审查**(用户已同意以独立模型替代 Codex 那一道,结论仍须逐条核验后就地修订);plan 无人类门,审毕直接执行。
|
||||
> 目标:把治理单位从"一次尝试"补齐到"一次逻辑调用"——四种响应带 `CallStats`,遥测补 10 列诊断/归因字段与 `terminal_failure` 行(issue #19、#23)。
|
||||
> 方案:设计方案 B——每调用一个显式传递的私有可变上下文 + 领域异常下沉到单一遥测出口;复用既有 Emitter / schema / 三条治理循环,不新增表、不新增依赖、不引入追踪平台。
|
||||
> 技术:Python 3.12+、asyncio、frozen dataclass、`inspect.signature`、pytest + FakeClock + MockTransport、临时 SQLite、真实 PG 沙箱、ruff、import-linter。
|
||||
> 基线 HEAD:`a81cc91`(分支 `feature/1.3.5-call-observability`,工作区仅 `CLAUDE.md` 既有 markdown 格式差异与未跟踪 `.pi/`,两者本计划一律不动、不暂存)。
|
||||
|
||||
本计划不涉及 `reference/` 参考实现迁移(治理主循环、Lua 限流、熔断状态机、退避公式一字不改),**保真校验不适用**。不得变更:重试预算与退避、429 免预算与 stall 算法、取消结算、限流/熔断语义、推理能力表与档位解析、缓存 key 公式、`#22`/`#24` 的调度机制。
|
||||
|
||||
## 1. 边界、授权与执行纪律
|
||||
|
||||
| 项目 | 固定边界 |
|
||||
| --- | --- |
|
||||
| 唯一 writer | 一工作区一 writer;父会话负责前台委派与审核派发。用户已授权本轮无需逐步请示,1.3.X 合并/发布授权沿用;跨到 1.4、新公共决策或验证豁免须停下确认 |
|
||||
| 公共承诺 | 只做设计已批准的四项公共面变更:`CallStats` 导出、四响应新增字段、`TelemetryRecorder` 扩 10 参、`terminal_failure` 行;**不新增其它公共 API、不改既有字段名与位置** |
|
||||
| 依赖铁律 | `ports.py`/`types.py`/`errors.py` 保持最内层,`types.py` 不 import 任何实现层;`middleware/` 只依赖端口与内核;`transports/`、`telemetry/`、`backends/` 互不依赖(import-linter 执法) |
|
||||
| 取消 | `CancelledError` 永不吞没;终态写**取消优先、不 shield、不开后台任务**;快照冻结是同步动作,不 await |
|
||||
| 降级方向 | recorder 写失败仍只 warning;限流/熔断后端仍 fail-closed;装配期签名不符 → **当场报错**(不是 warning) |
|
||||
| 证据与秘密 | 不打印 `.env`、token、Authorization;不提交 `.pi/`、`tests/outputs/`、reference;命令输出只记安全路径、状态与退出码 |
|
||||
| 证据复用 | 已发布 1.3.4 的有效证据(能力矩阵、live 轮次、发布外部验证)**直接引用不重建**;本版**不补跑未变更的模型能力矩阵**,无需新增付费调用 |
|
||||
|
||||
Skill 纪律:T0 已执行 `writing-plans`;T1–T3 行为变更执行 `test-driven-development`(先失败后通过的证据须落在本会话工具输出里);T1 触及运行时数据落库,执行 `structured-logging`;每次提交执行 `commit`(英文祈使标题、无 AI 签名、显式路径暂存);T4 前执行 `requesting-code-review` 与 `verification-before-completion`,收到意见执行 `receiving-code-review`;异常先 `systematic-debugging` 定根因。
|
||||
|
||||
## 2. 文件职责与不变接缝
|
||||
|
||||
| 动作 | 精确路径 | 职责 |
|
||||
| --- | --- | --- |
|
||||
| 修改 | `src/polygateway/types.py` | 新增 `CallStats`、私有 `_CallContext`、`CallOperation`/`EventKind` 词表;`ChatRequest` 追加内部上下文字段;四种响应追加 `call_stats` |
|
||||
| 修改 | `src/polygateway/__init__.py` | 导出 `CallStats`(`__all__` 保序插入);版本号在 T4 统一 |
|
||||
| 修改 | `src/polygateway/ports.py` | `TelemetryRecorder.record_llm_call` 追加 10 个 keyword-only 无默认值参数(唯一签名事实源) |
|
||||
| 修改 | `src/polygateway/telemetry/schema.py` | `SQLITE_DDL` / `PG_DDL` / `SQLITE_BACKFILL` / `_PG_BACKFILL_DECLS` / `COLUMNS` **五处**同序同增 10 列,追加在物理列末尾(`PG_BACKFILL`、`_ALTER_BY_BACKEND`、`telemetry_schema_sql` 自动派生,不手改) |
|
||||
| 修改 | `src/polygateway/middleware/telemetry.py` | Emitter 构造期注入 `scope` + 装配闸;三个 emit 入口收 `operation` 与领域异常对象;诊断字段单一提取 helper;`event_kind`;终态出口 helper;`TelemetryMW` 只保留 cache_hit |
|
||||
| 修改 | `src/polygateway/middleware/structured.py` | 有界错误说明常量与拼装函数改为库内可复用(重问反馈与遥测说明同一口径,数值只一份) |
|
||||
| 修改 | `src/polygateway/middleware/retry.py` | transport 调用前登记一次尝试;`_emit` 传异常对象与 `operation="chat"` |
|
||||
| 修改 | `src/polygateway/middleware/cache.py` | `_serialize` 剔除 `call_stats`;`_rehydrate` 显式覆盖为 `None`(防历史 dict 冒充) |
|
||||
| 修改 | `src/polygateway/client.py` | 自存 `now`/`emitter`;`chat` 创建上下文、附加统计、领域失败与取消经单一终态出口 |
|
||||
| 修改 | `src/polygateway/embedding.py` | `embed` 拥有上下文(空输入 0 尝试 0 遥测行);逐批传递;终态请求摘要按 §3.5 |
|
||||
| 修改 | `src/polygateway/ocr.py` | 上下文在 `image` 校验通过后创建;两公开方法各自给 `operation`;错误文本保留类名前缀(经出口的显式策略参数) |
|
||||
| 修改 | `src/polygateway/transports/openai_compat.py` | `_status_to_error` 增 keyword-only `operation`;三个调用点显式给值:`:512`(`embed` 非 200)传 `"embedding"`(修正历史误标)、`:527`(流式 chat)与 `:625`(非流式 chat)传 `"chat"`;`ocr_text`/`parse`/`download_result` 词表不动。**归属以本表为准**:设计 §5 把 `:527` 也叙述成 `embed()` 的分支,实测 `:527` 属 `_complete_stream`(流式 chat)、`embed` 只有 `:512` 一处;勿按设计行号把两条 chat 失败回退误标成 embedding |
|
||||
| 修改 | `tests/unit/test_types.py`、`tests/unit/test_ports.py` | `CallStats`/上下文纯行为、端口签名实测(含无默认值与 keyword-only) |
|
||||
| 修改 | `tests/unit/test_telemetry.py` | 列序/列数、三类行语义、诊断保真、装配闸、SQLite 新旧表、`_record_minimal` 字段字典 |
|
||||
| 修改 | `tests/unit/test_retry.py`、`test_client.py`、`test_cache.py`、`test_structured.py` | 计数/计时/缓存命中/结构化耗尽终态与历史 dict 防护 |
|
||||
| 修改 | `tests/unit/test_embedding.py`、`test_ocr_client.py`、`test_openai_compat.py`、`test_monkey_ocr.py` | 多批、空输入、双 HTTP 一次尝试、`exc.operation` 修正、200 失败行 |
|
||||
| 修改 | `tests/unit/test_pricing.py`、`tests/unit/test_usage_source_domain.py` | **仅 Emitter 调用点机械迁移**(`TelemetryEmitter(` 补 `scope=`、`emit_*` 补 `operation=`、`emit_terminal_failure` 去 `latency_ms` 改传 `stats`);两文件的 `_MemoryRecorder` 是 `**fields` 形态,recorder 签名不动 |
|
||||
| 修改 | `tests/integration/test_postgres_telemetry.py` | PG auto 追加 / manual 缺列裁剪 / 旧行 NULL / 新旧进程混写(真实 `pg_sandbox`) |
|
||||
| 修改 | `README.md`、`CHANGELOG.md`、`.env.example`、`research-wiki/ARCHITECTURE.md`、`research-wiki/schemas/llm-calls.md`、`research-wiki/metrics/call-telemetry-coverage.md` | 字段数实测改写、cap 覆盖面澄清、SQL 迁移五项、终态行语义 |
|
||||
| 新建(验收时) | `research-wiki/findings/2026-09-09-135-call-observability-validation.md` | 红绿、变异、失败与豁免索引,≤300 行;原始输出留 `tests/outputs/135/` |
|
||||
|
||||
**不改**:`errors.py`(异常类与分类逐字不动,不加可变字段)、`middleware/admission.py`、`middleware/ratelimit.py`、`middleware/breaker.py`、`sources.py`、`thinking.py`、`providers.py`、`telemetry/sqlite.py`、`telemetry/postgres.py`(两个 recorder 靠 `**fields` + `schema.COLUMNS` 自动吃到新列,逻辑零改动)、`transports/monkey_ocr.py`(双 HTTP 仍在同一 transport 调用内,不拆)。若实现时发现必须突破本清单,先说明设计依据与最小原因交父会话核定,不顺手改。
|
||||
|
||||
## 3. 跨任务接口(可执行定义,禁止占位)
|
||||
|
||||
### 3.1 统计内核(`types.py`)
|
||||
|
||||
`types.py` 需新增 `import uuid`、`from collections.abc import Callable` 与 `from typing import Literal`(`Mapping`/`Any` 已在;全文现无 `Literal`)。
|
||||
|
||||
**运行时求值约束(`types.py` 无 `from __future__ import annotations`,实测 :1-17)**:`Callable` 与 `Literal` 必须是**运行时 import**,不得放进 `TYPE_CHECKING`(`_CallContext.__init__` 的函数注解与 `CallOperation` 别名都在 def/赋值时求值);且 `_CallContext` 必须**定义在 `ChatRequest`(:339)之前**,否则 `call_context: _CallContext | None` 的类注解在类创建时即 `NameError`。
|
||||
|
||||
上下文是**每调用一个实例**的单任务对象:chat 重试、结构化重问、embedding 分批都在同一任务内串行推进,故计数无需锁;严禁提升为 client 实例属性。
|
||||
|
||||
```python
|
||||
CallOperation = Literal["chat", "embed", "recognize_text", "parse_layout"]
|
||||
CALL_OPERATIONS: tuple[CallOperation, ...] = ("chat", "embed", "recognize_text", "parse_layout")
|
||||
EventKind = Literal["attempt", "cache_hit", "terminal_failure"]
|
||||
EVENT_KINDS: tuple[EventKind, ...] = ("attempt", "cache_hit", "terminal_failure")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CallStats:
|
||||
"""一次公开调用的统计快照;第三方合成响应的 `None` 表示未知,不得伪造 0。"""
|
||||
|
||||
logical_call_id: str
|
||||
attempts: int
|
||||
total_latency_ms: int
|
||||
|
||||
|
||||
class _CallContext:
|
||||
"""私有可变逻辑调用上下文:只持计数、单调时钟与终态去重位,不做 I/O。"""
|
||||
|
||||
__slots__ = ("logical_call_id", "_now", "_started", "_attempts", "_terminal_claimed")
|
||||
|
||||
def __init__(self, *, now: Callable[[], float]) -> None: ...
|
||||
|
||||
def register_attempt(self) -> None:
|
||||
"""transport 调用前登记一次尝试(含免预算 429 与端口本地拒绝)。"""
|
||||
|
||||
def snapshot(self) -> CallStats:
|
||||
"""同步冻结当前快照;绝不 await,可多次调用。"""
|
||||
|
||||
def claim_terminal(self) -> bool:
|
||||
"""首次 True、其后 False:保证每逻辑调用至多写一条终态行。"""
|
||||
```
|
||||
|
||||
`ChatRequest` 追加(末尾,`dataclasses.replace` 保留同一引用;`StructuredMW` 的 replace 已核对不重建它):
|
||||
|
||||
```python
|
||||
call_context: _CallContext | None = field(default=None, compare=False, repr=False)
|
||||
"""库内部逻辑调用上下文;`None` = 库内现场构造的请求,遥测 logical_call_id 落 NULL。"""
|
||||
```
|
||||
|
||||
`LLMResponse`、`EmbeddingResponse`、`OcrTextResult`、`OcrLayoutResult` 各追加**末尾**字段:
|
||||
|
||||
```python
|
||||
call_stats: CallStats | None = None
|
||||
```
|
||||
|
||||
### 3.2 遥测出口(`middleware/telemetry.py`)
|
||||
|
||||
Emitter 是全库唯一 `record_llm_call` 调用点,本版新增的一切诊断/归因取值也只在这里定型。
|
||||
|
||||
```python
|
||||
class TelemetryEmitter:
|
||||
def __init__(
|
||||
self,
|
||||
recorder: TelemetryRecorder,
|
||||
*,
|
||||
scope: str,
|
||||
pricing: PricingTable | None = None,
|
||||
text_cap: int | None,
|
||||
) -> None:
|
||||
"""`scope` 构造期注入(三个 client 各一行),使三类行都带池名;不拿 source_name 顶替。
|
||||
|
||||
同处执行装配闸 `_assert_recorder_shape(recorder)`。
|
||||
"""
|
||||
|
||||
async def emit_attempt(
|
||||
self,
|
||||
*,
|
||||
request: ChatRequest,
|
||||
source: SourceConfig,
|
||||
call_id: str,
|
||||
latency_ms: int,
|
||||
response: LLMResponse | None,
|
||||
error: PolyGatewayError | str | None,
|
||||
reasoning_applies: bool,
|
||||
operation: CallOperation,
|
||||
class_prefixed_error: bool = False,
|
||||
) -> None:
|
||||
"""`event_kind='attempt'`;失败异常对象在此提取 status/cause/body 四列。"""
|
||||
|
||||
async def emit_cache_hit(
|
||||
self, *, request: ChatRequest, response: LLMResponse, operation: CallOperation
|
||||
) -> None:
|
||||
"""`event_kind='cache_hit'`;attempts/total_latency_ms 列恒 NULL。"""
|
||||
|
||||
async def emit_terminal_failure(
|
||||
self,
|
||||
*,
|
||||
request: ChatRequest,
|
||||
call_id: str,
|
||||
error: PolyGatewayError | str,
|
||||
operation: CallOperation,
|
||||
stats: CallStats,
|
||||
class_prefixed_error: bool = False,
|
||||
) -> None:
|
||||
"""`event_kind='terminal_failure'`;`latency_ms` 与 `total_latency_ms` 同取 `stats`。"""
|
||||
```
|
||||
|
||||
`emit_terminal_failure` **不再收 `latency_ms`**(同一冻结快照供两列,避免双时钟微差)。诊断提取收敛为一个纯 helper,只读领域异常的既有属性,**不遍历任意对象、不猜正文**:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class _ErrorFields:
|
||||
"""一行遥测的错误列;未知一律 None。"""
|
||||
|
||||
error: str | None
|
||||
error_type: str | None
|
||||
cause_type: str | None
|
||||
http_status_code: int | None
|
||||
error_body: str | None
|
||||
|
||||
|
||||
def _error_fields(
|
||||
error: PolyGatewayError | str | None,
|
||||
*,
|
||||
event_kind: EventKind,
|
||||
class_prefixed: bool,
|
||||
) -> _ErrorFields:
|
||||
"""三种入参形态的唯一定型点(设计 §5/§6)。"""
|
||||
```
|
||||
|
||||
判定规则(测试逐条钉死):
|
||||
|
||||
| 入参 | error | error_type | cause_type / http_status_code / error_body |
|
||||
| --- | --- | --- | --- |
|
||||
| `None` | None | None | 全 None |
|
||||
| `str`(如既有 `"cancelled"`) | 原样 | None | 全 None(**不解析字符串猜诊断**) |
|
||||
| 领域异常,`event_kind='attempt'` | 见下 | `type(exc).__name__` | `type(exc.__cause__).__name__ or None` / `exc.status_code` / `summarize_body` 已产出的 `exc.body_text or None` |
|
||||
| 领域异常,`event_kind='terminal_failure'` | 见下 | `type(exc).__name__` | **全 None**(C1:不搬最后一次 attempt 的状态与正文冒充整池归因) |
|
||||
|
||||
error 文本:`str(exc)` 为空退回 `type(exc).__name__`;`class_prefixed=True` 时前置 `"{类名}: "`(OCR 既有 metric 归组口径);取消路径一律传字符串(attempt 行沿用既有 `"cancelled"`,终态行用同一明确取消文案),故 `error_type` 与其余三列均 NULL;`ResultInvalidError` 且为终态时并入有界结构化说明:
|
||||
|
||||
```python
|
||||
def _structured_detail(exc: ResultInvalidError) -> str:
|
||||
"""结构化耗尽的有界说明:`repair=` 至多 200 字符 + `validation=` 至多 3 条×200 字符;
|
||||
**不含 raw_text**(模型正文预算已由 attempt 行的 response 列承担),两段以 ` | ` 拼接。"""
|
||||
```
|
||||
|
||||
复用 `structured.py` 的既有规则所有者,不复制数值:把 `_MAX_FEEDBACK_ERRORS`/`_MAX_ERROR_CHARS` 改名为 `MAX_FEEDBACK_ERRORS`/`MAX_ERROR_CHARS`,`_format_errors` 改名 `format_bounded_errors(errors: Sequence[str]) -> str`(行为逐字不变),`StructuredMW` 与 `_structured_detail` 两个消费者共同引用。
|
||||
|
||||
装配闸(设计 §7,C3):
|
||||
|
||||
```python
|
||||
def _assert_recorder_shape(recorder: TelemetryRecorder) -> None:
|
||||
"""装配期一次 `signature.bind` 形状校验:不执行写入,只证明该形状能被接受。
|
||||
|
||||
参数名从 `TelemetryRecorder.record_llm_call` 的协议签名派生(不手抄第四份清单),
|
||||
绑定用哨兵 `None`,不读真实请求数据。`**kwargs`(VAR_KEYWORD)自动通过;
|
||||
`TypeError` → `ValueError` 装配期报错;不可 inspect(C 实现等)同样按配置错误报错。
|
||||
"""
|
||||
```
|
||||
|
||||
终态唯一出口(公开边界 helper,三个 client 共用):
|
||||
|
||||
```python
|
||||
async def emit_terminal_once(
|
||||
emitter: TelemetryEmitter | None,
|
||||
*,
|
||||
request: ChatRequest,
|
||||
context: _CallContext,
|
||||
error: PolyGatewayError | str,
|
||||
operation: CallOperation,
|
||||
class_prefixed_error: bool = False,
|
||||
) -> None:
|
||||
"""去重(claim_terminal)+ 同步冻结快照 + best effort 写入。
|
||||
|
||||
`emitter is None` 或已写过 → 直接返回;写入侧异常按既有降级只落 warning;
|
||||
**`CancelledError` 原样传播**(取消优先,不 shield、不开后台任务)。
|
||||
"""
|
||||
```
|
||||
|
||||
`TelemetryMW` 相应收缩:删除其 `except (GatewayUnavailableError, GovernanceBackendError, SourceNotConfiguredError)` 与 `except CancelledError` 两个终态分支(改由 `GatewayClient.chat` 的边界统一写,避免两处同时写),保留 `cache_hit` 记录并传 `operation="chat"`。
|
||||
|
||||
### 3.3 `_record` 的 10 个新列取值(唯一定型点)
|
||||
|
||||
| 列 | attempt | cache_hit | terminal_failure |
|
||||
| --- | --- | --- | --- |
|
||||
| `scope` | 构造期注入值 | 同 | 同 |
|
||||
| `operation` | 调用点给定的四值之一,**绝不读 `exc.operation`** | 同 | 同 |
|
||||
| `logical_call_id` | `request.call_context.logical_call_id`,上下文缺席 → NULL(不造 ID) | 同 | 同 |
|
||||
| `event_kind` | `'attempt'` | `'cache_hit'` | `'terminal_failure'` |
|
||||
| `http_status_code` / `cause_type` / `error_body` | 见 §3.2 表(成功行全 NULL,成功不统一填 200) | 全 NULL | 全 NULL |
|
||||
| `error_type` | 该行自身错误类名 | NULL | 自身类名(`AllSourcesExhausted` / `CircuitOpenError` / `ResultInvalidError` / …) |
|
||||
| `attempts` / `total_latency_ms` | NULL | NULL | `stats` 两字段 |
|
||||
|
||||
其余 26 列口径逐字不变;终态行仍 `cost=None`、`usage_source='unavailable'`、token 0,不复制 attempt 的用量与费用。
|
||||
|
||||
### 3.4 schema 与端口(`telemetry/schema.py`、`ports.py`)
|
||||
|
||||
10 列按**同一顺序**追加进五处(`SQLITE_DDL`、`PG_DDL`、`SQLITE_BACKFILL`、`_PG_BACKFILL_DECLS`、`COLUMNS`),全部可空、无默认值,物理位置在现有末列 `reasoning_effort` 之后:
|
||||
|
||||
| 顺序 | 列名 | SQLite | Postgres |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | `scope` | TEXT | TEXT |
|
||||
| 2 | `operation` | TEXT | TEXT |
|
||||
| 3 | `logical_call_id` | TEXT | TEXT |
|
||||
| 4 | `event_kind` | TEXT | TEXT |
|
||||
| 5 | `http_status_code` | INTEGER | INTEGER |
|
||||
| 6 | `error_type` | TEXT | TEXT |
|
||||
| 7 | `cause_type` | TEXT | TEXT |
|
||||
| 8 | `error_body` | TEXT | TEXT |
|
||||
| 9 | `attempts` | INTEGER | INTEGER |
|
||||
| 10 | `total_latency_ms` | INTEGER | INTEGER |
|
||||
|
||||
`COLUMNS` 由 26 → 36,物理列(含 `created_at`)27 → 37。**不建索引、不改旧列、不回填旧行、不 ALTER 默认生产 PG**(manual 档仍裁剪写入)。`ports.py` 按同序追加 10 个 keyword-only 无默认值参数,docstring 更新为"36 字段冻结",并说明 `error_body` 沿用 `summarize_body` 上限、不进 `PGW_TELEMETRY_TEXT_CAP` 覆盖面。
|
||||
|
||||
### 3.5 三条链路的上下文所有权
|
||||
|
||||
| 链路 | 创建点 | 传递 | 附加统计 | 终态 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| chat | `GatewayClient.chat`:overlay/维度/档位三项校验**之后**(校验异常保持原行为,在统计边界外) | 放进 `ChatRequest.call_context`,洋葱各层经 `replace` 保留同一引用;`RetryMW._attempt` 在 `transport.complete` 前 `register_attempt()`(**上下文为 `None` 时跳过**——库内现场构造的 `ChatRequest` 不得因此报错) | 返回前 `dataclasses.replace(response, call_stats=ctx.snapshot())`(含缓存命中路径与内联遥测耗时) | `except PolyGatewayError` / `except CancelledError` 各调 `emit_terminal_once`,随后原样 `raise`;非领域异常不捕、无终态 |
|
||||
| embed | `EmbeddingClient.embed`:`texts` 类型校验与**调用方维度校验**(`validate_caller_dimensions`)之后;`expected_dim` 校验在 `_attempt` 内,本就在统计边界内 | 显式参数传到 `_embed_batch` → `_attempt`;`_emit` 现场构造的 `ChatRequest` 带 `call_context=ctx`;`transport.embed` 前登记 | 合并结果 `replace(..., call_stats=ctx.snapshot())`;**空输入返回真实 `attempts=0` 统计且不写任何遥测行** | 同上,`operation="embed"`;终态 messages = `<embed texts=N batches=M>` 占位 + 第一批(至多 `batch_size` 条、每条 200 字符,与逐批行同款构造) |
|
||||
| OCR | `OcrClient._call`:`image` 的 `bytes`/非空校验**通过之后**(M1 例外) | `_call` 拥有上下文并返回 `tuple[_AttemptOutcome, CallStats]`,两个公开方法据此附加统计;`transport` 调用前登记(layout 的 POST+ZIP GET 计 **1** 次) | `OcrTextResult` / `OcrLayoutResult` 的 `call_stats` | `_call` 内 `except PolyGatewayError` / `except CancelledError` 调 `emit_terminal_once`,`operation` 由公开方法传入(`recognize_text` / `parse_layout`),`class_prefixed_error=True`;`no_sources` 的 `raise` 在循环之前,`try` 必须**包住该 raise**,否则无源终态行写不出 |
|
||||
|
||||
`GatewayClient` 需自存 `self._now` 与 `self._emitter`(现未保存);统计生效与否**不由 telemetry 是否启用决定**(`emitter is None` 时统计照常,仅无行可写)。
|
||||
|
||||
## 4. 任务与提交点
|
||||
|
||||
### T0:设计批准状态、计划与基线(本任务)
|
||||
|
||||
- [x] 设计文档状态行改为"人类已批准(2026-09-09)",新建本计划(≤600 行)。
|
||||
- [x] 记录基线:`git status --short --branch`、`git log --oneline -3`、HEAD `a81cc91`;确认源码零差异,`CLAUDE.md` 既有 markdown 差异与 `.pi/` 一律不动、不暂存。
|
||||
- [ ] 基线验证(由执行者在 T1 之前跑一次,作为"先失败"的对照底):`make check`;`conda run -n PolyGateway pytest tests/unit -q` → 预期全绿。**基线计数以本会话实跑输出为准**(近期会话记录为 1385 项量级),不拿计划里的数字当期待值;实际失败照录,不改期待绕过。
|
||||
- [ ] 计划自审 + 独立模型审查(替代 Codex,用户已同意);意见逐条核验后就地修订。
|
||||
- [ ] wiki 登记:`.claude/tools/research_wiki.py add_entity/add_edge/rebuild_index`(design ↔ plan `implements`),登记前先确认工具不覆盖同路径已有文档。
|
||||
- [ ] 提交点:`docs: record approved call observability design and plan`(生产改动前的回滚点)。
|
||||
|
||||
### T1:统计内核与三条链路的逻辑边界(不动遥测列)
|
||||
|
||||
**文件**:`types.py`、`__init__.py`、`client.py`、`embedding.py`、`ocr.py`、`middleware/retry.py`、`middleware/cache.py`;测试 `tests/unit/test_types.py`、`test_client.py`、`test_retry.py`、`test_cache.py`、`test_embedding.py`、`test_ocr_client.py`、`test_structured.py`。
|
||||
|
||||
按 §3.1 与 §3.5 实现。先写测试并确认在旧实现上红(`AttributeError: 'LLMResponse' object has no attribute 'call_stats'` 不算行为红——先落一个"同一次调用的重试次数无处可读"的行为断言,再实现)。
|
||||
|
||||
| 红绿组 | 必须证明 |
|
||||
| --- | --- |
|
||||
| 计数 | 一次成功=1;失败重试=实际尝试数;免预算 429 计入;多源拒绝(端口本地拒绝)计入;结构化重问计入同一上下文且不重置;embedding 三批=3;OCR layout 双 HTTP=1;缓存命中=0;空输入=0 |
|
||||
| 逻辑 ID | 重试/重问/分批共享同一 `logical_call_id`;同一 client 并发两次调用互不串(`asyncio.gather` 两路各自计数与 ID);`parent_call_id` 语义不变 |
|
||||
| 计时(FakeClock) | 缓存 IO、退避 sleep、准入等待、重问、内联遥测收尾全部计入 `total_latency_ms`;毫秒/秒不混用(1.5s → 1500);`emitter=None` 时统计仍正确。**替身构造要求**:假缓存后端的 `get`/`set` 与假 recorder 的 `record_llm_call` 内部**必须主动推进 FakeClock**,否则"缓存 IO/内联遥测计入总耗时"的断言会退化成恒等于 0 的空转绿 |
|
||||
| 缓存不毒化 | `_serialize` 输出无 `call_stats` 键;手写含 `"call_stats": {...}` 的历史条目命中后 `response.call_stats is None`(**dict 不得冒充 `CallStats`**);缓存 key 逐字节不变(黄金值) |
|
||||
| 附加点 | 失败异常上**不附任何统计字段**(`hasattr(exc, "call_stats") is False`);`CancelledError` 类型与语义不变;非领域异常原样传播 |
|
||||
| 空输入 | `embed([])` 返回 `attempts=0`、真实 `logical_call_id`,且注入的内存 recorder **零行** |
|
||||
|
||||
**验证**:`conda run -n PolyGateway pytest tests/unit/test_types.py tests/unit/test_client.py tests/unit/test_retry.py tests/unit/test_cache.py tests/unit/test_embedding.py tests/unit/test_ocr_client.py tests/unit/test_structured.py -q` → 目标断言先红后绿,其余保留行为绿;`make check`(含 import-linter:`types.py` 不得引入实现层 import)。
|
||||
|
||||
- [ ] 提交点:`feat: track logical call statistics across governed calls`
|
||||
|
||||
### T2:遥测 10 列、诊断保真、scope/operation 与装配闸
|
||||
|
||||
**文件**:`telemetry/schema.py`、`ports.py`、`middleware/telemetry.py`、`middleware/structured.py`、`middleware/retry.py`、`transports/openai_compat.py`、三个 client 的 Emitter 构造行;测试 `tests/unit/test_telemetry.py`、`test_ports.py`、`test_openai_compat.py`、`test_monkey_ocr.py`、`test_structured.py`。
|
||||
|
||||
按 §3.2–§3.4 实现(本任务只产出 attempt / cache_hit 两类行的新列,终态行留 T3)。先写测试确认旧实现红:现状下 `error` 列是被 `str()` 压平的自由文本、无 `scope`/`operation` 列、旧签名 recorder 只落 warning。
|
||||
|
||||
| 红绿组 | 必须证明 |
|
||||
| --- | --- |
|
||||
| 列与 SQL | `len(COLUMNS) == 36`、物理列 37、五处列序一致(新建库与 ALTER 追加列序相同);`insert_sql` 两端语句更新;`telemetry_schema_sql` 输出与库内 DDL 同源 |
|
||||
| 端口 | `inspect.signature` 实测 10 个新参存在、keyword-only、无默认值;docstring 字段数与实测一致 |
|
||||
| 诊断保真 | 中转把 529 改写成 503 → 记 503(**不猜回 529**);直接 529 → 529;空 `str()` 的 Connect/Read/Write/PoolTimeout → error 落类名、`cause_type` 为对应 httpx 类名;`error_body` 为 `summarize_body` 摘要且不等于 `raw_text`;成功行五列 NULL(不填 200) |
|
||||
| operation | `embed` 非 200 → `exc.operation == "embedding"`(历史误标修正);新列 `operation` 恒为四值之一,与 `exc.operation` 无关、不随异常变化;`monkey_ocr` 的 `success != true` 带 **200** 的失败行 → `http_status_code == 200`(该列不可作失败判据) |
|
||||
| scope | attempt 与 cache_hit 行都带 scope;model/provider/source 未选出时仍留原空值 |
|
||||
| 字符串入参 | 取消 attempt 的 `"cancelled"` 仍原样落 `error`,其余四列 NULL(**不解析字符串**) |
|
||||
| 装配闸 | 旧签名 recorder → 构造 `TelemetryEmitter`(即三个 client 装配)**抛错**而非 warning;`**kwargs` recorder 通过;不可 inspect 的对象 → 配置错误报错;参数名确由协议签名派生(改协议后闸自动跟随的断言) |
|
||||
| 有界说明 | `format_bounded_errors` 重命名后 `StructuredMW` 反馈文案逐字不变(黄金串);常量只有一份定义 |
|
||||
|
||||
**验证**:`conda run -n PolyGateway pytest tests/unit/test_telemetry.py tests/unit/test_ports.py tests/unit/test_openai_compat.py tests/unit/test_monkey_ocr.py tests/unit/test_structured.py -q`;`make check`。
|
||||
|
||||
- [ ] 提交点:`feat: record scope, operation and failure diagnostics per row`
|
||||
|
||||
### T3:终态行、取消口径与统一出口
|
||||
|
||||
**文件**:`middleware/telemetry.py`(终态 helper 与 `TelemetryMW` 收缩)、`client.py`、`embedding.py`、`ocr.py`;测试 `tests/unit/test_telemetry.py`、`test_client.py`、`test_embedding.py`、`test_ocr_client.py`、`test_structured.py`、`test_retry.py`。
|
||||
|
||||
按 §3.2 的 `emit_terminal_once` 与 §3.5 的三条链路实现。先写测试确认旧实现红:结构化耗尽当前**没有任何失败行**;embedding/OCR 的无源、准入拒绝、重试耗尽与尝试外取消同样无终态行。
|
||||
|
||||
| 红绿组 | 必须证明 |
|
||||
| --- | --- |
|
||||
| 补漏 | chat 结构化耗尽、embedding/OCR 的 `no_sources`、准入拒绝、`retry_exhausted`、尝试外取消各恰有 **1** 条 `terminal_failure` |
|
||||
| 不变量 I3 | 领域失败每调用至多 1 条(recorder 写失败仅 warning,SQL 可见 ≤1);重复调用出口不产生第二条(`claim_terminal`);直接 `RequestRejectedError` / `ResultInvalidError` 现在**既有 attempt 错误行也有终态行**(400 密集负载错误行翻倍是已批准的下游可见变化) |
|
||||
| 非领域异常 | 编程错(如 `KeyError`)→ **0 条**终态行、原样传播、分类不被改写 |
|
||||
| 取消三路 | chat / embed / OCR 同策略尽力写一条(允许 0);**终态写的 await 上被取消 → `CancelledError` 传播**(不 shield、无后台任务);permit 与探针释放行为不变(`inflight == 0`) |
|
||||
| 归因 SQL | 设计 §5 那条 `WHERE logical_call_id = :lcid` 查询同时给出终态 reason 文案与逐源状态码/正文;终态行 `http_status_code`/`cause_type`/`error_body` 三列 NULL;结构化耗尽的终态 `error` 含有界 validation/repair 说明且**不含 `raw_text`** |
|
||||
| 行语义 | 三类行均带 scope;`event_kind` 可区分;`SUM(cost)` 不因终态行变化(终态 `cost IS NULL`、usage `unavailable`);`AVG(latency_ms)` 按 `event_kind` 分组的断言;终态 `latency_ms == total_latency_ms`(同一快照) |
|
||||
| 快照时机 | 终态快照不含自身写入耗时(FakeClock:写入内推进时钟,列值不变);成功响应快照含返回前已完成的内联遥测耗时 |
|
||||
| 摘要 | embedding 终态 messages 为占位 + 第一批截断文本,不含全量原输入;OCR 终态沿用 `<ocr:{kind} image_bytes=…>`,图像 bytes 不入库;OCR 终态 error 保留类名前缀 |
|
||||
|
||||
**验证**:`conda run -n PolyGateway pytest tests/unit -q`(全量单测,含上述文件);`make check`。
|
||||
|
||||
- [ ] 提交点:`feat: emit one terminal failure row per logical call`
|
||||
|
||||
### T4:旧测试机械核对、存储兼容、变异证据、文档与独立验证
|
||||
|
||||
**文件**:下述机械核对清单 + `tests/integration/test_postgres_telemetry.py` + 文档清单 + 验收 finding。
|
||||
|
||||
**一次机械核对(禁止分散反复修)**:本版真正的破坏面**不只是 recorder 签名,还有 Emitter 侧**(`TelemetryEmitter.__init__` 新增必填 `scope`、三个 `emit_*` 新增必填 `operation`、`emit_terminal_failure` 去 `latency_ms` 改收 `stats`),这些关键词不含 `record_llm_call`/`COLUMNS`。故全量清单用:
|
||||
|
||||
```bash
|
||||
grep -rn "record_llm_call\|COLUMNS\|_EXPECTED_COLUMNS\|TelemetryEmitter(\|emit_attempt(\|emit_cache_hit(\|emit_terminal_failure(" tests/ --include=*.py
|
||||
```
|
||||
|
||||
一轮改完再跑,不逐个文件试错。已核实的免改项:`tests/unit/test_backpressure.py:231` 的假 emitter 是 `emit_attempt(self, *args, **kwargs)`,兼容;`tests/integration/` 无 Emitter 构造点。已知点:
|
||||
|
||||
| 位置 | 动作 |
|
||||
| --- | --- |
|
||||
| `tests/unit/test_ports.py:99` `_DummyRecorder` | 显式签名补齐 10 参(它是"新签名可实现"的活证据,不改成 `**kwargs`) |
|
||||
| `tests/unit/test_ports.py:264` `TestTelemetryRecorderSignature` | 新参进 `no default` / keyword-only 参数化 |
|
||||
| `tests/unit/test_telemetry.py:40` `_EXPECTED_COLUMNS`、:220/:321/:597/:639/:647/:686-698 计数与尾部断言 | 26→36、27→37、尾部 10 列、旧表 backfill 目标列数 |
|
||||
| `tests/unit/test_telemetry.py:108` `_record_minimal` | 字段字典补 10 键(默认 NULL 形态) |
|
||||
| `tests/integration/test_postgres_telemetry.py:100` 字段字典 | 同上(该函数返回值被逐列断言消费,改动须与 `COLUMNS` 同序) |
|
||||
| `test_client.py` / `test_embedding.py` / `test_ocr_client.py` / `test_openai_compat.py` / `test_pricing.py` / `test_cache.py` / `test_usage_source_domain.py` 的 `_MemoryRecorder` | 均为 `**fields` 形态,**recorder 签名无需改**;只需核对断言里的字段计数与新列期望 |
|
||||
| `tests/unit/test_pricing.py`、`tests/unit/test_usage_source_domain.py` 的 **Emitter 调用点** | **须改**:`TelemetryEmitter(` 补 `scope=`、`emit_attempt(`/`emit_cache_hit(` 补 `operation=`;`test_usage_source_domain.py:297` 的 `emit_terminal_failure(...)` 同时缺 `scope`/`operation`/`stats` 且多传 `latency_ms`,不改必 `TypeError`(recorder 形态兼容 ≠ emitter 调用点兼容) |
|
||||
|
||||
**存储兼容**(真实 PG,复用 `tests/integration/conftest.py` 的 `pg_sandbox` / `pg_catalog_probe`,不新建沙箱设施;**不引用 `assert_no_leftovers`**——它是 `tests/integration/test_pg_sandbox.py:19` 的模块级 fixture,对 `test_postgres_telemetry.py` 不可见,上提它要改 §2 清单外的 `conftest.py`。本版不新增沙箱资源创建路径,残留风险与 1.3.4 逐字相同,由该文件既有用例覆盖,属可复用的既有证据):
|
||||
|
||||
- SQLite:新建库 37 列;旧表(1.3.4 形态 27 列)auto 档补齐 10 列且列序与新建库一致;manual 档不发 DDL、按现有列裁剪写入并发一条点名缺列的 warning;旧行新列为 NULL。
|
||||
- PG(验收取以下**四项**):auto 追加 10 列;**manual 缺列裁剪**写入成功且不抛(`_trim_columns` 路径);旧行新列为 NULL;新旧进程混写同一表(旧列集写入 + 新列集写入并存)。
|
||||
|
||||
**变异证据**(仓库外临时副本 + `PYTHONPATH=<副本>/src`,先确认 `polygateway.__file__` 指向副本;逐个变异 → 跑指定节点记 exit 1 与被杀断言 → 恢复校验散列 → exit 0;**绝不在主工作区改生产代码凑红**):
|
||||
|
||||
| 变异 | 必须被杀死的断言 |
|
||||
| --- | --- |
|
||||
| `register_attempt()` 移到 transport 调用之后的 `except` 分支外/内错位 | 失败重试与 429 计数断言 |
|
||||
| `ChatRequest` 上下文字段改为 `replace` 时新建实例(模拟上下文复制) | 重问/分批共享同一 `logical_call_id` 的断言 |
|
||||
| `_rehydrate` 去掉 `call_stats=None` 覆盖 | 历史 dict 冒充 `CallStats` 的断言 |
|
||||
| Emitter 入口提前 `str(exc)` 压平 | `error_type`/`http_status_code`/`error_body` 保真断言 |
|
||||
| 终态行复制最后一次 attempt 的 token/cost | 费用聚合与终态 `cost IS NULL` 断言 |
|
||||
| 去掉 `claim_terminal` 去重 | 每失败调用至多一条终态的断言 |
|
||||
| 装配闸改为捕获 `TypeError` 后 warning | 旧签名 recorder 装配期报错的断言 |
|
||||
|
||||
**文档同步(发布前必须同批;wiki 站点自 2026-08-02 下线,按 docs-convention §2 的下线期条款,承接方为 README / CHANGELOG / .env.example / ARCHITECTURE 四处)**:
|
||||
|
||||
| 位置 | 改什么 |
|
||||
| --- | --- |
|
||||
| `README.md:23` | "必录 26 字段" → 实测值(`len(inspect.signature(TelemetryRecorder.record_llm_call).parameters) - 1`,预期 36),不凭记忆 |
|
||||
| `README.md:26/399/455`、`ARCHITECTURE.md:592`、`.env.example:109` | `PGW_TELEMETRY_TEXT_CAP` 覆盖面仍是四处;明确 `error_body` 沿用 `summarize_body` 上限、`error` 的新增结构化说明另有独立限长,**二者都不在 cap 覆盖内** |
|
||||
| `README.md` 能力表 + 新增小节 | 四响应的 `call_stats` 读法;设计 §8 的 SQL 迁移**五项**(失败行数改判据、`error IS NOT NULL` 不再是判据、`AVG(latency_ms)` 须按 `event_kind` 分组、费用口径不变、失败行可能带 200) |
|
||||
| `ARCHITECTURE.md:565` 必录字段清单 + §7.8 补列一节 | 追加 10 列语义、`event_kind` 三态、终态行不变量 I3/I4、`operation` 与 `exc.operation` 是两个语义 |
|
||||
| `CHANGELOG.md` 未发布段 | 公共面四项 + 下游动作清单(点名"计失败调用改 `WHERE event_kind = 'terminal_failure'`"与自定义 recorder 的装配期报错) |
|
||||
| `research-wiki/schemas/llm-calls.md`、`metrics/call-telemetry-coverage.md` | 复用既有实体登记新列与三类行口径;真实覆盖基线待首次运行填,不写伪百分比 |
|
||||
|
||||
**验收命令与证据**:
|
||||
|
||||
| 检查 | 命令 / 要求 |
|
||||
| --- | --- |
|
||||
| 静态 | `make check`;`git diff --check`;`conda run -n PolyGateway python -m compileall -q src/polygateway` |
|
||||
| 日常全量 | `make test`(真实退出码、coverage ≥80%,连接依赖 skip 单列) |
|
||||
| 集成 | `conda run -n PolyGateway pytest tests/integration/test_postgres_telemetry.py tests/integration/test_governance_stack.py -q` |
|
||||
| 契约 | `conda run -n PolyGateway pytest tests/contracts -q`(限流契约随实现一起交付的既有套件) |
|
||||
| 变异 | 上表七项逐条 exit 1 → 还原 exit 0,日志留 `tests/outputs/135/` |
|
||||
| 独立验证 | 由**父会话前台派全新上下文 verifier**:只给批准设计、本计划、分支 diff 与命令,不给实现自评;至少覆盖"正确性/回归"与"下游可见变化/范围"两角度,Critical/Important 清零 |
|
||||
| slow | `pytest -m slow` 属发布清单第 4 步(CLAUDE §4.4.1),在合并 main 之后统一跑;**本版不新增 live 轮次、不补跑未变更的模型能力矩阵**,1.3.4 已有有效证据直接引用 |
|
||||
|
||||
长跑用 tmux,`PYTHONUNBUFFERED=1`,输出重定向到 `tests/outputs/135/`,命令后**不得接管道**(退出码失真),完成判定用 `wait`/PID 轮询,不用会自匹配的 `pgrep -f`。
|
||||
|
||||
- [ ] 提交点:`docs: document logical call telemetry and migration impact`(机械核对与存储兼容若需单独回滚点,可先提 `test: align telemetry fixtures with the 36-field recorder`,仍在 5 个提交内)
|
||||
|
||||
## 5. 测试矩阵 → 任务映射(设计 §9 逐族落位)
|
||||
|
||||
| 设计测试族 | 任务 |
|
||||
| --- | --- |
|
||||
| logical 计数(含空输入 0 尝试 0 行) | T1 |
|
||||
| 计时(缓存/退避/准入/重问/收尾、关 recorder) | T1(快照时机的终态部分在 T3) |
|
||||
| 失败与取消(三路同策略、终态 await 取消传播、非领域异常 0 行) | T3 |
|
||||
| 保真诊断(529/503、空超时文案、`exc.operation`、新列四值) | T2 |
|
||||
| 归因 SQL(终态 reason + 逐源现场、结构化说明无 `raw_text`) | T3 |
|
||||
| 行语义(三类行 scope、至多一条终态、费用不重复、`AVG` 分组) | T2(前两项)+ T3 |
|
||||
| 装配闸(旧签名报错、`**kwargs` 通过、不可 inspect) | T2 |
|
||||
| 存储兼容(SQLite 新旧表、PG manual/auto、旧行 NULL、混写) | T4 |
|
||||
| 变异(计数位置、上下文复制、缓存回放、提前压平、终态双计费用) | T4 |
|
||||
|
||||
**共享测试设施一律复用,不新建**:`tests/contracts/conftest.py:26` 与 `tests/unit/test_ocr_client.py:344` 的 `FakeClock`(注入 `now`/`sleep`,确定性计时);各测试文件既有的 `_MemoryRecorder`(`**fields`)与脚本化 MockTransport;`tmp_path` + 真实 `SQLiteRecorder(auto_migrate=...)`;`tests/integration/conftest.py` 的 PG 沙箱三件套。真实 LLM 输出如产生,按 CLAUDE §4.6 落 `tests/outputs/<module>/`。
|
||||
|
||||
## 6. 阻塞矩阵与发布交接
|
||||
|
||||
| 情形 | 本库可完成 | 不可自行宣称/处置 |
|
||||
| --- | --- | --- |
|
||||
| 无 PG 可用 | 单测与 SQLite 全部覆盖;PG 用例 skip 并单列 | 不得把 skip 记作通过;发布前须补跑或取人类具名豁免 |
|
||||
| 下游自定义 recorder 未知 | 装配闸 + 迁移文档 + `**kwargs` 兼容路径 | 不能声称"后端完全不受影响":其 schema、INSERT 字段与契约测试仍须同步 |
|
||||
| 设计外漏洞 | 独立记录实际文件与反例,交父会话核定 | 不顺手实施 #20 之外的 issue、不新增表/端口/deadline/hedging |
|
||||
| 发布 | 门全绿后按 CLAUDE §4.4.1 逐步执行(文档先行 → CHANGELOG 定版 → 双处版本号 → 合并 main → `make lint`/`make test`/`pytest -m slow` → tag → 构建 → 上传 → 下载验证 → Release + 挂仓库 + 页面核对) | 本计划**不复制**该清单,也不预先勾选任何发布步骤;只 bump 版本号不叫发布 |
|
||||
|
||||
## 7. 自审
|
||||
|
||||
- 设计每节可指到任务:§3→T1、§4→T1、§5→T2/T3、§6→T3、§7→T2、§8→T4、§9→§5 映射表、§10 六项批准项全部落在 T1–T3 的公共面改动内。
|
||||
- 无占位符与待定项:跨任务消费的类型(`CallStats`、`_CallContext`、`_ErrorFields`)、四个 emit 入口、两个 helper(`_assert_recorder_shape`、`emit_terminal_once`)、10 列取值表、10 列 DDL 类型均已写出可执行定义;被引用的 `PolyGatewayError`、`ResultInvalidError`、`summarize_body`、`format_bounded_errors`、`FakeClock`、`pg_sandbox` 全部指向既有实现或本计划已定义项。
|
||||
- 一致性核对:`emit_terminal_failure` 去掉 `latency_ms` 后,唯一调用者是 `emit_terminal_once`;`TelemetryMW` 的终态分支删除后 chat 终态只剩客户端边界一处;`operation` 只由调用点给定,链路上无任何位置读 `exc.operation`。
|
||||
- 未采纳项(异常上挂可变统计、终态搬运最后一次 attempt 的状态与正文、成功侧汇总行)理由在设计正文,本计划不复活。
|
||||
- 本计划编写过程**未运行 pytest、未做变异、未调用任何模型、未提交**;T0 基线验证与其后各任务的红绿证据由执行者在自己的会话内出示。
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
type: plan
|
||||
node_id: plan:2026-09-10-136-call-deadline
|
||||
title: "1.3.6 可选调用期限与取消结算修复实施计划"
|
||||
date: 2026-09-10
|
||||
---
|
||||
|
||||
# 1.3.6 可选调用期限与取消结算修复实施计划
|
||||
|
||||
> 设计:`research-wiki/designs/2026-09-09-136-call-budgets-design.md`,**人类于 2026-09-10 正式批准**(§9 七项批准项全数获批,H3 取 (a′):取消结算修复并入本版而非另立前置 issue)。
|
||||
> 计划审核门:Claude 自审 + 独立模型审查;plan 无人类门,审毕直接执行。
|
||||
> 目标:① 修好取消路径的 TPM 结算(§6.3 矩阵 S1–S7);② 给一次逻辑调用一条**可选**墙钟硬边界(issue #22),缺省 `None` 时行为逐字等于 1.3.5;③ 修 `Retry-After` 非有限值防御缺口。
|
||||
> 方案:设计 §3 方案 A——三个公开边界各一次 `asyncio.timeout`,配**局部变量身份比较**判据(探针实证:只看 `cm.expired()` 会把清理期自抛的 `TimeoutError` 误标成 deadline)。
|
||||
> 技术:Python 3.12+、asyncio、frozen dataclass、pytest + FakeClock + 真实 `asyncio.Event`、真实实验室 Redis、ruff、import-linter。
|
||||
> 基线 HEAD:`d2455e8`(分支 `feature/1.3.6-call-budgets`;工作区仅 `CLAUDE.md` 既有 markdown 差异、未跟踪 `.pi/` 与本轮两份文档,前两者一律不动、不暂存)。
|
||||
|
||||
**不实现 issue #24(长尾对冲)**:未获批准,任何提交、测试与文档均不得出现 hedge/对冲机制,也不得声称本版缓解 #24。
|
||||
|
||||
## 1. 边界、授权与执行纪律
|
||||
|
||||
| 项目 | 固定边界 |
|
||||
| --- | --- |
|
||||
| 唯一 writer | 一工作区一 writer;父会话负责前台委派与审核派发。1.3.X 合并/发布授权沿用;跨到 1.4、新公共面变化或验证豁免须停下确认 |
|
||||
| 公共面 | 只做设计 §9 已批准四项:新错误类 `CallDeadlineExceeded`、新配置键 `{SCOPE}__CALL_DEADLINE_S`、三个 client 构造参数 + 四个公开方法 keyword-only 参数、取消路径结算口径。**不新增其它键/端口方法/遥测列** |
|
||||
| 记账边界 | 只改**取消路径**的 `settle()` 入参取值(设计 §6.3 S3/S5/S7);成功、`RequestRejected`、`ResultInvalid`、`SourceDead` 四条既有路径与**未分类异常逃逸路径(S8,仍 `0`)**的结算值逐字不变;实现只能用**函数内局部阶段变量**,不得新增公开参数;限流 Lua、`Permit` 端口签名、幂等语义一律不动 |
|
||||
| 依赖铁律 | 新模块 `deadline.py` 只 import stdlib + `errors.py`;`middleware/` 仍只依赖端口与内核;import-linter 契约新增一层执法 |
|
||||
| 取消 | `CancelledError` 永不吞没;不引入 `shield`、不开后台任务;清理仍在 `finally`,允许超出期限 |
|
||||
| 降级方向 | 限流/熔断后端仍 fail-closed;遥测/缓存仍 warning 降级;非法期限值 → **当场 `ValueError`**(装配错误不属降级面) |
|
||||
| 证据与秘密 | 不打印 `.env`、token、Authorization;不提交 `.pi/`、`tests/outputs/`;命令输出只记路径、状态与退出码 |
|
||||
| 证据复用 | 复用既有 FakeClock / 假 transport / `settle_and_release` 出口 / 限流契约套件 / 真实 Redis 跨连接用例;**不重跑模型能力矩阵**,本版零付费调用 |
|
||||
|
||||
Skill 纪律:T0 已执行 `writing-plans`;T1–T3 行为变更执行 `test-driven-development`(先失败后通过的证据须落在本会话工具输出里);每次提交执行 `commit`(英文祈使标题、无 AI 签名、显式路径暂存);T4 前执行 `requesting-code-review` 与 `verification-before-completion`;异常先 `systematic-debugging` 定根因。
|
||||
|
||||
## 2. 文件职责与不变接缝
|
||||
|
||||
| 动作 | 精确路径 | 职责 |
|
||||
| --- | --- | --- |
|
||||
| 新建 | `src/polygateway/deadline.py` | `ensure_call_deadline()` 值域校验 + `with_call_deadline()` 单一硬边界(§3.1) |
|
||||
| 修改 | `src/polygateway/errors.py` | 追加 `CallDeadlineExceeded(PolyGatewayError)`(§3.2);`SCOPE_REASONS`/四分类**不动** |
|
||||
| 修改 | `src/polygateway/__init__.py` | `from polygateway.errors import ... CallDeadlineExceeded`;`__all__` 插在 `"CallStats"` 之后、`"CircuitOpenError"` 之前(现读 `:61-62`;该列表并非全字母序,头部 `DEFAULT_PROFILES`/`EFFORT_ORDER`/`Effort` 是既有例外,**不得顺手重排**) |
|
||||
| 修改 | `src/polygateway/config.py` | `GatewaySettings` 末尾追加 `call_deadline_s: float \| None = None`;`_load_call_deadline()`;`_validate_call_deadline()` 进 `__post_init__`(§3.3) |
|
||||
| 修改 | `src/polygateway/client.py` | `__init__` 追加 `call_deadline_s`(入口即校);`chat()` 追加 per-call 参数;`:398` 包裹;`from_settings` 透传 |
|
||||
| 修改 | `src/polygateway/embedding.py` | 同上三处(`:209` 包裹整次 `_embed_all`);`_attempt` 结算矩阵(§3.5) |
|
||||
| 修改 | `src/polygateway/ocr.py` | `__init__`/两个公开方法/`_call` 两级透传;`:274` 包裹 `_run`;`:449 settle_and_release(permit, 0)` **保持 0** |
|
||||
| 修改 | `src/polygateway/middleware/retry.py` | `_attempt` 结算矩阵(§3.5:`actual` 初值仍 `0` + 局部 `settlement_known`);`__call__` 循环、`backoff_delay`、`StallClock` 一字不动 |
|
||||
| 修改 | `src/polygateway/transports/openai_compat.py` | `_parse_retry_after`(`:111-119`)增 `math.isinf` 判据 + 一条 warning + 必填私有 kw `source_name`;同步唯一调用处 `_translate_429`(`:140`)(§3.6) |
|
||||
| 修改 | `pyproject.toml` | import-linter layers 在 `"polygateway.thinking"` 与 `"polygateway.providers : polygateway.sources"` 之间插入 `"polygateway.deadline"` 一行 |
|
||||
| 新建 | `tests/unit/test_deadline.py` | `deadline.py` 的值域与五种形态区分(§5 批次 A/B) |
|
||||
| 修改 | `tests/unit/test_retry.py` | 取消结算红绿(S1/S3/S4/S5)+ `FakeTransport` 加 `entered` Event |
|
||||
| 修改 | `tests/unit/test_embedding.py` | 取消结算(S7)、多批共享一份期限 |
|
||||
| 修改 | `tests/unit/test_ocr_client.py` | 取消结算恒 0(S6)、两个入口的期限与 per-call 校验 |
|
||||
| 修改 | `tests/unit/test_client.py` | chat 期限命中、终态遥测行、已计费成功被丢弃、注入钟无关 |
|
||||
| 修改 | `tests/unit/test_config.py` | 键未设/非法值 × env / 直接构造 / `dataclasses.replace` / 三个 `__init__` 直传 |
|
||||
| 修改 | `tests/unit/test_openai_compat.py` | F1 四类取值 |
|
||||
| 修改 | `tests/integration/test_redis_cross_connection.py` | 真实 Redis 上的取消结算契约(复用既有 `clients`/`_limiter`/`_client`/`ScriptedTransport`,**不改 Lua、不改契约套件**) |
|
||||
| 修改 | `CHANGELOG.md`、`README.md`、`.env.example` | 新键、新异常、对外承诺三句话(§4 C4) |
|
||||
| 新建 | `research-wiki/findings/2026-09-10-136-call-deadline-validation.md` | 红绿、命令、豁免索引,≤300 行 |
|
||||
|
||||
**不改**:`ports.py`(`Permit.settle` 签名与语义不动)、`middleware/admission.py`(`settle_and_release` 逐字不动)、`middleware/ratelimit.py`、`middleware/breaker.py`、`middleware/structured.py`、`middleware/cache.py`、`middleware/telemetry.py`、`telemetry/schema.py`(零新增列)、`backends/**`(含全部 Lua)、`sources.py`、`streaming.py`、`types.py`、`transports/monkey_ocr.py`(`:58` 不解析 `Retry-After`)、`tests/contracts/**`(复用现有 settle 契约,只跑不改)。若实施时发现必须突破本清单,先说明最小原因交父会话核定。
|
||||
|
||||
## 3. 跨任务接口(可执行定义,禁止占位)
|
||||
|
||||
### 3.1 `src/polygateway/deadline.py`
|
||||
|
||||
```python
|
||||
def ensure_call_deadline(value: object, origin: str) -> float | None:
|
||||
"""全装配路径共用的值域校验: None 或有限正数, 否则 ValueError(消息含 origin)。"""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
raise ValueError(f"{origin} 必须是 None 或有限正数秒: {value!r}") # bool 先判
|
||||
v = float(value)
|
||||
if not math.isfinite(v) or v <= 0:
|
||||
raise ValueError(f"{origin} 必须是有限正数秒: {value!r}") # NaN/inf/0/负
|
||||
return v
|
||||
|
||||
|
||||
async def with_call_deadline[T](aw: Awaitable[T], *, deadline_s: float | None, scope: str) -> T:
|
||||
if deadline_s is None:
|
||||
return await aw # 未启用: 不进上下文, 逐字旧路径
|
||||
inner_timeout: BaseException | None = None
|
||||
try:
|
||||
async with asyncio.timeout(deadline_s) as cm:
|
||||
try:
|
||||
return await aw
|
||||
except TimeoutError as exc:
|
||||
inner_timeout = exc # 体内(含清理路径)自抛, 非本层期限
|
||||
raise
|
||||
except TimeoutError as exc:
|
||||
if cm.expired() and exc is not inner_timeout:
|
||||
raise CallDeadlineExceeded(scope=scope, deadline_s=deadline_s) from None
|
||||
raise
|
||||
```
|
||||
|
||||
三条实现红线:① **校验先于构造 awaitable**(否则非法值抛错时遗留未 await 协程 → `RuntimeWarning` + 资源不释放);② 只传**相对时长**,绝不把注入 `now` 加偏移换算成绝对截止时刻;③ 身份比较**不可退化**为只看 `cm.expired()`——探针 `/tmp/pgw_deadline_probe.py` E1/E2 实测:清理路径自抛的 `TimeoutError` 会被只看 `expired()` 的写法改标成 `CallDeadlineExceeded`;`__cause__` 启发式同样失效(内层 `asyncio.timeout` 的 `TimeoutError` 其 `__cause__` 也是 `CancelledError`)。本机制**不新增公共配置、不开后台任务、不改异常对象**。
|
||||
|
||||
### 3.2 `errors.py` 新类(唯一定义点)
|
||||
|
||||
```python
|
||||
class CallDeadlineExceeded(PolyGatewayError):
|
||||
"""调用方设定的整体期限到期; 不是网关不可用、也不是源故障。"""
|
||||
|
||||
def __init__(self, *, scope: str, deadline_s: float) -> None:
|
||||
super().__init__(f"{scope} 调用期限 {deadline_s}s 到期")
|
||||
self.scope = scope
|
||||
self.deadline_s = deadline_s
|
||||
```
|
||||
|
||||
无 `retry_after_s`(期限到期不含"何时可再试",给 `0.0` 会按既定语义指示下游立刻重打饱和渠道);不进 `SCOPE_REASONS`;不属四分类。`__init__.py` 导出后,三个边界既有的 `except PolyGatewayError` 自动接住并写终态行——**遥测零改动**。
|
||||
|
||||
### 3.3 配置(`config.py`)
|
||||
|
||||
| 项 | 精确定义 |
|
||||
| --- | --- |
|
||||
| 键名 | `{SCOPE}__CALL_DEADLINE_S`(两段式,`"LLM__CALL_DEADLINE_S".split("__")` 长度 **2** ≠ 4,故 `_load_sources`(函数定义 `:380`,判据行 `:384`)天然跳过,**不必**加进 `_RESERVED_SEGMENTS`) |
|
||||
| loader | `_load_call_deadline(scope, env)`:`found = _first(env, f"{scope}__CALL_DEADLINE_S")`;`None → None`;否则 `ensure_call_deadline(_cast(found[1], "float", found[0]), found[0])`。**用 `_first` 不用 `_require`**(`_require` 会把未设当成配置缺失报错 = 破坏性变更);**origin 传实际命中的 env 键名 `found[0]`**,不传 `"GatewaySettings.call_deadline_s"`——否则 env 里写 `LLM__CALL_DEADLINE_S=0` 的人会拿到一条指向字段名的错误,在多 scope 部署里无法定位是哪个键(`_cast` 只能接住“不是数字”,`0`/负/`inf` 会穿过它) |
|
||||
| 字段 | `GatewaySettings` **末尾**追加 `call_deadline_s: float \| None = None`(有默认值,不扰动既有位置构造;`EmbeddingSettings.gateway` / `OcrSettings.gateway` 自动继承) |
|
||||
| 守卫 | `_validate_call_deadline()` 加入 `__post_init__`(`:186-194`)末位,实现体是 `object.__setattr__(self, "call_deadline_s", ensure_call_deadline(self.call_deadline_s, "GatewaySettings.call_deadline_s"))`——盖住直接构造与 `dataclasses.replace` 两条路;env 路已在 loader 里拿真键名报过错,此处重跑对合法值是幂等空操作 |
|
||||
| 装配透传 | `GatewaySettings.from_env`(`:349`)返回字典加 `call_deadline_s=_load_call_deadline(scope_u, env)`;`GatewayClient.from_settings`(`client.py:449`)传 `settings.call_deadline_s`;`EmbeddingClient.from_settings`(`embedding.py:584`)与 `OcrClient.from_settings`(`ocr.py:631`)传 `gw.call_deadline_s`;三个 `from_env` 签名不变 |
|
||||
| 不耦合 | **不校验** `call_deadline_s` 与 `timeout_s`/`stall_window_s` 的大小关系:期限短于单次超时是调用方的合法选择 |
|
||||
|
||||
### 3.4 三个接入点与 per-call 透传(唯一三处)
|
||||
|
||||
| 文件:行 | 改后 |
|
||||
| --- | --- |
|
||||
| `client.py:398` | `response = await with_call_deadline(self._handler(request), deadline_s=deadline, scope=self._scope)` |
|
||||
| `embedding.py:209` | 同款包住 `self._embed_all(...)`(**整次调用一份**,N 批共享) |
|
||||
| `ocr.py:274` | 同款包住 `self._run(...)` |
|
||||
|
||||
- 三处均在既有 `try` 之内、`_CallContext` 创建之后 → 到期照走 `except PolyGatewayError → emit_terminal_once`。
|
||||
- `StructuredMW._run_ladder`(`structured.py:67-98`)与 `_embed_batch`(`embedding.py:295`)**严禁**新建 scope:同级重问/分批共享同一份期限,否则期限被轮数放大 N 倍。
|
||||
- 三个 `__init__` 追加 keyword-only `call_deadline_s: float | None = None`,函数体首行即 `self._call_deadline_s = ensure_call_deadline(call_deadline_s, "<Class>(call_deadline_s=...)")`;`client.py` 需自存 `self._scope = scope`(现只传给 RetryMW,未自存)。
|
||||
- 四个公开方法(`chat`/`embed`/`recognize_text`/`parse_layout`)追加 keyword-only `call_deadline_s: float | None = None`;`None` = 继承装配值,正数 = 本次覆盖,**不提供"本次关闭"**。取值一行:`deadline = self._call_deadline_s if call_deadline_s is None else ensure_call_deadline(call_deadline_s, "<method>(call_deadline_s=...)")`,位置在既有输入校验之列、`_CallContext` 创建**之前**。
|
||||
- OCR 两个入口经 `recognize_text`/`parse_layout` → `_call` 形参透传;`_call` 内的校验须与 `image` 校验同列(`ocr.py:266-269`),即仍在 `_CallContext`(`:272`)之前。
|
||||
- `embedding.py:205-217` 的 `texts == []` 早返回在 `try` 之前,天然落在期限之外——保持原样,测试显式记一笔。
|
||||
|
||||
### 3.5 取消结算矩阵落到代码(设计 §6.3 的唯一实现形态)
|
||||
|
||||
`middleware/retry.py::_attempt`(对照现读行号)。人类只批准了“**取消路径**的结算口径”这一条,故 `actual` 初值**不得**改成 `est`——那会把语义泛化到一切未分类异常(`RuntimeError`、`KeyError` 逃逸),属未批准范围。改用**局部阶段变量**:
|
||||
|
||||
```python
|
||||
actual = 0 # 初值不动:未分类异常逃逸时仍逐字走 1.3.5 语义
|
||||
settlement_known = False # 局部变量:该刻库是否已算出确定结算(不进任何签名)
|
||||
```
|
||||
|
||||
| 现状行 | 现状 | 改后 |
|
||||
| --- | --- | --- |
|
||||
| `:281` | `actual = 0` | **保持 `0`**,紧随一行新增 `settlement_known = False`(局部阶段变量,带注释:只服务于取消分支的兜底取值,不进任何签名) |
|
||||
| `:288` | `await self._transport.complete(...)` | 不动(S3 的“端口已开始”窗口就是它未返回的那段) |
|
||||
| `:301` | `actual = source.effective_est_tokens()`(usage 不可得) | 值不变,其后置 `settlement_known = True` |
|
||||
| `:303` | `actual = result.prompt_tokens + result.completion_tokens` | 值不变,其后置 `settlement_known = True`(S4;**真实 usage 恰为 0 也算已知**,取消不得覆写) |
|
||||
| `:311` `RequestRejectedError` | 隐式 0 | 分支**首句**:`actual = 0; settlement_known = True`(逐字保住 1.3.5,且取消落在本分支 await 中途仍得 0) |
|
||||
| `:315` `ResultInvalidError` | 隐式 0 | 同上(已计费的坏结果仍退全款属另一族缺口,本版不动) |
|
||||
| `:320` `CancelledError` | 不动 `actual` | **唯一**新增赋值点,且在本分支首句:`if not settlement_known: actual = source.effective_est_tokens()`;`is_probe` / `_emit` / `raise` 三行原样 |
|
||||
| `:325` 失败分支入口 | `dead = isinstance(exc, SourceDeadError)` | 完成该分支原有同步分类/选源反馈后,紧贴第一个 `await record_failure` 之前插入 `actual = 0 if dead else source.effective_est_tokens(); settlement_known = True`;不得前移到同步分类之前改变其异常结算 |
|
||||
| `:335-336` | `if not dead: actual = est` | **删除**(已上移);非取消路径的最终值与 1.3.5 逐字相同,只是算得更早 |
|
||||
| `:339-341` | `finally: pacer.leave(); settle_and_release(permit, actual)` | 一字不动 |
|
||||
|
||||
**“确定结算”的界桩就是上表的 await 位置**:失败分支的结算决定前移到两个记账 await 之前,故**取消发生在已知 `SourceDead` 之后时保留那个既有的 `0`**(源已判死就不该继续占额度)。**为何必须靠位置而不能只靠标志位**:`except asyncio.CancelledError` 与 `except (SourceDeadError, TransientError)` 是**同级**分支,落在后者块内 await 上的取消**不会**被前者接住,直接穿到 `finally`——那一刻 `actual` 是什么就结什么,标志位没有机会被读到。故失败分支必须在其**第一个 await 之前**就把 `actual` 定死。本版**不把 S5 泛化成“一切失败按 `est` 结算”**;`est` 只是“取消且结算未定”这一格的兜底值。
|
||||
|
||||
`embedding.py::_attempt` 同构:`:345` 保持 `actual = 0` 并新增 `settlement_known = False`;`:358`/`:360` 值不变、其后置 `True`;`:377`(`RequestRejected`/`ResultInvalid` 合并分支)首句 `actual = 0; settlement_known = True`;`:408` 失败分支在 `dead` 之后、`record_failure`(`:414`)之前插入 `actual = 0 if dead else est; settlement_known = True` 并删掉 `:414-415` 的 `if not dead:` 赋值;`:392` 取消分支首句加同款条件赋值;`:429-430 finally` 不动。
|
||||
|
||||
`ocr.py:449 settle_and_release(permit, 0)` **保持 0**:OCR 无 token 是事实而非"未知"(`ocr.py:9` 既有声明),不得改成 est,也不引入 `settlement_known`。
|
||||
|
||||
**防越界回归(必带)**:假 transport 抛 `RuntimeError`(不属四分类、无 except 接住)→ `tpm_used == 0` 且异常原样上抛;真实 usage 恰为 0 的成功 → `tpm_used == 0`。两条把“不得扩到未批准语义”钉成可回归的断言。
|
||||
|
||||
共享状态与探针一律不变:`pacer.leave()`、`permit.release()`、`breaker.release_probe()`(`retry.py:321-322`、`ocr.py:410-411`、`embedding.py:393-394`)、`mark_progress` 的调用点、次数与顺序全部逐字保留;**不新增任何公开参数**。
|
||||
|
||||
### 3.6 F1:`Retry-After` 非有限值(`transports/openai_compat.py:111-119`)
|
||||
|
||||
签名改为 `_parse_retry_after(raw: str | None, *, source_name: str) -> float | None`——`source_name` 是**必填 keyword-only 参数**(私有模块内函数,不属公共面,故不给默认值;漏传即 `TypeError`);**唯一调用处**是 `_translate_429`(`:140`),改传 `source_name=source.name`(该函数已持有 `source`,不需新参数)。
|
||||
|
||||
判据与告警(按已批设计 §6.1 原文精确定义,不得自行扩大):
|
||||
|
||||
| 输入形态 | 返回 | 日志 |
|
||||
| --- | --- | --- |
|
||||
| `inf` / `-inf` / `1e999`(`float()` 成功且 `math.isinf(seconds)`) | `None` | **一条 `logger.warning`**,只写源名与判据词(如 `retry_after_not_finite`),**不拼接、不截断、不打印原始头字符串** |
|
||||
| `nan` | `None` | **无告警**:沿用既有 `seconds > 0` 恒假的值语义,本版**不为它新增分支、不改判据顺序** |
|
||||
| HTTP-date / 空串 / 负数 / 不可解析 | `None` | 无告警(429 风暴下逐次告警会淹掉真信号) |
|
||||
| 有限正数 | 该值 | 无 |
|
||||
|
||||
实现上只在 `float()` 成功后、`seconds > 0` 之前插一段 `if math.isinf(seconds): warning; return None`;`_translate_429` 的分类、`backoff_delay` 的 `max(delay, retry_after)` 取大逻辑一字不动(设计 §6.2:不夹 `backoff_max_s`)。
|
||||
|
||||
## 4. 任务与提交点(4 个原子提交)
|
||||
|
||||
### T0:设计批准状态与本计划(本任务,无代码)
|
||||
|
||||
产出:设计文档状态改批准 + §6.3 结算矩阵 + §5.2 五形态;本计划。不提交代码、不动测试。
|
||||
|
||||
### T1 → 提交 1 `fix: settle cancelled attempts against the source estimate`
|
||||
|
||||
1. **先红**:按 §5 批次 C 写 S3/S7 用例(`test_retry.py`、`test_embedding.py`),确认失败信息是 `tpm_used == 0 != 400`(不是构造错误);同批写 S5-dead 与 S8 两条**防越界**用例(实现前应已绿,作回归锁)。
|
||||
2. 改 `middleware/retry.py::_attempt` 与 `embedding.py::_attempt`(§3.5:初值保 0 + 局部 `settlement_known`,失败分支结算决定上移到两个 await 之前),`ocr.py` 只补注释不改值;**不新增任何公开参数、不改未分类异常路径**。
|
||||
3. **后绿**:新用例通过;`pytest tests/unit -q` 全绿(S1/S4/S5-dead/S6/S8 回归断言在批次 C 内一并落地)。
|
||||
4. 真实 Redis:`tests/integration/test_redis_cross_connection.py` 新增取消结算用例(§5 批次 F),跑 `pytest tests/integration/test_redis_cross_connection.py -q`。
|
||||
5. 暂存路径:`src/polygateway/middleware/retry.py`、`src/polygateway/embedding.py`、`src/polygateway/ocr.py`、三个测试文件。
|
||||
|
||||
### T2 → 提交 2 `feat: add an optional per-call wall-clock deadline`
|
||||
|
||||
1. 新建 `deadline.py`(§3.1)、`errors.py` 新类(§3.2)、`__init__.py` 导出、`pyproject.toml` layers 一行。
|
||||
2. `config.py` 四处(字段/loader/守卫/`from_env`)、三个 client 的构造参数 + 公开方法参数 + 包裹点 + `from_settings` 透传(§3.3/§3.4)。
|
||||
3. 先红后绿顺序:批次 A(`test_deadline.py` 值域)→ 批次 B(五形态)→ 批次 D(三链路命中与覆盖面)→ 批次 E(配置四条路)。
|
||||
4. 回归门:`pytest tests/unit tests/contracts -q` 全绿且**未改一行既有断言**;`make lint`(含 import-linter 新层)通过。
|
||||
5. 暂存路径:`src/polygateway/deadline.py`、`errors.py`、`__init__.py`、`config.py`、`client.py`、`embedding.py`、`ocr.py`、`pyproject.toml`、`tests/unit/test_deadline.py` 及四个改动测试文件。
|
||||
|
||||
### T3 → 提交 3 `fix: ignore non-finite Retry-After hints`
|
||||
|
||||
1. 先红:`tests/unit/test_openai_compat.py` 加 `inf`/`-inf`/`1e999`/`nan`/空/负/HTTP-date 七例,并加一例漏传 `source_name` 的 `TypeError`(批次 G)。
|
||||
2. 改 `_parse_retry_after`:加必填私有 kw `source_name`、加 `math.isinf` 判据与一条 warning,同步唯一调用处 `_translate_429`(`:140`);跑该文件与 `tests/unit -q`。
|
||||
3. 暂存:`src/polygateway/transports/openai_compat.py`、`tests/unit/test_openai_compat.py`。
|
||||
|
||||
### T4 → 提交 4 `docs: document the optional call deadline and cancellation settlement`
|
||||
|
||||
1. `CHANGELOG.md` 未发布段:三句强制措辞——**期限治理的是等待、返回时刻 = 期限 + 清理耗时(实测 5–7 倍)**;**到期不等于未产出、未计费**;**不配置即保持 1.3.5 语义**(纯 429 序列仍可能长等、有限大 `Retry-After` 仍照睡)。另记取消结算口径变化:**仅当取消发生在“端口已开始、结算尚未确定”时**按 `est` 保留预扣(方向为宁多扣不空退);已知结算(含真实 usage 恰为 0、已判 `SourceDead` 的 `0`)不被覆写,**未分类异常仍按 `0`**。另列"`except GatewayUnavailableError` 接不住新异常"。
|
||||
2. `README.md` **四处同步(缺一不可,按行号定位)**:① 能力表(`:10-22` 区间)新增一行“调用期限”,措辞用 §5.3 三句;② “### 4. 业务侧异常处理”示例(`:185-195`)——该段 `except GatewayUnavailableError` **接不住** `CallDeadlineExceeded`,必须加一条 `except CallDeadlineExceeded` 分支并注明它无 `retry_after_s`;③ “哪些异常会到达调用方”表(`:466-476`)左列新增 `CallDeadlineExceeded` 行,并写明它**不属四分类、不属 `GatewayUnavailableError` 族**,只在显式配期限后才可能出现;④ 错误模型段补一句**有限大 `Retry-After` 残留**(能力表 `:15` 写的“尊重 Retry-After”仍成立:库不夹 `backoff_max_s`,服务端给 3600s 就睡 3600s,唯一制约手段是本版的调用期限;`inf`/`1e999` 自 1.3.6 起按无提示处理)。另:`.env.example` 在 `LLM__CIRCUIT_OPEN`(`:71`)之后加注释行 `# LLM__CALL_DEADLINE_S=`(缺省不启用,说明其治理对象是等待)。
|
||||
3. `research-wiki/findings/2026-09-10-136-call-deadline-validation.md`:红绿证据、命令与退出码、豁免索引。
|
||||
4. 独立验证(全新上下文 verifier)与整分支审查在本提交前完成;版本号与 wiki 同步留给发布清单(本计划不 bump、不发布)。
|
||||
|
||||
## 5. 测试矩阵 → 任务映射
|
||||
|
||||
**测试设施复用与"哪一份副本"的硬性核对**(历史坑,动手前必须核对):
|
||||
|
||||
| 事实 | 证据 | 纪律 |
|
||||
| --- | --- | --- |
|
||||
| `tests/unit/test_retry.py:35`、`tests/unit/test_embedding.py:176` 从 `tests.contracts.conftest` import `FakeClock` | 现读 | 改这一份即影响契约与两个单测文件 |
|
||||
| `tests/unit/test_ocr_client.py:344` **自带一份同名 `FakeClock`** | 现读 | OCR 用例只吃这一份;给 OCR 加期限用例时不得误改 contracts 那份并以为生效 |
|
||||
| `tests/unit/test_embedding.py:177` 从 `tests.unit.test_backpressure` import `BoundedSleep` | 现读 | 复用它做"轮询次数有界"断言,不新造 |
|
||||
| 无 `tests/conftest.py` / `tests/unit/conftest.py` | `ls` 实测 | 新 fixture 只能进各文件本地,或复用 `tests/contracts/conftest.py`(已被 unit 直接 import) |
|
||||
|
||||
**取消白箱的确定性纪律**:既有取消用例用 `await asyncio.sleep(0.05)` 撞窗口(`test_retry.py:474`、`test_ocr_client.py:365`)——新用例**不得**沿用。做法:给 `test_retry.py::FakeTransport` 的 `"hang"` 分支加 `self.entered.set()`(构造期 `self.entered = asyncio.Event()`,3.10+ 不绑定 loop),用例 `await transport.entered.wait()` 后再 `task.cancel()`;embedding/OCR 的 `ScriptedEmbedTransport`/`ScriptedOcrTransport` 同款加一个 `entered`。既有用例不动。
|
||||
|
||||
| 批次 | 断言(→ 任务) | 落点 |
|
||||
| --- | --- | --- |
|
||||
| A 值域 | `None` 通过;`0`/负/`nan`/`inf`/`"1"`/`True`(bool 不得当 1 秒)/`object()` → `ValueError` 且消息含 origin(→T2) | `tests/unit/test_deadline.py` |
|
||||
| B 形态区分 | ①到期 → `CallDeadlineExceeded`(`scope`/`deadline_s` 正确);②未到期内层自抛 `TimeoutError` → 原样上抛;③**到期后清理自抛 `TimeoutError`** → 原样上抛且**断言不是** `CallDeadlineExceeded`(钉住身份比较);④外部 `task.cancel()`(先于/晚于到期各一例)→ `CancelledError`;⑤擦边成功 → 正常返回且 `task.cancelling() == 0`;⑥到期窗口内体内先抛领域异常(同步自旋构造)→ 上抛该异常,**不断言必为 deadline**;⑦`deadline_s=None` → 逐字旧路径(→T2) | `tests/unit/test_deadline.py`(真实 loop 时钟,期限 0.05s、体 0.3s,4–10× 余量,不标 slow) |
|
||||
| C 取消结算 | S3:`tpm=1000, est_tokens=400`、transport `hang`、`entered` 后取消 → `tpm_used == 400`(**红→绿核心**)且 `inflight == 0`;S4:假 gate 在 `record_success` 处 `set()` 后挂起 → 取消 → `tpm_used == 15`(真实 usage 未被覆盖);S1:熔断开路使 `pick` 走 `settle_and_release(permit, 0)` → `tpm_used == 0`;S5-dead:假 gate 在 `SourceDead` 的 `record_failure` 处挂起 → 取消 → `tpm_used == 0`(**不得**变 `est`);S5-transient:同位置但瞬时失败 → `tpm_used == est`;S6:OCR 源 `tpm=600`、transport `hang` → 取消 → `tpm_used == 0`;S7:embedding 同 S3;**S8 防越界**:假 transport 抛 `RuntimeError` → `tpm_used == 0` 且异常原样上抛;真实 usage 恰为 0 的成功 → `tpm_used == 0`;四条既有路径(成功/`SourceDead`/`RequestRejected`/`ResultInvalid`)结算值逐字不变(→T1) | `test_retry.py`、`test_embedding.py`、`test_ocr_client.py` |
|
||||
| D 覆盖面 | 期限分别落在 ①退避 `sleep`(注入真 `asyncio.sleep`)②准入排队(配额满轮询)③结构化重问 ④embedding 多批 → 均抛 `CallDeadlineExceeded`;embedding 断言 **N 批共享一份**期限(总时长不随批数放大);`texts == []` 早返回不受期限影响(→T2) | `test_client.py`、`test_embedding.py`、`test_ocr_client.py` |
|
||||
| D2 到期代价 | ①终态遥测:到期恰好一条 `event_kind='terminal_failure'`、`error_type='CallDeadlineExceeded'`,被取消的 attempt 行仍 `cancelled`,两行 `logical_call_id` 一致,列数不变;②清理不可越过 + 量化:假 permit/假 emitter 各注入已知 sleep → 返回时刻 ≈ 期限 + 已知清理时长(断言 > 期限的若干倍,不断言上界);③已计费成功被丢弃:假缓存后端 `set` 慢于期限 → 抛 deadline 且断言 transport **已成功调用一次**(→T2) | `test_client.py` |
|
||||
| E 配置四条路 | 键未设 → `None`;非法值 × {env、`GatewaySettings(...)` 直接构造、`dataclasses.replace`、三个 client `__init__` 直传} 各一例 → `ValueError`;per-call 非法值抛错且**无 "coroutine was never awaited" 警告**(`pytest.warns` 反向断言 / `-W error::RuntimeWarning`);`call_deadline_s < timeout_s` 合法不报错;注入钟跳变 10^6 秒**不**触发期限,而期限触发时 `total_latency_ms` 仍取自注入钟(→T2) | `test_config.py`、`test_client.py` |
|
||||
| F 真实 Redis | 复用 `tests/integration/test_redis_cross_connection.py` 的 `clients`/`_limiter`/`_client`/`ScriptedTransport(hang=True)`:源 `tpm=1000, est_tokens=400`,`inflight` 出现后取消 → `source_stats.tpm_used == 400` 且 `inflight == 0`。**不改 Lua、不改 `tests/contracts/`**;另跑既有 `pytest tests/contracts/test_limiter_contract.py -q`(memory+redis 双参数)证明后端算术未被触碰(→T1) | `tests/integration/test_redis_cross_connection.py` |
|
||||
| G F1 | `inf`/`-inf`/`1e999` → `None` + 各一条 warning(断言日志**含源名、不含**原始字符串);`nan`/空/负/HTTP-date → `None` 且**无** warning(`nan` 仍走既有 `seconds > 0` 值语义,不新增分支);漏传 `source_name` → `TypeError`(钉住必填 kw);有限正数仍参与 `max(delay, retry_after)`;`insufficient_quota` 仍归 `SourceDead`(→T3) | `test_openai_compat.py` |
|
||||
| H 未启用回归 | `call_deadline_s=None` 时 `tests/unit`、`tests/contracts` 全绿且**未改一行既有断言**(→T2 门) | 全套件 |
|
||||
|
||||
命令(全部 `conda run -n PolyGateway`,禁止接管道以免退出码失真):`pytest tests/unit -q`、`pytest tests/contracts -q`、`pytest tests/integration/test_redis_cross_connection.py -q`、`make lint`。真实网关 e2e 与 `-m slow` 变体本计划**不跑**,由发布清单第 4 步统一负责。
|
||||
|
||||
## 6. 阻塞矩阵与交接
|
||||
|
||||
| 触发条件 | 处置 |
|
||||
| --- | --- |
|
||||
| 需要新增本计划外的公共键/端口方法/遥测列 | **停下上报**(设计 §9 边界之外即未批准) |
|
||||
| 批次 B③(清理期自抛 `TimeoutError`)在实现里无法确定性构造 | 改用假端口在 `except CancelledError` 内直接 `raise TimeoutError`(探针 D3 已证可复现);仍不可得则记入 findings 的豁免索引,不得删断言 |
|
||||
| 批次 D2② 的量化断言在 CI 机器上抖动 | 只断言下界(返回时刻 > 期限 × 2),不断言上界;不得改成 `sleep` 猜测 |
|
||||
| 真实 Redis 不可用(`REDIS_URL` 未配置) | 用例自动 skip;findings 必须显式记"未取得真实 Redis 证据",不得以 memory 结果冒充 |
|
||||
| 发现 `ResultInvalid`/`RequestRejected` 退全款想顺手修 | **不修**(设计 §6.3 末段:未批准的另一族记账语义),登记为新 issue 交父会话 |
|
||||
| 失败分支结算决定上移后发现某条既有用例变红 | 先 `systematic-debugging` 定根因;如确为语义变化(非取消路径的最终值应与 1.3.5 逐字相同)则**停下上报**——说明本项只改算得更早、不改算出什么 |
|
||||
| 想把保守口径扩到未分类异常(`RuntimeError` 等) | **不扩**(人类仅批准取消路径);S8 回归用例就是这道锁,需要就另立 issue |
|
||||
| issue #24 相关想法 | 一律不实现、不写进代码与文档 |
|
||||
|
||||
交接物:4 个提交、1 份 findings、CHANGELOG 未发布段。版本号 bump、tag、构建、上传 registry 与 wiki 同步**不在本计划内**,按 CLAUDE.md §4.4.1 另行执行。
|
||||
|
||||
## 7. 自审
|
||||
|
||||
| 检查 | 结论 |
|
||||
| --- | --- |
|
||||
| 路径/行号/签名是否可执行无 TBD | 是——所有接入点均现读行号(`retry.py:281/288/301/303/311/315/320/325/335/339`、`embedding.py:345/349/358/360/377/392/408/414/429`、`ocr.py:449`、`client.py:398`、`embedding.py:209`、`ocr.py:274`、`config.py:186/349/380-384`、`openai_compat.py:111-119/140`、`__init__.py:61-62`) |
|
||||
| 是否复用而非重造 | 是——`settle_and_release`、`emit_terminal_once`、`claim_terminal`、`asyncio.timeout` 范式、FakeClock/BoundedSleep/ScriptedTransport、限流契约套件与真实 Redis 用例全部复用;新增仅 1 文件 + 1 异常类 + 1 配置键 |
|
||||
| 是否有先失败后通过的证据点 | 是——T1 的 S3/S7、T2 的 A/B/D、T3 的 G 均先红 |
|
||||
| 取消与降级铁律 | 未新增 `except Exception`;`CancelledError` 无新捕获点;期限未启用时不进任何上下文 |
|
||||
| 反 gold-plating | `ResultInvalid`/`RequestRejected` 退全款、#24、`shield`、遥测列、Lua 一律不碰 |
|
||||
| 残余诚实标注 | S3 是保守选择而非"已计费"的证明;**未分类异常(S8)仍按 `0` 退全款,属已知残留、本版不动**;清理期自抛 `TimeoutError` 时无终态行;跨 Python 版本仅 3.12.13 有探针实证——四条均已写进设计 §12,findings 需复述 |
|
||||
@@ -0,0 +1,338 @@
|
||||
---
|
||||
type: plan
|
||||
node_id: plan:2026-09-10-24-hedged-requests
|
||||
title: "issue #24 长尾对冲请求与裸生成时间实施计划"
|
||||
date: 2026-09-10
|
||||
---
|
||||
|
||||
# issue #24 长尾对冲请求与裸生成时间实施计划
|
||||
|
||||
> 设计:`research-wiki/designs/2026-09-10-24-hedged-requests-design.md`,**人类于 2026-09-10 正式批准**(§9 H1–H7 及增补 H8 全数获批;H1 取方案 A 并发对冲,H3 取零新列档,H5 取 v1 只允许单路)。
|
||||
> 计划审核门:Claude 自审 + 独立模型审查;plan 无人类门,审毕直接执行。
|
||||
> 目标:① chat 链路可选对冲(挂起超阈值时并发向**异源**再发一次,先回者赢、输家取消);② `CallStats` 增 `hedges`/`generation_ms`/`hedge_won`,三条链路 `_attempt` 加 transport 级计时;③ 默认关闭,缺省行为逐字等于 1.3.6。
|
||||
> 方案:设计 §3 方案 A——对冲轮 = 一次"超级尝试",复用完整准入(QuotaGate + BreakerGate + pacer + 冷却备忘),输家落 1.3.6 取消 S3 格结算,零新错误分类。
|
||||
> 技术:Python 3.12+、asyncio 任务组编排(`asyncio.wait(FIRST_COMPLETED)` + 显式收口)、frozen dataclass、pytest + 事件驱动假 transport + 真实 loop 钟(4–10× 余量)、ruff、import-linter。
|
||||
> 基线 HEAD:`166b286`(main,1.3.6 已发布);分支 `feature/24-hedged-requests`。
|
||||
|
||||
**范围纪律**: 不改四分类/熔断语义/429 分账/限流 Lua/`deadline.py`/`Permit` 端口;不做 embedding/OCR 对冲、分位数触发、同源对冲、per-call 对冲参数;不引入 shield/后台任务;不加遥测新列(`hedge_cancelled` 字符串 + `CallStats` 三字段经既有通道带出)。
|
||||
|
||||
## 1. 边界、授权与执行纪律
|
||||
|
||||
| 项目 | 固定边界 |
|
||||
| --- | --- |
|
||||
| 唯一 writer | 一工作区一 writer;父会话负责前台委派与审核派发。1.3.X 合并/发布授权沿用;跨到 1.4 或新公共面变化须停下确认 |
|
||||
| 公共面 | 只做设计 §9 已批准项:`Transport.complete` 加 `first_token_event` 必填 kw(H2)、两配置键 `{SCOPE}__HEDGE__AFTER_S`/`{SCOPE}__HEDGE__MAX_EXTRA`(H4)、`GatewayClient.__init__` 两 keyword-only 参数、`CallStats` 三字段(H3+H8)、输家 `hedge_cancelled` 标签。**不新增其它键/端口方法/遥测列/异常类** |
|
||||
| 对冲边界 | 仅 chat(RetryMW);EmbeddingClient/OcrClient 不加对冲参数(非目标 A),但它们的 `_attempt` 照样加 generation 计时点(H8);v1 单路对冲(H5) |
|
||||
| 记账边界 | 输家取消走既有取消路径:`settlement_known=False` → `settle(est)` 保留预扣(`retry.py:327-331`);输家**不** `record_failure`、不喂熔断/健康分(挂起 ≠ 源死亡);两任务都失败才进重试且 `fails += 1` 只计一次(H6) |
|
||||
| 取消铁律 | 外部取消到达时两任务同消并穿透;不 shield、不留后台任务;收口 await 允许被再取消(同 136 清理纪律) |
|
||||
| 降级方向 | 限流/熔断后端不可用 → 照常冒泡(fail-closed);准入失败 → **静默放弃对冲**,原请求继续等(不抛错、不硬等);遥测仍 warning 降级 |
|
||||
| 时钟纪律 | 对冲触发只用**相对时长 + 事件循环钟**(`asyncio.wait` timeout),绝不读注入 `now`;`generation_ms` 计时用该链路既有注入钟(与 `total_latency_ms` 同钟,差值才有意义;生产即 `time.monotonic`) |
|
||||
| 证据纪律 | 不打印 `.env`/token/Authorization;不提交 `.pi/`、`tests/outputs/`;测试事件驱动,**禁 sleep 撞窗口**;对冲阈值用 0.05s 级真实 loop 钟,断言容差 4–10×(`tests/unit/test_streaming.py` 既有范式,不标 slow) |
|
||||
|
||||
Skill 纪律:T1–T3 行为变更执行 `test-driven-development`(先红后绿证据落在本会话工具输出);每次提交执行 `commit`;T4 前执行 `requesting-code-review` 与 `verification-before-completion`;异常先 `systematic-debugging`。
|
||||
|
||||
**保真校验**: 本计划不涉及 `reference/` 参考实现迁移(对冲编排为 D13 自研语义,蓝本即本库 1.3.6 的准入/结算/取消机制),保真校验不适用。
|
||||
|
||||
## 2. 文件职责与不变接缝
|
||||
|
||||
| 动作 | 精确路径 | 职责 |
|
||||
| --- | --- | --- |
|
||||
| 修改 | `src/polygateway/ports.py` | `Transport.complete`(:51-60)加 `first_token_event` 必填 kw;顶部加 `import asyncio`(stdlib,不违 P7) |
|
||||
| 修改 | `src/polygateway/transports/openai_compat.py` | `complete`(:426-436)加参透传;`_complete_stream`(:552 起)首 token 处(:573-575)置位;`_complete_once`(:646 起)接收但**永不置位**(docstring 明写) |
|
||||
| 修改 | `src/polygateway/middleware/retry.py` | `_attempt`(:270-353)加 `first_token_event`/`generation_sink` 私有 kw + transport 级计时;`__call__`(:224-267)单轮尝试 → 任务组编排;`_past_hedge_window`/`_attempt_hedged`/`_combine_failures` 新方法;CancelledError 分支(:327-334)`hedge_cancelled` 标签;`__init__` 加 `hedge_after_s` |
|
||||
| 修改 | `src/polygateway/middleware/admission.py` | `pick`(:171-213)加 keyword-only `exclude: frozenset[str] \| None = None` |
|
||||
| 修改 | `src/polygateway/types.py` | `CallStats`(:296-318)增三字段(全带默认值,追加在 `total_latency_ms` 后);`_CallContext`(:321-360)docstring 修订 + 三个计数 + `record_generation`/`register_hedge`;`snapshot`(:348-353)填三字段 |
|
||||
| 修改 | `src/polygateway/config.py` | `_RESERVED_SEGMENTS`(:57)加 `"HEDGE"`;`GatewaySettings` 字段(:190 后)加 `hedge_after_s`/`hedge_max_extra`;`__post_init__`(:201 后)加 `_validate_hedge()`;新增 `_load_hedge`(:695 `_load_call_deadline` 之后)与模块级 `check_hedge_assembly` 守卫;`from_env`(:396 同列)透传 |
|
||||
| 修改 | `src/polygateway/client.py` | `__init__`(:239 后)加两 keyword-only 参数,入口即校(复用 `check_hedge_assembly`);RetryMW 构造(:254-273)传 `hedge_after_s`;`from_settings`(:511 同列)透传 |
|
||||
| 修改 | `src/polygateway/embedding.py` | `_attempt`(:351 起,transport 调用 :373)两侧计时 + 成功分支 `record_generation(accumulate=True)` |
|
||||
| 修改 | `src/polygateway/ocr.py` | `_attempt`(:376 起,`_invoke` 调用 :397)两侧计时 + 成功分支 `record_generation(accumulate=False)` |
|
||||
| 新建 | `tests/unit/test_hedge.py` | 对冲编排全部用例(批次 G) |
|
||||
| 修改 | `tests/unit/test_openai_compat.py` | 批次 A(端口加参);`_complete` helper(:82-90)同步签名 |
|
||||
| 修改 | `tests/unit/test_retry.py` `test_types.py` `test_embedding.py` `test_ocr_client.py` `test_client.py` `test_config.py` | 批次 B–F、H;`test_retry.py:82` FakeTransport 签名同步 |
|
||||
| 修改 | `tests/unit/test_backpressure.py:213` `tests/unit/test_ports.py:75` `tests/unit/test_client.py:1741` `tests/integration/test_redis_cross_connection.py:78` `tests/e2e/conftest.py:284-303` | fake/包装 transport 签名同步(e2e 包装**转发** `first_token_event`) |
|
||||
| 修改 | `tests/unit/test_live_evidence.py:270-279`(`_complete` helper)+`:1258`(直调 `ObservedTransport(...).complete(...)`);`tests/unit/test_usage_source_domain.py:135`(直调真实 `OpenAICompatTransport.complete`) | **调用方**同步(独立审 B1): helper 加 `first_token_event=None` 转发、两直调传 `None`;不传则必填 kw 报 `TypeError`,unit 门必红 |
|
||||
| 修改 | `CHANGELOG.md`、`README.md`、`.env.example` | 新键、三字段、对外承诺措辞(§4 T4) |
|
||||
| 新建 | `research-wiki/findings/2026-09-10-24-hedged-requests-validation.md` | 红绿、命令、豁免索引,≤300 行 |
|
||||
|
||||
**不改**:`errors.py`(零新异常)/`deadline.py`/`telemetry/schema.py`(零新列)/`middleware/{ratelimit,breaker,structured,cache,telemetry}.py`/`backends/**`(含全部 Lua)/`tests/contracts/**`;`embedding.py`/`ocr.py` 除计时点外一字不动;`StallClock`/`backoff_delay`/`settle_and_release` 逐字不动。若必须突破本清单,先说明最小原因交父会话核定。
|
||||
|
||||
## 3. 跨任务接口(可执行定义,禁止占位)
|
||||
|
||||
### 3.1 T1:`Transport.complete` 加首 token 事件(H2)
|
||||
|
||||
`ports.py:51-60` 签名改为(顺序追加在 `reasoning_effort` 后,**必填、不设默认值**,与端口既有约定同款):
|
||||
|
||||
```python
|
||||
async def complete(
|
||||
self, *, messages: list[dict[str, Any]], source: SourceConfig, stream: bool,
|
||||
overlay: dict[str, Any], call_id: str, reasoning_effort: Effort | None,
|
||||
first_token_event: asyncio.Event | None,
|
||||
) -> TransportResult: ...
|
||||
```
|
||||
|
||||
docstring 补两句:`None` = 调用方不观测首 token(未启用对冲);非流式实现**永不置位**(物理上无中途信号,事件自然退化为纯时间阈值)。`OpenAICompatTransport.complete`(:426-436)加同款必填 kw 并透传两条路径;`_complete_stream` 在 :573-575 `if ttft_ms is None:` 块内加:
|
||||
|
||||
```python
|
||||
if first_token_event is not None:
|
||||
first_token_event.set()
|
||||
```
|
||||
|
||||
`_complete_once`(:646)接收该参数但永不置位,docstring 明写"非流式无中途信号"。`retry.py:291-300` 调用处 T1 先传字面 `first_token_event=None`(必填参数不传即全库 TypeError;T3 换成真事件)。
|
||||
|
||||
假 transport 同步纪律(`test_retry.py:71-73` 既有注释的同款):签名加 `first_token_event`,**不给默认值**;除 `test_hedge.py` 外所有 fake 忽略该参数即可。e2e 包装(`tests/e2e/conftest.py:284-303`)必须**转发**给被包 transport。
|
||||
|
||||
### 3.2 T2:`CallStats` 三字段与 `_CallContext` 计数(H3+H8)
|
||||
|
||||
`types.py` `CallStats`(:296-318)在 `total_latency_ms` 后追加:
|
||||
|
||||
```python
|
||||
hedges: int = 0
|
||||
"""本次逻辑调用实际并发发出的对冲路数(触发但准入失败静默不计);1.3.6 及以前恒 0。"""
|
||||
generation_ms: int = 0
|
||||
"""裸生成时间: 赢家/成功那次 transport 调用的墙钟时长(口径见设计 §4.5 H8)。"""
|
||||
hedge_won: bool = False
|
||||
"""赢家是否为对冲路;无对冲恒 False。"""
|
||||
```
|
||||
|
||||
`_CallContext`(:321-360):docstring "每调用一个实例的**单任务**对象" 修订为 "每逻辑调用一个实例,**可多任务并发登记**(对冲);全部方法无 await,事件循环内任务安全";`__slots__` 与 `__init__` 加 `_generation_ms: int`/`_hedges: int`/`_hedge_won: bool`;新增两个同步方法:
|
||||
|
||||
```python
|
||||
def record_generation(self, elapsed_ms: int, *, accumulate: bool) -> None:
|
||||
"""chat/OCR 覆盖(结构化重问最后一轮为准);embedding 分批累加。"""
|
||||
self._generation_ms = self._generation_ms + elapsed_ms if accumulate else elapsed_ms
|
||||
|
||||
def register_hedge(self, *, hedge_won: bool) -> None:
|
||||
"""对冲路实际发出即计数;赢家裁定后一次性登记。"""
|
||||
self._hedges += 1
|
||||
self._hedge_won = hedge_won
|
||||
```
|
||||
|
||||
`snapshot`(:348-353)按字段名填 `hedges=self._hedges, generation_ms=self._generation_ms, hedge_won=self._hedge_won`。
|
||||
|
||||
### 3.3 T2:三条链路 `_attempt` 的 transport 级计时点
|
||||
|
||||
统一形态:计时**只包 transport 调用本身**,用该链路既有注入钟 `self._now`(生产 = `time.monotonic`);起点紧贴调用前、终点在返回后首句,**中间无 await**(取消落进来时 transport 未返回,本就不计)。
|
||||
|
||||
| 链路 | 计时点 | 记录点 |
|
||||
| --- | --- | --- |
|
||||
| chat(`retry.py:291-300`) | `gen_started = self._now()` 紧贴 `await self._transport.complete(...)` 前;返回后首句 `generation_sink.append(int((self._now() - gen_started) * 1000))` | **不在 `_attempt` 内记录**——对冲赢家归属由编排裁定。`_attempt` 加私有 kw `generation_sink: list[int]`;`__call__`(:248-256)每轮建 sink,`outcome` 为 `LLMResponse` 且 `call_context` 非 None 时 `record_generation(sink[0], accumulate=False)`(与 `register_attempt` 同款 None 守卫) |
|
||||
| embedding(`embedding.py:373`) | 同款两侧包 `await self._transport.embed(...)` | 成功分支(`:386 settlement_known = True` 之后)`context.record_generation(gen_ms, accumulate=True)`——分批累加 |
|
||||
| ocr(`ocr.py:397`) | 同款两侧包 `await self._invoke(...)` | 成功分支 `context.record_generation(gen_ms, accumulate=False)` |
|
||||
|
||||
缓存命中/空输入不产生 transport 调用 → `generation_ms` 恒 0(0 是实测,不违 `types.py` "None 表未知" 惯例——本字段语义是时长不是用量)。
|
||||
|
||||
### 3.4 T3:对冲编排(retry.py,设计 §4.6 的唯一实现形态)
|
||||
|
||||
`RetryMW.__init__` 加 keyword-only `hedge_after_s: float | None = None`(存 `self._hedge_after_s`;`hedge_max_extra` **不下传**——v1 编排固定单路,H5;配置面值域与 v1 生效口径由 §3.6 守卫负责)。模块级:
|
||||
|
||||
```python
|
||||
_HEDGE_LOSER_ATTR = "_polygateway_hedge_loser"
|
||||
"""编排在 cancel() 之前给输家任务置位的标记;_attempt 读它选遥测标签。"""
|
||||
```
|
||||
|
||||
`__call__`(:244-256)循环体内:`self._hedge_after_s is None` → **逐字旧路径**(`_attempt` 传 `first_token_event=None`,单任务,默认关闭回归门据此成立);否则 `outcome = await self._attempt_hedged(request, picked, reasons, attempt_fails)`,其后的 `_is_rate_limited`/refund/`fails` 计数机制一字不动。
|
||||
|
||||
`_attempt_hedged` 编排(Phase 注释组织;`_attempt` 相应加 `first_token_event` kw 取代 T1 的字面 None):
|
||||
|
||||
```python
|
||||
# Phase 1 启动原路: 事件与 sink 每轮新建(局部状态,严禁实例属性)
|
||||
source, permit, entry = picked
|
||||
first_token: asyncio.Event = asyncio.Event()
|
||||
sink_p: list[int] = []
|
||||
primary = asyncio.create_task(
|
||||
self._attempt(request, source, permit, entry, reasons, attempt_fails,
|
||||
first_token_event=first_token, generation_sink=sink_p)
|
||||
)
|
||||
# Phase 2 触发窗: 只认"阈值到 + 首 token 未至 + 原路在途"(loop 相对时长,不读注入 now)
|
||||
if not await self._past_hedge_window(primary, first_token):
|
||||
return await primary # 原路已了结/首 token 已至: 等价于未配置对冲
|
||||
# Phase 3 异源准入(完整 pick 路径, 无旁路): 拿不到候选 = 静默等原路
|
||||
hedge_picked, _ = await self._admission.pick(
|
||||
reasons, attempt_fails, exclude=frozenset({source.name})
|
||||
)
|
||||
if hedge_picked is None:
|
||||
return await primary
|
||||
```
|
||||
|
||||
Phase 4-5(赢家裁定与收口)规则:
|
||||
|
||||
```python
|
||||
sink_h: list[int] = []
|
||||
hedge = asyncio.create_task(
|
||||
self._attempt(request, *hedge_picked, reasons, attempt_fails,
|
||||
first_token_event=None, generation_sink=sink_h) # v1 单路: 对冲路不再触发梯次
|
||||
)
|
||||
done, pending = await asyncio.wait({primary, hedge}, return_when=asyncio.FIRST_COMPLETED)
|
||||
```
|
||||
|
||||
- **裁定**:done 中有成功(LLMResponse)即赢家;两路同时成功(竞速)→ **原路优先**(`hedge_won=False`,保守不弃原路成果);done 全是失败且 pending 非空 → 等 pending 了结后再裁定。
|
||||
- **收口**:赢家产生后,对 pending 中的输家先 `setattr(task, _HEDGE_LOSER_ATTR, True)` 再 `task.cancel()`,然后 `await asyncio.gather(*pending, return_exceptions=True)`——输家 finally 的结算/遥测跑完才返回(快照含输家,`attempts==2`;`client.py:442` 快照在返回后,天然在收口之后)。两路同时完成的竞速落选者**不置标记**(它没被取消,attempt 行是正常成功/失败行)。
|
||||
- **登记**:对冲路实际发出(Phase 3 之后)即计一次;赢家裁定后 `context.record_generation(赢家 sink[0], accumulate=False)` + `context.register_hedge(hedge_won=winner is hedge)`;**两败轮次(无赢家)同样照登** `register_hedge(hedge_won=False)`——设计 §4.5 已批准口径是"实际并发发出即计,触发但准入失败静默不计";`call_context is None` 时跳过(同 `register_attempt` 守卫)。
|
||||
- **两败汇合**(H6,只计一次预算;deterministic):
|
||||
|
||||
```python
|
||||
def _combine_failures(primary_f: _Failed, hedge_f: _Failed) -> _Failed:
|
||||
"""任一非 429 优先(计预算);两路皆 429 才按 429 免预算退还 stall 账;同类取原路。"""
|
||||
if _failure_reason(primary_f.exc) == "rate_limited" != _failure_reason(hedge_f.exc):
|
||||
return hedge_f
|
||||
return primary_f
|
||||
```
|
||||
|
||||
- **取消穿透**:Phase 2-5 全程包 `except BaseException`(含 `CancelledError` 与准入冒泡的 `GovernanceBackendError`)→ 两任务(存在者)`cancel()` + `gather(return_exceptions=True)` 尽力收口后 `raise` 原异常;收口 await 允许被再取消,不 shield。
|
||||
|
||||
`_past_hedge_window` 实现红线(waiter 任务必须收口,且**不得吞外部取消**):
|
||||
|
||||
```python
|
||||
async def _past_hedge_window(self, primary: asyncio.Task, first_token: asyncio.Event) -> bool:
|
||||
waiter = asyncio.create_task(first_token.wait())
|
||||
try:
|
||||
await asyncio.wait({primary, waiter}, timeout=self._hedge_after_s,
|
||||
return_when=asyncio.FIRST_COMPLETED)
|
||||
finally:
|
||||
waiter.cancel()
|
||||
try:
|
||||
await waiter
|
||||
except asyncio.CancelledError:
|
||||
if asyncio.current_task().cancelling(): # 外部取消,穿透
|
||||
raise
|
||||
return not primary.done() and not first_token.is_set()
|
||||
```
|
||||
|
||||
输家标签:`_attempt` 的 CancelledError 分支(:327-334)把 `error="cancelled"` 换成按标记选择——`label = "hedge_cancelled" if getattr(asyncio.current_task(), _HEDGE_LOSER_ATTR, False) else "cancelled"`。竞速误贴(外部取消与对冲取消同时到达)记账方向一致(est 保留),属设计 §4.5 已批准的可接受残留。
|
||||
|
||||
### 3.5 T3:`admission.pick` 私有排除参数(设计 §4.2)
|
||||
|
||||
`pick`(:171-173)签名加 keyword-only `exclude: frozenset[str] | None = None`;候选循环**首部**加:
|
||||
|
||||
```python
|
||||
if exclude and cand.name in exclude:
|
||||
continue # 被排除不是源的拒绝: 不计 gate_rejections、不写 reasons
|
||||
```
|
||||
|
||||
理由:写进 `reasons`/`gate_rejections` 会污染 `on_no_runnable`(:224)的分派判据与 `per_source_reasons` 对账。对冲调用方拿到 `None` 的处置是静默等原路(§3.4 Phase 3),**严禁**对它调 `on_no_runnable`(那会按 quota/circuit 策略抛错或睡觉,语义全错)。三条既有调用方(`retry.py:244`、`embedding.py:332`、`ocr.py:348` 所在循环)不传该参数,行为逐字不变。
|
||||
|
||||
### 3.6 T3:配置两键 + 两守卫 + client 透传(H4)
|
||||
|
||||
`config.py` 改动(单一定义点纪律,值域/交叉守卫只写一份):
|
||||
|
||||
| 项 | 精确定义 |
|
||||
| --- | --- |
|
||||
| 保留段 | :57 `_RESERVED_SEGMENTS` 加 `"HEDGE"`(防 provider 段撞名);两键均 3 段,`:405` 的 `len(parts) != 4` 判据天然跳过 `_load_sources` |
|
||||
| loader | 新增 `_load_hedge(scope, env)`(:695 `_load_call_deadline` 之后):`AFTER_S` 用 `_first` + `_cast(..., "float", ...)` + `ensure_call_deadline`(origin 传实际命中键名,同 `_load_call_deadline` 纪律);`MAX_EXTRA` 用 `_first` + `_cast(..., "int", ...)`,未设 = 1;返回 `{"hedge_after_s": ..., "hedge_max_extra": ...}` |
|
||||
| 字段 | `GatewaySettings` :190 后追加 `hedge_after_s: float \| None = None`、`hedge_max_extra: int = 1` |
|
||||
| 守卫 | 新增模块级 `check_hedge_assembly(*, hedge_after_s, hedge_max_extra, sources, call_deadline_s, origin)`,返回归一化后的 `hedge_after_s`;`GatewaySettings.__post_init__`(:201 后)加 `_validate_hedge()` 调它并 `object.__setattr__` 写回归一化值(同 `_validate_call_deadline` 形态);`GatewayClient.__init__` 调同一份(client.py:22 已 import config,合法) |
|
||||
| from_env | :396 同列加 `**_load_hedge(scope_u, env)` |
|
||||
|
||||
`check_hedge_assembly` 守卫全表(设计 §5;`hedge_after_s is None` 时值域归一化后直接返回,交叉守卫不查):
|
||||
|
||||
| 守卫 | 判定 |
|
||||
| --- | --- |
|
||||
| 值域 | `ensure_call_deadline(hedge_after_s, origin)`(None 或有限正数,复用 `deadline.py` 同款校验) |
|
||||
| `hedge_after_s ≥ min(源 timeout_s)` | `ValueError`(对冲永不可能触发,配置即错误) |
|
||||
| `hedge_after_s ≥ call_deadline_s`(两者皆设) | `ValueError`(期限先于对冲触发) |
|
||||
| `hedge_after_s ≥ min(已设 ttft_timeout_s)` | 装配期 **warning**(流式档被 TTFT 看门狗先行切断;非流式仍有效,不升 ValueError) |
|
||||
| 单源 scope 设了阈值 | 装配期 **warning**,允许(运行期拿不到候选自然静默) |
|
||||
| `hedge_max_extra` | 非 int/bool 或不在 [1,3] → `ValueError`;**>1 → warning**"v1 仅单路对冲生效,梯次追加为 H5 预留"(值域按设计 §5 表放到 3,运行期 H5 只允许 1,warning 保 fail-loud 不静默) |
|
||||
|
||||
`client.py`:`__init__` :239 后加 keyword-only `hedge_after_s: float | None = None, hedge_max_extra: int = 1`,在 :245-247 期限校验同列调 `check_hedge_assembly(hedge_after_s=..., hedge_max_extra=..., sources=sources, call_deadline_s=self._call_deadline_s, origin="GatewayClient(...)")`;RetryMW 构造(:254-273)传 `hedge_after_s=` 归一化值(**max_extra 不下传**,§3.4);`from_settings` :511 同列传 `settings.hedge_after_s`/`settings.hedge_max_extra`。`EmbeddingClient`/`OcrClient` 不加对冲参数;它们的 settings 嵌 `GatewaySettings` 故守卫照常跑(文档明写对冲键只对 chat 生效)。
|
||||
|
||||
## 4. 任务与提交点(4 个原子提交)
|
||||
|
||||
### T0:设计增补并入与本计划(本任务,无代码)
|
||||
|
||||
产出:设计文档 §4.5/§7/§8/§9 增补(已完成)+ 本计划。不提交代码。
|
||||
|
||||
### T1 → 提交 1 `feat: add a required first-token event to the transport port`
|
||||
|
||||
1. **先红**:批次 A(`tests/unit/test_openai_compat.py` 四用例),确认失败为 `TypeError`(签名无此 kw)而非断言值不符。
|
||||
2. 按 §3.1 改 `ports.py`、`openai_compat.py`、`retry.py:291-300` 传 None;同步六处 fake/包装(§2 表)+ `test_live_evidence.py`/`test_usage_source_domain.py` 三处调用点(§2 表末行);`_complete` helper(:82-90)加 `first_token_event=None` 默认转发(测试设施,与生产端口的"必填无默认"约定不冲突——生产端口不变)。
|
||||
3. **后绿**:批次 A 通过;`pytest tests/unit -q` 全绿且不改一行既有断言。
|
||||
4. 暂存:`src/polygateway/ports.py`、`src/polygateway/transports/openai_compat.py`、`src/polygateway/middleware/retry.py`、六个测试文件。
|
||||
|
||||
### T2 → 提交 2 `feat: expose bare generation time and hedge flags in CallStats`
|
||||
|
||||
1. **先红**:批次 B(`test_types.py` 三用例,字段不存在 → `TypeError`/`AttributeError`)+ C-F(各链路计时断言,字段恒 0 → 断言失败)。
|
||||
2. 按 §3.2 改 `types.py`;按 §3.3 改三条 `_attempt` 与 `retry.py::__call__` sink 接线;**对冲计数本提交保持 0/False**(T3 才登记)。
|
||||
3. **后绿**:批次 B–F 通过;`pytest tests/unit tests/contracts -q` 全绿不改既有断言。
|
||||
4. 暂存:`src/polygateway/types.py`、`middleware/retry.py`、`embedding.py`、`ocr.py`、五个测试文件。
|
||||
|
||||
### T3 → 提交 3 `feat: add opt-in cross-source hedged requests for chat`
|
||||
|
||||
1. **先红**:批次 G(`test_hedge.py`,对冲未实现 → 挂起用例超时或 `hedges==0` 断言失败)+ H(`test_config.py`,键未识 → `ValueError`/`None` 断言失败)。
|
||||
2. 按 §3.5 改 `admission.py` → §3.6 改 `config.py`/`client.py` → §3.4 改 `retry.py` 编排。
|
||||
3. **后绿**:批次 G/H 通过;**默认关闭回归门**:`pytest tests/unit tests/contracts -q` 全绿且不改一行既有断言(批次 I);`make lint` 通过。
|
||||
4. 暂存:`src/polygateway/middleware/{retry,admission}.py`、`src/polygateway/{config,client}.py`、`tests/unit/test_hedge.py`、`tests/unit/test_config.py`。
|
||||
|
||||
### T4 → 提交 4 `docs: document hedged requests and bare generation time`
|
||||
|
||||
1. `CHANGELOG.md` 未发布段:对冲三句强制措辞——**默认关闭,开启即用配额换延迟**(挂起窗口内 in-flight 翻倍);**输家可能已被上游计费**(est 保留只是闸内保守记账,非上游计量);**对冲只对 chat 生效,embedding/OCR 仅获得 `generation_ms` 计时**。另记 `CallStats` 三字段口径(`generation_ms` 与 `total_latency_ms` 差值 = 波动开销)与 `hedge_cancelled` 标签的遥测 join 用法。
|
||||
2. `README.md`:能力表加"长尾对冲(可选)"一行(三句措辞同上);配置键清单加两键与值域/守卫;`CallStats` 说明处加三字段。
|
||||
3. `.env.example`:`LLM__CALL_DEADLINE_S` 注释行后加 `# LLM__HEDGE__AFTER_S=` 与 `# LLM__HEDGE__MAX_EXTRA=1`(缺省关闭,说明触发语义与成本含义)。
|
||||
4. `research-wiki/findings/2026-09-10-24-hedged-requests-validation.md`:红绿证据、命令与退出码、豁免索引(含 H5 的 v1 单路口径与竞速误贴残留)。
|
||||
5. wiki 注册本计划与 findings(add_entity/add_edge/rebuild_index);独立验证(全新上下文 verifier)与整分支审查在本提交前完成;版本 bump/发布**不在本计划内**。
|
||||
|
||||
## 5. 测试矩阵 → 任务映射
|
||||
|
||||
**设施复用核对(动手前必读)**:`tests/unit/test_retry.py:120-127` 的 `FakeSleep` 只记录不推进时钟——退避推进须用例自带 `async def sleep(s): clock.advance(s)` 闭包;`tests/unit/test_embedding.py:217` 已有 `_ClockAdvancingEmbedTransport`(尝试内推进时钟),批次 D 直接复用;`tests/unit/test_config.py:31-33` 已有 loguru WARNING 捕获 fixture,warning 断言用它(caplog 抓不到 loguru);`tests/unit/test_client.py:134-139` 已有 `InMemoryCache` 命中回路,批次 F 复用;对冲编排用例(`test_hedge.py`)用**真实 loop 钟**(不注入 FakeClock),阈值 0.05s、断言容差 4–10×;取消窗口用 `entered` Event 范式(`test_retry.py:79-89` 既有),禁 sleep 撞窗口。
|
||||
|
||||
| 批次 | 断言(→ 任务) | 落点(精确测试名) |
|
||||
| --- | --- | --- |
|
||||
| A 端口加参 | 流式首 token 置位事件;非流式永不置位;`None` 不观测行为不变;漏传 → `TypeError`(钉住必填)(→T1) | `test_openai_compat.py::test_stream_sets_first_token_event`、`test_non_stream_never_sets_first_token_event`、`test_none_first_token_event_keeps_behavior`、`test_first_token_event_is_required_keyword` |
|
||||
| B 三字段 | 三字段默认值(0/0/False);仅旧三参数构造 `CallStats(...)` 不炸;`record_generation` 覆盖/累加语义;`register_hedge` 计数;`snapshot` 带出三字段(→T2) | `test_types.py::test_callstats_hedge_fields_default`、`test_callcontext_record_generation_overwrite_and_accumulate`、`test_callcontext_register_hedge_counts`、`test_snapshot_includes_hedge_fields` |
|
||||
| C chat 计时 | 脚本 [Transient, ok]:退避推进时钟 5s、成功次 transport 推进 0.2s → `generation_ms == 200` 且 `total_latency_ms ≥ 5200`(证明排除 backoff);无对冲时 `hedges == 0`/`hedge_won is False`(→T2) | `test_retry.py::test_generation_ms_excludes_backoff_and_admission`、`test_generation_ms_zero_hedge_flags_without_hedging`(配 `_GenClockTransport` 薄包装:委托 FakeTransport 并在返回前 `clock.advance(delta)`) |
|
||||
| C2 重问覆盖 | 结构化首轮坏 JSON(transport 推进 1s)、重问轮好 JSON(推进 0.2s)→ `generation_ms == 200`(最后一轮覆盖,非累加)(→T2) | `test_client.py::test_generation_ms_structured_last_round_wins`(复用 `_client(structured_strategy=...)` 与 :129-132 范式;若 `_client` 未暴露 `now` 注入,按其既有模式补 keyword 参数——测试设施非公共面) |
|
||||
| D embedding 计时 | 两批各推进 0.3s → `generation_ms == 600`(批次和)(→T2) | `test_embedding.py::test_generation_ms_sums_batch_transports`(复用 `_ClockAdvancingEmbedTransport`) |
|
||||
| E ocr 计时 | 单次 transport 推进 0.4s → `generation_ms == 400`(→T2) | `test_ocr_client.py::test_generation_ms_single_transport_call`(同款薄包装,推进 `test_ocr_client.py:344` 那份本地 FakeClock——历史坑:不是 contracts 那份) |
|
||||
| F 缓存命中 | 第二次同 key 调用 `generation_ms == 0` 且 `attempts == 0` 且 `cache_hit is True`(→T2) | `test_client.py::test_cache_hit_generation_ms_zero`(复用 :134-139 回路) |
|
||||
| G 对冲编排 | 见下表(→T3) | `tests/unit/test_hedge.py`(新建) |
|
||||
| H 配置守卫 | 两键 env 解析(3 段键不被当源字段、`HEDGE` 在保留段);非法值四路(env/直接构造/`dataclasses.replace`/client 直传)→ `ValueError`;`after_s ≥ min(timeout_s)` → ValueError;`after_s ≥ min(ttft)` → warning;单源 → warning;`after_s ≥ call_deadline_s` → ValueError;`max_extra` 0/"x" → ValueError、2/3 → warning 且生效 1(→T3) | `test_config.py::test_hedge_keys_from_env_skip_source_loader`、`test_hedge_after_s_domain_four_paths`、`test_hedge_guard_below_min_timeout_raises`、`test_hedge_guard_ttft_warns`、`test_hedge_guard_single_source_warns`、`test_hedge_guard_deadline_conflict_raises`、`test_hedge_max_extra_v1_cap`;client 直传入口校验 `test_client.py::test_client_hedge_params_entry_validation` |
|
||||
| I 默认关闭回归 | 不配对冲键时 `tests/unit` + `tests/contracts` 全绿,**不改一行既有断言**(→T3 门) | 全套件 |
|
||||
|
||||
批次 G(`test_hedge.py`)用例全表——设施:两源 scope(`s1`/`s2`),event-driven 假 transport(每源一对 `entered`/`release` Event + 可脚本化"先置 first_token 再挂起"),真实 loop 钟,`hedge_after_s=0.05`:
|
||||
|
||||
| 测试名 | 断言(设计 §8 矩阵编号) |
|
||||
| --- | --- |
|
||||
| `test_non_stream_triggers_hedge_and_fast_leg_wins` | s1 挂起、s2 即时成功:总时长 < 10× 阈值(①);`hedges==1`、`hedge_won is True`;`generation_ms < total_latency_ms` 且 ≥ s2 实际 transport 耗时下界(⑩:不含触发前等待) |
|
||||
| `test_stream_triggers_only_when_first_token_absent` | 两例:首 token 未至 → 触发;假 transport 先 `first_token_event.set()` 再挂起 → **不触发**,`attempts==1`(①) |
|
||||
| `test_hedge_goes_to_other_source` | 对冲请求落在 s2(transport.calls 断言);`logical_call_id` 两行一致(②⑤) |
|
||||
| `test_hedge_silent_when_no_candidate` | s2 permit 预占满 → 不对冲:`hedges==0`、`attempts==1`、原请求放行后正常成功(③) |
|
||||
| `test_hedge_silent_when_single_source` | 单源 scope:运行期自然静默,行为与不配阈值逐字相同(②) |
|
||||
| `test_winner_settles_actual_loser_keeps_est` | memory limiter:赢家源 `tpm_used == 真实 usage`,输家源 `tpm_used == est`(S3 格);调用结束后两源 `inflight == 0`(④) |
|
||||
| `test_loser_row_labelled_hedge_cancelled` | 假 emitter:输家 attempt 行 `error=="hedge_cancelled"`,赢家行无 error;两行 `logical_call_id` 相同;无 `terminal_failure` 行(④⑤) |
|
||||
| `test_loser_does_not_feed_breaker` | memory gate:挂起源 `failure_count` 不变、健康喂数无 `ok=False`;赢家照常 `record_success`(④,§3 关键判断) |
|
||||
| `test_attempts_two_and_no_task_leak` | `call_stats.attempts == 2`;返回后 `asyncio.all_tasks()` 无本调用残留任务(⑤) |
|
||||
| `test_external_cancel_cancels_both_legs` | 两路均挂起,`entered` 双置位后 `task.cancel()`:`CancelledError` 上抛;两行 attempt 均 `"cancelled"`(标记只在赢家产生后置,外部取消无 `hedge_cancelled`);两 permit 释放(⑦) |
|
||||
| `test_deadline_cuts_hedged_tree` | client 级 `call_deadline_s=0.2` + 两路挂起 → `CallDeadlineExceeded`;两 permit 释放(⑧) |
|
||||
| `test_primary_late_success_wins_back` | s1 挂 0.3s 后成功、s2 对冲路挂起:对冲已触发但原路先完成 → `hedge_won is False`、`generation_ms` 为原路时长、s2 行 `hedge_cancelled`(⑩ 取快者) |
|
||||
| `test_both_fail_counts_budget_once` | 两路 Transient:`max_attempts=2` 时恰进第二轮(两败只计一次);最终 `retry_exhausted` 在第二轮两败后(H6) |
|
||||
| `test_both_429_refund_no_budget` | 两路 429:不耗预算(`max_attempts=1` 不抛 `retry_exhausted`),stall 账退还——小 `stall_window_s` 下终局 `reason=="stalled"` 而非 `"retry_exhausted"` |
|
||||
| `test_mixed_429_and_failure_counts_budget` | 一路 429 一路 Transient → 计一次预算、不退还 stall 账(§3.4 `_combine_failures`) |
|
||||
|
||||
命令(全部 `conda run -n PolyGateway`,禁接管道):`pytest tests/unit/test_openai_compat.py -q`、`pytest tests/unit -q`、`pytest tests/unit/test_hedge.py -q`、`pytest tests/unit tests/contracts -q`、`make lint`。真实 Redis/网关 slow 用例本计划不新增、不跑,由发布清单第 4 步按 diff 交集选子集(本 diff 触及 retry/限流结算路径,Redis 时间语义变体届时在交集内)。
|
||||
|
||||
## 6. 阻塞矩阵与交接
|
||||
|
||||
| 触发条件 | 处置 |
|
||||
| --- | --- |
|
||||
| 需要新增本计划外的公共键/端口方法/遥测列/异常类 | **停下上报**(设计 §9 边界之外即未批准) |
|
||||
| `asyncio.current_task().cancelling()` 在目标 Python 版本语义不符 | 3.12 语义同 136 探针已验证的取消计数;若实测不符,改用"取消标志位置于 `_attempt_hedged` 局部"方案并记入 findings,不得吞取消 |
|
||||
| 两路同时成功的竞速在测试中无法确定性构造 | 用双 Event 栅栏(两 transport 都等同一放行事件)构造;仍不可得则记入 findings 豁免索引,不得删"原路优先"断言 |
|
||||
| 批次 G 计时断言在 CI 机器抖动 | 只断言下界与相对比较(`generation_ms < total_latency_ms`、总时长 < 10× 阈值),不断言精确值;精确值断言只在注入钟批次(C–F) |
|
||||
| 想顺手让对冲路再触发梯次对冲 | **不做**(H5:v1 单路;`hedge` 任务恒传 `first_token_event=None`) |
|
||||
| 想把输家记进熔断/健康分 | **不记**(§3 关键判断:挂起 ≠ 源死亡);运维面靠 `hedge_cancelled` 遥测行统计 |
|
||||
| `hedge_max_extra > 1` 应 warning 还是 ValueError 存疑 | 本计划取 warning(§5 值域 [1,3] 与 H5 "v1 只允许 1" 的并存解);若人类审定应 ValueError,改 `check_hedge_assembly` 一处 + 批次 H 一条断言 |
|
||||
| 发现 embedding/OCR 也想加对冲参数 | **不加**(非目标 A,H7);登记为后续 issue |
|
||||
|
||||
交接物:4 个提交、1 份 findings、CHANGELOG 未发布段。版本号 bump、tag、构建、上传 registry 与 wiki 同步**不在本计划内**,按 CLAUDE.md §4.4.1 另行执行。
|
||||
|
||||
## 7. 自审
|
||||
|
||||
| 检查 | 结论 |
|
||||
| --- | --- |
|
||||
| 路径/行号/签名是否可执行无 TBD | 是——接入点均现读:`ports.py:51-60`、`openai_compat.py:426/552/573-575/646`、`retry.py:244/248-256/270/291-300/327-334`、`admission.py:171-213/224`、`types.py:296-318/321-360`、`config.py:57/190/201/396/405/695`、`client.py:239/245-247/254-273/511`、`embedding.py:373/386`、`ocr.py:397`、六个 fake transport 精确行号 |
|
||||
| 是否复用而非重造 | 是——取消结算 S3 格、准入全链路、`settle_and_release`、`asyncio.wait` 范式、`ensure_call_deadline` 值域校验、`_ClockAdvancingEmbedTransport`/loguru 捕获 fixture/InMemoryCache 回路全部复用;新增仅 1 测试文件 + 2 配置键 + 3 字段 |
|
||||
| 先失败后通过证据点 | 是——T1 批次 A(TypeError)、T2 批次 B–F(字段缺失/恒 0)、T3 批次 G/H(未实现/键未识)均先红 |
|
||||
| 取消与降级铁律 | 取消穿透路径显式收口不吞没;准入失败静默(设计 §4.3 批准);后端不可用照常冒泡;无 shield/后台任务 |
|
||||
| 反 gold-plating | 四分类/熔断语义/429 分账/Lua/deadline.py/遥测列/embedding-OCR 对冲/分位数/同源/per-call 参数一律不碰;`hedge_max_extra` 不下传 RetryMW(v1 无消费者) |
|
||||
| 跨任务签名一致 | `first_token_event`(T1 端口 → T3 接线)、`generation_sink`/`record_generation`(T2 定义,T3 编排消费)、`check_hedge_assembly`(config 定义,client 消费)三处接缝均在 §3 写出实际代码 |
|
||||
| 残余诚实标注 | 输家 est 保留 ≠ 上游真实计费计量;非流式误对冲慢生物理不可分;竞速误贴标签可接受(设计 §4.5);`hedge_max_extra` v1 生效口径取 warning(§6 阻塞矩阵已列复核点) |
|
||||
@@ -1,11 +1,11 @@
|
||||
---
|
||||
type: schema
|
||||
node_id: schema:llm-calls
|
||||
title: "表结构: llm_calls(遥测 26 字段)"
|
||||
title: "表结构: llm_calls(遥测 36 字段)"
|
||||
date: 2026-07-20
|
||||
---
|
||||
|
||||
# 表结构: llm_calls(遥测 26 字段)
|
||||
# 表结构: llm_calls(遥测 36 字段)
|
||||
|
||||
## 列定义(冻结,M1 设计 §4.4 / ARCH §7.8)
|
||||
|
||||
@@ -31,6 +31,37 @@ date: 2026-07-20
|
||||
| meta | TEXT / JSONB NOT NULL DEFAULT '' / '{}' | 调用方自定义维度(同批,≤16 个 KV);SQLite 存 canonical JSON 串,PG 存 JSONB |
|
||||
| thinking_observation | TEXT | 本次推理是否真的发生的三态裁定(2026-08-25,issue #16/#17);`observed` / `absent` / `unknown`。见下方口径 |
|
||||
| reasoning_effort | TEXT | 按四种行来源记录的推理意图/实际编码档位(2026-09-09 澄清,issue #20/#26);八档 `Effort` 字面量之一,NULL = 调用方未表态(与 `none`「明确要求不推理」不可混同)。见下方口径 |
|
||||
| scope | TEXT | 池名(2026-09-09,issue #19/#23);Emitter 构造期注入,三类行都带,**不拿 `source_name` 顶替** |
|
||||
| operation | TEXT | `chat` / `embed` / `recognize_text` / `parse_layout`,由调用点给定;与 `PolyGatewayError.operation` 是两个语义,链路上不得读后者填本列 |
|
||||
| logical_call_id | TEXT | 一次逻辑调用一个 ID(重试/换源/重问/分批共享);上下文缺席落 NULL,**不造 ID** |
|
||||
| event_kind | TEXT | 三态 `attempt` / `cache_hit` / `terminal_failure`——三类行的唯一机械判据。见下方口径 |
|
||||
| http_status_code | INTEGER | 失败 attempt 行的实收状态码(中转改写过就记改写后的,**不猜回原值**);成功行与终态行恒 NULL,且 200 也可能是失败行(MonkeyOCR `success != true`) |
|
||||
| error_type | TEXT | 该行自身错误的类名;取消路径传字符串故为 NULL |
|
||||
| cause_type | TEXT | `__cause__` 的类名(如 `ReadTimeout`)——httpx 超时类的 `str()` 为空,只靠 error 列分不出是哪种超时;仅失败 attempt 行非空 |
|
||||
| error_body | TEXT | 网关响应正文摘要(`summarize_body` 上限,**不在 `PGW_TELEMETRY_TEXT_CAP` 覆盖面内**);仅失败 attempt 行非空 |
|
||||
| attempts | INTEGER | 该逻辑调用真实打出去的尝试次数(免预算 429 也计);**只属终态行** |
|
||||
| total_latency_ms | INTEGER | 该逻辑调用的总墙钟(含缓存 IO、退避、准入等待、重问);**只属终态行**,与该行 `latency_ms` 同取一份冻结快照 |
|
||||
|
||||
## 三类行与失败归因口径(2026-09-09,issue #19/#23)
|
||||
|
||||
遥测行不再只有"一次尝试"一种。`event_kind` 三态同时决定其余列的取值面:
|
||||
|
||||
| event_kind | 什么时候写 | 列取值 |
|
||||
| --- | --- | --- |
|
||||
| `attempt` | 每次真实尝试(含失败与取消) | 失败时诊断四列可非空;`attempts`/`total_latency_ms` NULL |
|
||||
| `cache_hit` | 缓存命中 | 诊断四列与快照两列全 NULL |
|
||||
| `terminal_failure` | 每次**领域失败**的整池终态,每逻辑调用至多一条 | `attempts`/`total_latency_ms` 非空;`http_status_code`/`cause_type`/`error_body` 恒 NULL;`cost` NULL、`usage_source='unavailable'`、token 0 |
|
||||
|
||||
两条不变量: 每次领域失败至多一条终态行(I3,`claim_terminal()` 去重);非领域异常(编程错)**零条**终态行、原样传播(I4)。
|
||||
|
||||
**终态行三列恒 NULL 是红线**: 把最后一次 attempt 的状态码与正文搬上来,等于拿最后一个源冒充整池归因。逐源现场由同一 `logical_call_id` 的 attempt 行给出:
|
||||
|
||||
```sql
|
||||
SELECT event_kind, source_name, http_status_code, error_type, cause_type, error, error_body
|
||||
FROM llm_calls WHERE logical_call_id = :lcid ORDER BY created_at;
|
||||
```
|
||||
|
||||
下游口径迁移四条: ① 计失败调用改 `WHERE event_kind = 'terminal_failure'`;② `error IS NOT NULL` 不再是失败调用判据(跨两类行);③ `AVG(latency_ms)` 须按 `event_kind` 分组(终态行是整个逻辑调用的总耗时);④ 费用口径不变(终态行 cost 恒 NULL)。
|
||||
|
||||
## usage/成本口径(2026-07-30,est_tokens 解耦)
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ from polygateway.config import EmbeddingSettings, GatewaySettings, OcrSettings
|
||||
from polygateway.embedding import EmbeddingClient
|
||||
from polygateway.errors import (
|
||||
AllSourcesExhausted,
|
||||
CallDeadlineExceeded,
|
||||
CircuitOpenError,
|
||||
GatewayUnavailableError,
|
||||
GovernanceBackendError,
|
||||
@@ -39,6 +40,7 @@ from polygateway.thinking import (
|
||||
)
|
||||
from polygateway.types import (
|
||||
EFFORT_ORDER,
|
||||
CallStats,
|
||||
Effort,
|
||||
EmbeddingResponse,
|
||||
LLMResponse,
|
||||
@@ -50,13 +52,15 @@ from polygateway.types import (
|
||||
ThinkingObservation,
|
||||
)
|
||||
|
||||
__version__ = "1.3.4"
|
||||
__version__ = "1.3.7"
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_PROFILES",
|
||||
"EFFORT_ORDER",
|
||||
"Effort",
|
||||
"AllSourcesExhausted",
|
||||
"CallStats",
|
||||
"CallDeadlineExceeded",
|
||||
"CircuitOpenError",
|
||||
"EmbeddingClient",
|
||||
"EmbeddingResponse",
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import dataclasses
|
||||
import hashlib
|
||||
import json
|
||||
import random
|
||||
@@ -18,12 +19,14 @@ from typing import TYPE_CHECKING, Any, Literal
|
||||
from polygateway.backends.memory.breaker import InMemoryGate
|
||||
from polygateway.backends.memory.cache import InMemoryCache
|
||||
from polygateway.backends.memory.limiter import InMemoryLimiter
|
||||
from polygateway.config import GatewaySettings
|
||||
from polygateway.config import GatewaySettings, check_hedge_assembly
|
||||
from polygateway.deadline import ensure_call_deadline, with_call_deadline
|
||||
from polygateway.errors import PolyGatewayError
|
||||
from polygateway.middleware.base import compose
|
||||
from polygateway.middleware.cache import CacheMW
|
||||
from polygateway.middleware.retry import RetryMW
|
||||
from polygateway.middleware.structured import StructuredMW
|
||||
from polygateway.middleware.telemetry import TelemetryEmitter, TelemetryMW
|
||||
from polygateway.middleware.telemetry import TelemetryEmitter, TelemetryMW, emit_terminal_once
|
||||
from polygateway.ports import TelemetryStatusProvider
|
||||
from polygateway.pricing import PricingTable
|
||||
from polygateway.providers import get_provider
|
||||
@@ -46,6 +49,7 @@ from polygateway.types import (
|
||||
Effort,
|
||||
LLMResponse,
|
||||
TelemetryStatus,
|
||||
_CallContext,
|
||||
coerce_effort,
|
||||
validate_caller_dimensions,
|
||||
validate_request_overlay,
|
||||
@@ -232,12 +236,28 @@ class GatewayClient:
|
||||
structured_strategy: StructuredOutputStrategy | None = None,
|
||||
structured_escalation: StructuredOutputStrategy | None = None,
|
||||
structured_max_retries: int = 1,
|
||||
call_deadline_s: float | None = None,
|
||||
hedge_after_s: float | None = None,
|
||||
hedge_max_extra: int = 1,
|
||||
now: Any = time.monotonic,
|
||||
sleep: Any = asyncio.sleep,
|
||||
rng: Any = random.random,
|
||||
) -> None:
|
||||
# 入口即校: 装配错误当场报,不等到第一次调用才炸
|
||||
self._call_deadline_s = ensure_call_deadline(
|
||||
call_deadline_s, "GatewayClient(call_deadline_s=...)"
|
||||
)
|
||||
# 对冲守卫与 GatewaySettings 共用同一份(issue #24 H4): 直接构造这条路
|
||||
# 不经过 settings,值域/交叉守卫若只挂在 settings 上就会被它绕过
|
||||
self._hedge_after_s = check_hedge_assembly(
|
||||
hedge_after_s=hedge_after_s,
|
||||
hedge_max_extra=hedge_max_extra,
|
||||
sources=sources,
|
||||
call_deadline_s=self._call_deadline_s,
|
||||
origin="GatewayClient(hedge_after_s=...)",
|
||||
)
|
||||
emitter = (
|
||||
TelemetryEmitter(telemetry, pricing=pricing, text_cap=text_cap)
|
||||
TelemetryEmitter(telemetry, scope=scope, pricing=pricing, text_cap=text_cap)
|
||||
if telemetry is not None
|
||||
else None
|
||||
)
|
||||
@@ -258,6 +278,8 @@ class GatewayClient:
|
||||
ceiling=float(max([64, *(s.max_concurrency for s in sources if s.max_concurrency)]))
|
||||
),
|
||||
emitter=emitter,
|
||||
# max_extra 不下传(issue #24 H5): v1 编排固定单路对冲,无消费者
|
||||
hedge_after_s=self._hedge_after_s,
|
||||
now=now,
|
||||
sleep=sleep,
|
||||
rng=rng,
|
||||
@@ -289,6 +311,12 @@ class GatewayClient:
|
||||
self._structured_available = structured_strategy is not None
|
||||
self._terminal = terminal # 内部引用: 装配自省/测试用
|
||||
self._handler = compose(middlewares, terminal)
|
||||
# 期限到期需要报出 scope(现之前只传给 RetryMW,未自存)
|
||||
self._scope = scope
|
||||
# 逻辑调用统计需要同一只注入钟(1.3.5);现之前只传给中间件未自存
|
||||
self._now = now
|
||||
# 终态行由公开边界统一写出(T3),故边界也需持有 emitter
|
||||
self._emitter = emitter
|
||||
self._transport = transport
|
||||
self._telemetry = telemetry
|
||||
self._cache = cache
|
||||
@@ -329,6 +357,7 @@ class GatewayClient:
|
||||
reasoning_effort: Effort | str | None = None,
|
||||
tenant_id: str | None = None,
|
||||
meta: Mapping[str, Any] | None = None,
|
||||
call_deadline_s: float | None = None,
|
||||
) -> LLMResponse:
|
||||
"""一次治理调用(签名冻结,ARCH §5.2;与三项目 LLMProvider 协议兼容)。
|
||||
|
||||
@@ -344,6 +373,11 @@ class GatewayClient:
|
||||
`tenant_id` 与 `meta` 是调用方自定义维度,只进遥测、**不进缓存 key**
|
||||
(租户隔离由 `cache_namespace` 负责,ARCH §7.5);前者享有真实列待遇
|
||||
(可挂 RLS、可进复合索引),后者是任意 KV 容器(issue #11)。
|
||||
|
||||
`call_deadline_s` 是本次调用的墙钟硬边界(issue #22): `None` = 继承装配值,
|
||||
正数 = 本次覆盖,**不提供"本次关闭"**。它治理的是**等待**: 到期抛
|
||||
`CallDeadlineExceeded`,但到期**不等于未产出、未计费**——在途请求可能已发出、
|
||||
已被上游计费,且清理仍在 `finally` 里完成,故返回时刻 = 期限 + 清理耗时。
|
||||
"""
|
||||
if structured is not None and not self._structured_available:
|
||||
raise ImportError(
|
||||
@@ -370,6 +404,15 @@ class GatewayClient:
|
||||
else coerce_effort(reasoning_effort, origin="chat(reasoning_effort=...)")
|
||||
)
|
||||
validate_thinking_raw(sampling, effort=effort, wire=None, origin="chat overlay")
|
||||
# 期限取值与校验必须在创建 awaitable **之前**: 否则非法值抛错时会遗留
|
||||
# 未 await 的协程(RuntimeWarning + 资源不释放)
|
||||
deadline = (
|
||||
self._call_deadline_s
|
||||
if call_deadline_s is None
|
||||
else ensure_call_deadline(call_deadline_s, "chat(call_deadline_s=...)")
|
||||
)
|
||||
# 三项校验均已通过 → 进入统计边界(设计 §3: 输入校验异常在边界之外,保持原行为)
|
||||
context = _CallContext(now=self._now)
|
||||
request = ChatRequest(
|
||||
messages=messages,
|
||||
session_id=session_id,
|
||||
@@ -383,8 +426,33 @@ class GatewayClient:
|
||||
reasoning_effort=effort,
|
||||
tenant_id=dimension_tenant_id,
|
||||
meta=dimensions,
|
||||
call_context=context,
|
||||
)
|
||||
return await self._handler(request)
|
||||
try:
|
||||
response = await with_call_deadline(
|
||||
self._handler(request), deadline_s=deadline, scope=self._scope
|
||||
)
|
||||
except PolyGatewayError as exc:
|
||||
# 统计边界内的一切领域失败均尝试写一条终态行(1.3.5 设计 §6 I3),
|
||||
# 包括已有 attempt 错误行的 RequestRejected / ResultInvalid——两类行描述
|
||||
# 的不是同一件事(尝试 vs 逻辑终态),由 `event_kind` 区分
|
||||
await emit_terminal_once(
|
||||
self._emitter, request=request, context=context, error=exc, operation="chat"
|
||||
)
|
||||
raise
|
||||
except asyncio.CancelledError:
|
||||
# 尽力而为且**取消优先**: 不 shield、不开后台任务;写入那一次 await 上
|
||||
# 再被取消则 `CancelledError` 照常传播(与 TelemetryMW 历史行为同款)
|
||||
await emit_terminal_once(
|
||||
self._emitter,
|
||||
request=request,
|
||||
context=context,
|
||||
error="cancelled",
|
||||
operation="chat",
|
||||
)
|
||||
raise
|
||||
# 快照在返回前冻结: 故它含缓存命中路径与已完成的内联遥测耗时
|
||||
return dataclasses.replace(response, call_stats=context.snapshot())
|
||||
|
||||
async def aclose(self) -> None:
|
||||
"""幂等释放**自建**资源: transport、遥测、缓存、限流/熔断后端。
|
||||
@@ -453,6 +521,9 @@ class GatewayClient:
|
||||
structured_strategy=strategy,
|
||||
structured_escalation=escalation,
|
||||
structured_max_retries=settings.structured_max_retries,
|
||||
call_deadline_s=settings.call_deadline_s,
|
||||
hedge_after_s=settings.hedge_after_s,
|
||||
hedge_max_extra=settings.hedge_max_extra,
|
||||
)
|
||||
_mark_owned_components(client, limiter=limiter, breaker=breaker, telemetry=telemetry)
|
||||
client._owns_cache = cache is None # 缓存后端可以是 None(backend=none),helper 会跳过
|
||||
@@ -578,6 +649,7 @@ async def gather_bounded[T](aws: Iterable[Awaitable[T]], *, concurrency: int) ->
|
||||
"""有界并发 gather(D5 便利函数,替代 VT 手搓 semaphore+gather 样板)。
|
||||
|
||||
语义与 `asyncio.gather` 默认一致: 结果保序、首个异常上抛;仅增加并发上限。
|
||||
期限计时从每次调用真正开始执行起算,信号量排队时长不在 `call_deadline_s` 之内。
|
||||
"""
|
||||
if concurrency < 1:
|
||||
raise ValueError("concurrency 必须 ≥ 1")
|
||||
|
||||
+179
-2
@@ -19,6 +19,7 @@ from typing import TYPE_CHECKING
|
||||
from dotenv import dotenv_values
|
||||
from loguru import logger
|
||||
|
||||
from polygateway.deadline import ensure_call_deadline
|
||||
from polygateway.types import (
|
||||
BackpressurePolicy,
|
||||
BreakerConfig,
|
||||
@@ -29,7 +30,7 @@ from polygateway.types import (
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Mapping
|
||||
from collections.abc import Mapping, Sequence
|
||||
|
||||
# FIELD → (SourceConfig 属性, 类型);CHS config.py:95-104 全集 + M1 新增
|
||||
_SOURCE_FIELDS: dict[str, tuple[str, str]] = {
|
||||
@@ -53,7 +54,7 @@ _SOURCE_FIELDS: dict[str, tuple[str, str]] = {
|
||||
"TRUST_ENV": ("trust_env", "bool"),
|
||||
"EXTRA_BODY": ("extra_body", "json"),
|
||||
}
|
||||
_RESERVED_SEGMENTS = frozenset({"GLOBAL", "RETRY", "BREAKER", "BACKPRESSURE"})
|
||||
_RESERVED_SEGMENTS = frozenset({"GLOBAL", "RETRY", "BREAKER", "BACKPRESSURE", "HEDGE"})
|
||||
_SELECTORS = frozenset({"round_robin", "least_inflight", "health_aware"})
|
||||
_QUOTA_FULL = frozenset({"wait", "fail_fast"})
|
||||
# 熔断全拒时的处置(issue #14);值域与 _QUOTA_FULL 相同但语义不同——配额满是
|
||||
@@ -182,6 +183,17 @@ class GatewaySettings:
|
||||
pricing_path: str | None
|
||||
structured_max_retries: int
|
||||
lease_ttl_s: float
|
||||
# 一次逻辑调用的**可选**墙钟硬边界(issue #22)。缺省 None = 不启用,行为逐字
|
||||
# 等于 1.3.5;有默认值故追加在末尾,不扰动既有位置构造。`EmbeddingSettings.gateway`
|
||||
# 与 `OcrSettings.gateway` 自动继承。值域由 `_validate_call_deadline` 把关,
|
||||
# 直接构造、`dataclasses.replace` 与 env 三条路一致
|
||||
call_deadline_s: float | None = None
|
||||
# 长尾对冲(issue #24 H4): 挂起超阈值时并发向异源再发一次,先回者赢、输家取消。
|
||||
# 缺省 None = 关闭,行为逐字等于 1.3.6。`hedge_max_extra` 值域 [1,3] 为 H5 梯次
|
||||
# 预留,v1 仅单路生效(>1 装配期 warning);**不下传 RetryMW**。值域与交叉守卫
|
||||
# 由 `check_hedge_assembly` 单一定义点把关,直接构造/replace/env 三路一致
|
||||
hedge_after_s: float | None = None
|
||||
hedge_max_extra: int = 1
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
self._normalize()
|
||||
@@ -192,6 +204,8 @@ class GatewaySettings:
|
||||
self._validate_lease()
|
||||
self._validate_stall()
|
||||
self._validate_probe()
|
||||
self._validate_call_deadline()
|
||||
self._validate_hedge()
|
||||
|
||||
def _normalize(self) -> None:
|
||||
"""把 `from_env` 一直在做的规范化补到构造路上,两条路必须产出同一个值。
|
||||
@@ -345,6 +359,37 @@ class GatewaySettings:
|
||||
f"timeout_s + {_PROBE_GRACE_S}({floor});调大 probe_ttl_s 或调小源的 timeout_s"
|
||||
)
|
||||
|
||||
def _validate_call_deadline(self) -> None:
|
||||
"""期限值域守卫: 盖住直接构造与 `dataclasses.replace` 两条路(issue #22)。
|
||||
|
||||
env 路已在 `_load_call_deadline` 里带真实键名报过错, 此处对合法值是幂等空操作。
|
||||
**不校验**它与 `timeout_s`/`stall_window_s` 的大小关系: 期限短于单次超时
|
||||
是调用方的合法选择(要的就是“不让这次调用拖过 N 秒”)。
|
||||
"""
|
||||
object.__setattr__(
|
||||
self,
|
||||
"call_deadline_s",
|
||||
ensure_call_deadline(self.call_deadline_s, "GatewaySettings.call_deadline_s"),
|
||||
)
|
||||
|
||||
def _validate_hedge(self) -> None:
|
||||
"""对冲装配守卫(issue #24 H4): 与期限同款,盖住直接构造与 replace 两条路。
|
||||
|
||||
env 路的值域错误已在 `_load_hedge` 里带真实键名报过;此处对合法值是幂等
|
||||
空操作,交叉守卫(阈值 vs timeout/deadline/ttft、单源)只在这里有一处。
|
||||
"""
|
||||
object.__setattr__(
|
||||
self,
|
||||
"hedge_after_s",
|
||||
check_hedge_assembly(
|
||||
hedge_after_s=self.hedge_after_s,
|
||||
hedge_max_extra=self.hedge_max_extra,
|
||||
sources=self.sources,
|
||||
call_deadline_s=self.call_deadline_s,
|
||||
origin="GatewaySettings.hedge_after_s",
|
||||
),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_env(
|
||||
cls,
|
||||
@@ -373,6 +418,8 @@ class GatewaySettings:
|
||||
selector=_load_choice(env, f"{scope_u}__SELECTOR", _SELECTORS, "health_aware"),
|
||||
quota_full=_load_choice(env, f"{scope_u}__QUOTA_FULL", _QUOTA_FULL, "wait"),
|
||||
circuit_open=_load_choice(env, f"{scope_u}__CIRCUIT_OPEN", _CIRCUIT_OPEN, "fail_fast"),
|
||||
call_deadline_s=_load_call_deadline(scope_u, env),
|
||||
**_load_hedge(scope_u, env),
|
||||
**_load_pgw(env),
|
||||
)
|
||||
|
||||
@@ -671,6 +718,136 @@ def _load_lease_ttl(env: Mapping[str, str]) -> float:
|
||||
return float(_cast(found[1], "float", found[0])) if found else _DEFAULT_LEASE_TTL_S
|
||||
|
||||
|
||||
def _load_call_deadline(scope: str, env: Mapping[str, str]) -> float | None:
|
||||
"""读 `{SCOPE}__CALL_DEADLINE_S`(issue #22);键未设即 None = 不启用。
|
||||
|
||||
用 `_first` 而非 `_require`: 后者会把"未设"当成配置缺失报错,对存量下游
|
||||
就是破坏性变更。键名两段式(`split("__")` 长度 2 ≠ 4),故 `_load_sources`
|
||||
天然跳过它,不必进 `_RESERVED_SEGMENTS`。
|
||||
|
||||
origin 传**实际命中的 env 键名**而非字段名: `_cast` 只接得住"不是数字",
|
||||
`0`/负数/`inf` 会穿过它落到 `ensure_call_deadline`——那时报一条指向字段名的错误,
|
||||
在多 scope 部署里无法定位是哪个键写错了。
|
||||
|
||||
Args:
|
||||
scope: 已大写的 scope 名。
|
||||
env: 已合并的环境映射。
|
||||
|
||||
Returns:
|
||||
一次逻辑调用的墙钟期限(秒);键未设或为空串时返回 None(不启用)。
|
||||
"""
|
||||
found = _first(env, f"{scope}__CALL_DEADLINE_S")
|
||||
if found is None:
|
||||
return None
|
||||
return ensure_call_deadline(_cast(found[1], "float", found[0]), found[0])
|
||||
|
||||
|
||||
def _load_hedge(scope: str, env: Mapping[str, str]) -> dict[str, object]:
|
||||
"""读 `{SCOPE}__HEDGE__AFTER_S`/`{SCOPE}__HEDGE__MAX_EXTRA`(issue #24 H4)。
|
||||
|
||||
两键均为 3 段键(`split("__")` 长度 3 ≠ 4),`_load_sources` 的段数判据天然
|
||||
跳过它们;`HEDGE` 已进 `_RESERVED_SEGMENTS`,4 段的 `{SCOPE}__HEDGE__{N}__*`
|
||||
也不会被当成 provider 段造出源。`AFTER_S` 未设 = 关闭(缺省逐字等于 1.3.6);
|
||||
`MAX_EXTRA` 未设 = 1。origin 传实际命中键名(同 `_load_call_deadline` 纪律);
|
||||
值域的交叉守卫(ttft/单源/max_extra 生效口径)归 `check_hedge_assembly` 一处。
|
||||
|
||||
Args:
|
||||
scope: 已大写的 scope 名。
|
||||
env: 已合并的环境映射。
|
||||
|
||||
Returns:
|
||||
`{"hedge_after_s": float | None, "hedge_max_extra": int}`,直传构造器。
|
||||
"""
|
||||
found_after = _first(env, f"{scope}__HEDGE__AFTER_S")
|
||||
after = (
|
||||
ensure_call_deadline(_cast(found_after[1], "float", found_after[0]), found_after[0])
|
||||
if found_after
|
||||
else None
|
||||
)
|
||||
found_extra = _first(env, f"{scope}__HEDGE__MAX_EXTRA")
|
||||
extra = int(_cast(found_extra[1], "int", found_extra[0])) if found_extra else 1
|
||||
return {"hedge_after_s": after, "hedge_max_extra": extra}
|
||||
|
||||
|
||||
def check_hedge_assembly(
|
||||
*,
|
||||
hedge_after_s: float | None,
|
||||
hedge_max_extra: int,
|
||||
sources: Sequence[SourceConfig],
|
||||
call_deadline_s: float | None,
|
||||
origin: str,
|
||||
) -> float | None:
|
||||
"""对冲装配守卫的唯一事实源(issue #24 设计 §5 全表,H4 批准)。
|
||||
|
||||
`GatewaySettings.__post_init__` 与 `GatewayClient.__init__` 调同一份,两条装配
|
||||
路的值域/交叉守卫不漂移。返回归一化后的 `hedge_after_s`(None 或有限正数,
|
||||
复用 `ensure_call_deadline` 的值域校验);`hedge_after_s is None`(未启用)时
|
||||
值域归一化后直接返回,交叉守卫不查——它们没有可校验的对象。
|
||||
|
||||
Args:
|
||||
hedge_after_s: 对冲触发阈值(秒);None = 关闭。
|
||||
hedge_max_extra: 每次逻辑调用最多并发对冲路数;v1 仅单路生效(H5)。
|
||||
sources: 本 scope 的源集合(交叉守卫要读 timeout_s/ttft_timeout_s)。
|
||||
call_deadline_s: 调用期限(秒);与对冲的组合守卫见设计 §6。
|
||||
origin: after_s 值域报错的定位串(env 键名 / `GatewaySettings.hedge_after_s` /
|
||||
`GatewayClient(hedge_after_s=...)`);跨字段守卫与 warning 沿用本模块先例,
|
||||
消息自带字段名与 env 键型,不挂 origin。
|
||||
|
||||
Raises:
|
||||
ValueError: max_extra 非 int/bool 或出 [1,3];阈值 ≥ 最小源 timeout_s(永不
|
||||
可能触发);阈值 ≥ call_deadline_s(期限先于对冲触发,对冲形同虚设)。
|
||||
"""
|
||||
if isinstance(hedge_max_extra, bool) or not isinstance(hedge_max_extra, int):
|
||||
raise ValueError(
|
||||
f"hedge_max_extra({{SCOPE}}__HEDGE__MAX_EXTRA)必须是 int: {hedge_max_extra!r}"
|
||||
)
|
||||
if not 1 <= hedge_max_extra <= 3:
|
||||
raise ValueError(
|
||||
f"hedge_max_extra({{SCOPE}}__HEDGE__MAX_EXTRA)须在 [1,3]: {hedge_max_extra};"
|
||||
"每次逻辑调用最多并发对冲路数,v1 仅单路生效"
|
||||
)
|
||||
after = ensure_call_deadline(hedge_after_s, origin)
|
||||
if after is None:
|
||||
return None
|
||||
if not sources:
|
||||
# 直传路(GatewayClient(sources=[], hedge_after_s=...))没有 settings 的
|
||||
# 非空守卫先行拦截;报错必须定位到 hedge,而不是裸 min() 空序列异常
|
||||
raise ValueError(
|
||||
"hedge_after_s({SCOPE}__HEDGE__AFTER_S)的交叉守卫要求 sources 不能为空;"
|
||||
"对冲已启用但没有可校验的源"
|
||||
)
|
||||
min_timeout = min(s.timeout_s for s in sources)
|
||||
if after >= min_timeout:
|
||||
raise ValueError(
|
||||
f"hedge_after_s({{SCOPE}}__HEDGE__AFTER_S={after})须 < 最小源 timeout_s"
|
||||
f"({min_timeout});对冲永不可能触发,配置即错误"
|
||||
)
|
||||
if call_deadline_s is not None and after >= call_deadline_s:
|
||||
raise ValueError(
|
||||
f"hedge_after_s({after})须 < call_deadline_s({call_deadline_s});"
|
||||
"期限会先于对冲触发,对冲形同虚设(设计 §6)"
|
||||
)
|
||||
ttfts = [s.ttft_timeout_s for s in sources if s.ttft_timeout_s is not None]
|
||||
if ttfts and after >= min(ttfts):
|
||||
logger.warning(
|
||||
"hedge_after_s({}) ≥ 最小源 ttft_timeout_s({}): 流式挂起会被 TTFT 看门狗"
|
||||
"先行切断,对冲对流式形同虚设(非流式仍有效)",
|
||||
after,
|
||||
min(ttfts),
|
||||
)
|
||||
if len(sources) == 1:
|
||||
logger.warning(
|
||||
"单源 scope 配置了对冲阈值 hedge_after_s={};运行期拿不到异源候选,对冲自然静默",
|
||||
after,
|
||||
)
|
||||
if hedge_max_extra > 1:
|
||||
logger.warning(
|
||||
"hedge_max_extra={} 已接受,但 v1 仅单路对冲生效(梯次追加为 H5 预留)",
|
||||
hedge_max_extra,
|
||||
)
|
||||
return after
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EmbeddingSettings:
|
||||
"""Embedding scope 装配配置(M2 §7): 复用 GatewaySettings + embedding 专用键。
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
"""一次逻辑调用的**可选**墙钟硬边界(issue #22;1.3.6 设计 §3 方案 A)。
|
||||
|
||||
只依赖标准库与 `errors.py`(依赖铁律最内层),供三个公开边界各包一次:
|
||||
期限治理的是**等待**,不是"到期即无副作用"——在途请求可能已发出、已被上游
|
||||
计费,清理照旧在 `finally` 完成,故返回时刻 = 期限 + 清理耗时。
|
||||
|
||||
缺省 `None` 时**完全不进上下文管理器**,行为逐字等于 1.3.5。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import math
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from polygateway.errors import CallDeadlineExceeded
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Awaitable
|
||||
|
||||
|
||||
def ensure_call_deadline(value: object, origin: str) -> float | None:
|
||||
"""全装配路径共用的期限值域校验: `None` 或**有限正数秒**,否则当场 `ValueError`。
|
||||
|
||||
装配错误不属降级面(缺失/非法配置直接报错,不静默取默认值)。`bool` 必须先判:
|
||||
`isinstance(True, int)` 为真,放行会让 `call_deadline_s=True` 变成"1 秒期限"
|
||||
这种没人写得出来的意图。巨大 int(如 `10**400`)超出 float 值域,`float()` 会抛
|
||||
`OverflowError`——它不是 `ValueError` 的子类,泄漏出去会绕过调用方的
|
||||
`except ValueError`,故在此统一成同一种装配错误。
|
||||
|
||||
`origin` 写进消息,用于在多 scope 部署里定位到底是哪个键/哪个参数非法。
|
||||
"""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
raise ValueError(f"{origin} 必须是 None 或有限正数秒: {value!r}")
|
||||
try:
|
||||
seconds = float(value)
|
||||
except OverflowError:
|
||||
raise ValueError(f"{origin} 必须是有限正数秒: {value!r}") from None
|
||||
if not math.isfinite(seconds) or seconds <= 0:
|
||||
raise ValueError(f"{origin} 必须是有限正数秒: {value!r}")
|
||||
return seconds
|
||||
|
||||
|
||||
async def with_call_deadline[T](aw: Awaitable[T], *, deadline_s: float | None, scope: str) -> T:
|
||||
"""给一个 awaitable 加一层可选期限;到期抛 `CallDeadlineExceeded`。
|
||||
|
||||
三条实现红线:
|
||||
|
||||
1. **校验先于构造 awaitable**——调用方必须先 `ensure_call_deadline`,否则非法值
|
||||
抛错时会遗留未 await 的协程(`RuntimeWarning` + 资源不释放)。
|
||||
2. 只用**相对时长**,绝不把注入的 `now` 换算成绝对截止时刻: 注入钟跳变
|
||||
10^6 秒不该凭空触发期限。
|
||||
3. 判据必须是**局部变量身份比较**,不可退化成只看 `cm.expired()`:
|
||||
到期后清理路径自抛的 `TimeoutError` 也发生在 `expired()` 为真时,只看它
|
||||
会把别人的超时改标成本层期限;`__cause__` 启发式同样失效(内层
|
||||
`asyncio.timeout` 抛出的 `TimeoutError` 其 `__cause__` 也是 `CancelledError`)。
|
||||
|
||||
不新增后台任务、不 `shield`、不改异常对象:外部取消照常以 `CancelledError` 穿透。
|
||||
"""
|
||||
if deadline_s is None:
|
||||
# 未启用: 不进上下文管理器,逐字走 1.3.5 旧路径
|
||||
return await aw
|
||||
# 体内(含清理路径)自抛的 TimeoutError 的**身份**,唯一可靠的区分依据
|
||||
inner_timeout: BaseException | None = None
|
||||
# 先建对象再进上下文: `as cm` 只在 `__aenter__` 返回后才绑定, 而 except 块无条件
|
||||
# 读 `cm`——进入阶段一旦抛 TimeoutError 就会变成 NameError 掩盖真实错误
|
||||
cm = asyncio.timeout(deadline_s)
|
||||
try:
|
||||
async with cm:
|
||||
try:
|
||||
return await aw
|
||||
except TimeoutError as exc:
|
||||
inner_timeout = exc
|
||||
raise
|
||||
except TimeoutError as exc:
|
||||
if cm.expired() and exc is not inner_timeout:
|
||||
raise CallDeadlineExceeded(scope=scope, deadline_s=deadline_s) from None
|
||||
raise
|
||||
+139
-11
@@ -16,6 +16,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import dataclasses
|
||||
import math
|
||||
import random
|
||||
import time
|
||||
@@ -27,6 +28,7 @@ from loguru import logger
|
||||
|
||||
from polygateway.client import _aclose_component, _telemetry_status_of
|
||||
from polygateway.config import EmbeddingSettings
|
||||
from polygateway.deadline import ensure_call_deadline, with_call_deadline
|
||||
from polygateway.errors import (
|
||||
AllSourcesExhausted,
|
||||
GovernanceBackendError,
|
||||
@@ -41,12 +43,13 @@ from polygateway.middleware.admission import SourceAdmission, settle_and_release
|
||||
from polygateway.middleware.breaker import BreakerGate
|
||||
from polygateway.middleware.ratelimit import QuotaGate
|
||||
from polygateway.middleware.retry import StallClock, _failure_reason, backoff_delay
|
||||
from polygateway.middleware.telemetry import TelemetryEmitter
|
||||
from polygateway.middleware.telemetry import TelemetryEmitter, emit_terminal_once
|
||||
from polygateway.types import (
|
||||
ChatRequest,
|
||||
EmbeddingResponse,
|
||||
LLMResponse,
|
||||
TelemetryStatus,
|
||||
_CallContext,
|
||||
strip_unsupported_extra_body,
|
||||
validate_caller_dimensions,
|
||||
)
|
||||
@@ -110,6 +113,7 @@ class EmbeddingClient:
|
||||
batch_size: int,
|
||||
normalize: bool = False,
|
||||
expected_dim: int | None = None,
|
||||
call_deadline_s: float | None = None,
|
||||
now: Callable[[], float] = time.monotonic,
|
||||
sleep: Callable[[float], Awaitable[None]] = asyncio.sleep,
|
||||
rng: Callable[[], float] = random.random,
|
||||
@@ -118,6 +122,10 @@ class EmbeddingClient:
|
||||
raise ValueError("batch_size 必须 ≥ 1")
|
||||
if expected_dim is not None and expected_dim < 1:
|
||||
raise ValueError("expected_dim 必须 ≥ 1")
|
||||
# 入口即校: 装配错误当场报,不等到第一次调用才炸
|
||||
self._call_deadline_s = ensure_call_deadline(
|
||||
call_deadline_s, "EmbeddingClient(call_deadline_s=...)"
|
||||
)
|
||||
self._scope = scope
|
||||
# embed payload 硬编码 {model, input},带 extra_body 的源必须先剥离,
|
||||
# 否则遥测会记录一个从未发出的采样参数(issue #4 决策 G)
|
||||
@@ -127,7 +135,9 @@ class EmbeddingClient:
|
||||
self._transport = transport
|
||||
self._retry = retry
|
||||
self._emitter = (
|
||||
TelemetryEmitter(telemetry, pricing=pricing, text_cap=text_cap) if telemetry else None
|
||||
TelemetryEmitter(telemetry, scope=self._scope, pricing=pricing, text_cap=text_cap)
|
||||
if telemetry
|
||||
else None
|
||||
)
|
||||
self._telemetry = telemetry
|
||||
# 限流/熔断后端在此之外只以 QuotaGate/BreakerGate 的形态存在,自持一份
|
||||
@@ -170,11 +180,16 @@ class EmbeddingClient:
|
||||
parent_call_id: str | None = None,
|
||||
tenant_id: str | None = None,
|
||||
meta: Mapping[str, Any] | None = None,
|
||||
call_deadline_s: float | None = None,
|
||||
) -> EmbeddingResponse:
|
||||
"""一次治理 embedding 调用: 按 batch_size 切批,批间串行,全批合并返回。
|
||||
|
||||
`tenant_id` 与 `meta` 是调用方自定义维度,只进遥测(issue #11);它们属于
|
||||
本次调用而非某一批,故每批的遥测行都带同一份维度。
|
||||
|
||||
`call_deadline_s` 是本次调用的墙钟硬边界(issue #22): `None` = 继承装配值。
|
||||
**整次调用共享一份**——N 批串行跑在同一条期限内,不按批数放大 N 倍。
|
||||
`texts == []` 的早返回在期限之外(零尝试,无等待可治)。
|
||||
"""
|
||||
if not isinstance(texts, list) or any(not isinstance(t, str) for t in texts):
|
||||
raise TypeError("texts 必须是 list[str](显式优于隐式,不收单条 str)")
|
||||
@@ -184,7 +199,17 @@ class EmbeddingClient:
|
||||
dimension_tenant_id, dimensions = validate_caller_dimensions(
|
||||
tenant_id, meta, origin="embed(tenant_id=..., meta=...)"
|
||||
)
|
||||
# 期限取值与校验必须在创建 awaitable **之前**(否则遗留未 await 的协程)
|
||||
deadline = (
|
||||
self._call_deadline_s
|
||||
if call_deadline_s is None
|
||||
else ensure_call_deadline(call_deadline_s, "embed(call_deadline_s=...)")
|
||||
)
|
||||
# 校验均已通过 → 进入统计边界(设计 §3.5: `texts` 类型与调用方维度校验之后)
|
||||
context = _CallContext(now=self._now)
|
||||
if not texts:
|
||||
# 合法零尝试: 返回真实统计(attempts=0),且**不写任何遥测行**
|
||||
# ——与 cache_hit 不同,不要按"遥测必录"推断它有台账行(设计 §3 M2)
|
||||
return EmbeddingResponse(
|
||||
vectors=[],
|
||||
dim=0,
|
||||
@@ -195,7 +220,44 @@ class EmbeddingClient:
|
||||
latency_ms=0,
|
||||
call_id=str(uuid.uuid4()),
|
||||
source_name="",
|
||||
call_stats=context.snapshot(),
|
||||
)
|
||||
try:
|
||||
return await with_call_deadline(
|
||||
self._embed_all(
|
||||
texts, session_id, parent_call_id, dimension_tenant_id, dimensions, context
|
||||
),
|
||||
deadline_s=deadline,
|
||||
scope=self._scope,
|
||||
)
|
||||
except PolyGatewayError as exc:
|
||||
await self._emit_terminal(
|
||||
texts, session_id, parent_call_id, dimension_tenant_id, dimensions, context, exc
|
||||
)
|
||||
raise
|
||||
except asyncio.CancelledError:
|
||||
# 三条链路同一口径尽力写一条(允许 0 条);取消优先,不 shield
|
||||
await self._emit_terminal(
|
||||
texts,
|
||||
session_id,
|
||||
parent_call_id,
|
||||
dimension_tenant_id,
|
||||
dimensions,
|
||||
context,
|
||||
"cancelled",
|
||||
)
|
||||
raise
|
||||
|
||||
async def _embed_all(
|
||||
self,
|
||||
texts: list[str],
|
||||
session_id: str | None,
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
) -> EmbeddingResponse:
|
||||
"""切批串行执行并合并;无源的 raise 必须在本方法内——否则无源终态行写不出。"""
|
||||
if not self._sources:
|
||||
raise AllSourcesExhausted(scope=self._scope, reason="no_sources", retry_after_s=0.0)
|
||||
outcomes = []
|
||||
@@ -205,11 +267,49 @@ class EmbeddingClient:
|
||||
texts[start : start + self._batch_size],
|
||||
session_id,
|
||||
parent_call_id,
|
||||
dimension_tenant_id,
|
||||
dimensions,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
)
|
||||
)
|
||||
return self._merge(outcomes)
|
||||
# 全批共享同一上下文,故分批是实现细节而非 N 次独立逻辑调用
|
||||
return dataclasses.replace(self._merge(outcomes), call_stats=context.snapshot())
|
||||
|
||||
async def _emit_terminal(
|
||||
self,
|
||||
texts: list[str],
|
||||
session_id: str | None,
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
error: PolyGatewayError | str,
|
||||
) -> None:
|
||||
"""终态行的请求摘要(设计 §6 M4): 计数占位 + 第一批截断文本。
|
||||
|
||||
描述的是**本次调用的整体输入**但不扩大单行正文预算: 失败批的具体文本由同
|
||||
`logical_call_id` 的 attempt 行给出,终态行不保存全量原输入。
|
||||
"""
|
||||
batches = math.ceil(len(texts) / self._batch_size)
|
||||
messages = [{"role": "user", "content": f"<embed texts={len(texts)} batches={batches}>"}]
|
||||
# 与逐批行同款构造(至多 `batch_size` 条、每条 200 字符)
|
||||
messages += [
|
||||
{"role": "user", "content": t[:_TELEMETRY_TEXT_CAP]} for t in texts[: self._batch_size]
|
||||
]
|
||||
await emit_terminal_once(
|
||||
self._emitter,
|
||||
request=ChatRequest(
|
||||
messages=messages,
|
||||
session_id=session_id,
|
||||
parent_call_id=parent_call_id,
|
||||
tenant_id=tenant_id,
|
||||
meta=meta,
|
||||
call_context=context,
|
||||
),
|
||||
context=context,
|
||||
error=error,
|
||||
operation="embed",
|
||||
)
|
||||
|
||||
# —— 治理循环(与 RetryMW 同构;设计 §7.1 已声明的有限重复)——
|
||||
|
||||
@@ -220,6 +320,7 @@ class EmbeddingClient:
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
) -> _BatchOutcome:
|
||||
fails = 0
|
||||
reasons: dict[str, str] = {}
|
||||
@@ -232,7 +333,7 @@ class EmbeddingClient:
|
||||
continue
|
||||
async with clock.attempting():
|
||||
outcome = await self._attempt(
|
||||
batch, *picked, reasons, session_id, parent_call_id, tenant_id, meta
|
||||
batch, *picked, reasons, session_id, parent_call_id, tenant_id, meta, context
|
||||
)
|
||||
if isinstance(outcome, _BatchOutcome):
|
||||
return outcome
|
||||
@@ -258,12 +359,22 @@ class EmbeddingClient:
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
) -> _BatchOutcome | _FailedBatch:
|
||||
call_id = str(uuid.uuid4())
|
||||
started = self._now()
|
||||
actual = 0
|
||||
# 局部阶段变量(与 RetryMW 同口径): 该刻库是否已算出确定结算。只服务于取消
|
||||
# 分支的兜底取值,不进任何签名; 未分类异常逃逸时仍逐字走旧的全额退还。
|
||||
settlement_known = False
|
||||
# 登记在 transport 调用**之前**(同 RetryMW): 失败与取消的尝试也真的发出去了
|
||||
context.register_attempt()
|
||||
try:
|
||||
# 裸生成时间(1.3.7 H8): 计时只包 transport 调用本身(同 RetryMW 口径),
|
||||
# 与本链路 total_latency_ms 同一只注入钟;分批累加在成功分支登记
|
||||
gen_started = self._now()
|
||||
result = await self._transport.embed(texts=batch, source=source, call_id=call_id)
|
||||
gen_ms = int((self._now() - gen_started) * 1000)
|
||||
if self._expected_dim is not None and result.dim != self._expected_dim:
|
||||
raise ResultInvalidError(
|
||||
f"{source.name} 维度 {result.dim} 不符期望 {self._expected_dim}",
|
||||
@@ -275,6 +386,9 @@ class EmbeddingClient:
|
||||
actual = source.effective_est_tokens()
|
||||
else:
|
||||
actual = result.prompt_tokens
|
||||
# 真实 usage 恰为 0 也是已知事实, 后续取消不得改写成 est
|
||||
settlement_known = True
|
||||
context.record_generation(gen_ms, accumulate=True)
|
||||
await self._record_quietly(self._breaker.record_success(entry))
|
||||
await self._record_quietly(self._quota.mark_progress())
|
||||
latency_ms = int((self._now() - started) * 1000)
|
||||
@@ -287,10 +401,12 @@ class EmbeddingClient:
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
result,
|
||||
)
|
||||
return _BatchOutcome(result, source, call_id, latency_ms)
|
||||
except (RequestRejectedError, ResultInvalidError) as exc:
|
||||
actual, settlement_known = 0, True # 逐字保住 1.3.5 口径(本版不改这一族记账)
|
||||
await self._gate_on_terminal(exc, entry)
|
||||
await self._emit(
|
||||
batch,
|
||||
@@ -301,10 +417,14 @@ class EmbeddingClient:
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
error=exc,
|
||||
)
|
||||
raise
|
||||
except asyncio.CancelledError:
|
||||
if not settlement_known:
|
||||
# 端口已开始、结算未定: 保守保留预扣(设计 §6.3 S7)
|
||||
actual = source.effective_est_tokens()
|
||||
if entry.is_probe:
|
||||
await self._record_quietly(self._breaker.release_probe(entry))
|
||||
await self._emit(
|
||||
@@ -316,6 +436,7 @@ class EmbeddingClient:
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
error="cancelled",
|
||||
)
|
||||
raise
|
||||
@@ -323,10 +444,11 @@ class EmbeddingClient:
|
||||
dead = isinstance(exc, SourceDeadError)
|
||||
reason = _failure_reason(exc)
|
||||
reasons[source.name] = reason
|
||||
# 结算决定定死在本分支第一个 await 之前: 同级 except CancelledError 接不住
|
||||
# 落在本块 await 上的取消,它直穿 finally。值与 1.3.5 逐字相同,只是算得更早。
|
||||
actual = 0 if dead else source.effective_est_tokens()
|
||||
settlement_known = True
|
||||
await self._record_quietly(self._breaker.record_failure(entry, reason, dead))
|
||||
if not dead:
|
||||
# 保守: 失败请求可能已被网关计费(CHS 同款);与入场预扣同源取值
|
||||
actual = source.effective_est_tokens()
|
||||
await self._emit(
|
||||
batch,
|
||||
source,
|
||||
@@ -336,6 +458,7 @@ class EmbeddingClient:
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
error=exc,
|
||||
)
|
||||
return _FailedBatch(exc, immediate=dead)
|
||||
@@ -372,8 +495,9 @@ class EmbeddingClient:
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
result: EmbeddingTransportResult | None = None,
|
||||
error: object | None = None,
|
||||
error: PolyGatewayError | str | None = None,
|
||||
) -> None:
|
||||
"""逐批遥测(经同一 Emitter): messages=截断 texts、向量绝不入库。"""
|
||||
if self._emitter is None:
|
||||
@@ -386,6 +510,7 @@ class EmbeddingClient:
|
||||
parent_call_id=parent_call_id,
|
||||
tenant_id=tenant_id,
|
||||
meta=meta,
|
||||
call_context=context,
|
||||
)
|
||||
response = None
|
||||
if result is not None:
|
||||
@@ -410,10 +535,12 @@ class EmbeddingClient:
|
||||
call_id=call_id,
|
||||
latency_ms=int((self._now() - started) * 1000),
|
||||
response=response,
|
||||
error=None if error is None else str(error),
|
||||
# 异常对象原样下传: 状态码/底层异常类型/网关正文在 Emitter 内定型
|
||||
error=error,
|
||||
# embedding payload 硬编码 {model, input},从不带推理参数;源上即便
|
||||
# 误配了 ENABLE_THINKING,记一个档也是替这次调用声称它没做过的事
|
||||
reasoning_applies=False,
|
||||
operation="embed",
|
||||
)
|
||||
|
||||
def _merge(self, outcomes: list[_BatchOutcome]) -> EmbeddingResponse:
|
||||
@@ -533,6 +660,7 @@ class EmbeddingClient:
|
||||
batch_size=settings.batch_size,
|
||||
normalize=settings.normalize,
|
||||
expected_dim=settings.expected_dim,
|
||||
call_deadline_s=gw.call_deadline_s,
|
||||
)
|
||||
_mark_owned_components(client, limiter=limiter, breaker=breaker, telemetry=telemetry)
|
||||
return client
|
||||
|
||||
@@ -217,3 +217,23 @@ class GovernanceBackendError(GatewayUnavailableError):
|
||||
# 父类会把 message 覆写为 "{scope} 网关暂时不可用: {reason}",而各构造点
|
||||
# 携带的诊断串(如"限流后端 try_acquire 失败: ...")是排障主线索,必须保住
|
||||
self.args = (message,)
|
||||
|
||||
|
||||
class CallDeadlineExceeded(PolyGatewayError): # noqa: N818 — 设计 §9 人类批准的公共名
|
||||
"""调用方设定的整体调用期限到期; 不是网关不可用、也不是源故障。
|
||||
|
||||
刻意**不属**四分类、**不进** `SCOPE_REASONS`、**不继承** `GatewayUnavailableError`:
|
||||
它描述的是调用方自己的耐心边界, 与"对方怎么了"正交——按四分类之一上报会让
|
||||
下游的重试/换源/熔断逻辑对着一次本地超时做治理决策(库铁律「错误分类驱动」)。
|
||||
|
||||
也刻意**没有** `retry_after_s`: 期限到期不含"何时可再试"的信息, 给 `0.0`
|
||||
会按既定语义指示下游立刻重打一条可能已经饱和的通道。
|
||||
|
||||
**到期不等于未产出、未计费**: 期限治理的是等待, 在途请求可能已经发出、
|
||||
已被上游计费, 清理仍在 `finally` 里完成, 故返回时刻 = 期限 + 清理耗时。
|
||||
"""
|
||||
|
||||
def __init__(self, *, scope: str, deadline_s: float) -> None:
|
||||
super().__init__(f"{scope} 调用期限 {deadline_s}s 到期")
|
||||
self.scope = scope
|
||||
self.deadline_s = deadline_s
|
||||
|
||||
@@ -169,15 +169,28 @@ class SourceAdmission:
|
||||
# —— 选源与准入(CHS _pick_runnable 120-167)——
|
||||
|
||||
async def pick(
|
||||
self, reasons: dict[str, str], attempt_fails: dict[str, int]
|
||||
self,
|
||||
reasons: dict[str, str],
|
||||
attempt_fails: dict[str, int],
|
||||
*,
|
||||
exclude: frozenset[str] | None = None,
|
||||
) -> tuple[tuple[SourceConfig, Permit, GateDecision] | None, int]:
|
||||
"""挑出第一个过闸的候选;返回 (选中三元组 | None, 熔断类拒绝计数)。"""
|
||||
"""挑出第一个过闸的候选;返回 (选中三元组 | None, 熔断类拒绝计数)。
|
||||
|
||||
`exclude` 是对冲编排的私有排除参数(issue #24): 以在途源名为排除集,
|
||||
保证对冲路落在**异源**。被排除不是源的拒绝——不计 gate_rejections、
|
||||
不写 reasons,否则会污染 `on_no_runnable` 的分派判据与 per_source_reasons
|
||||
对账。拿不到候选时返回 None,是否放弃由调用方决定(对冲方静默等原路,
|
||||
**严禁**对这个 None 调 `on_no_runnable`)。
|
||||
"""
|
||||
stats = {s.name: await self._quota.stats(s) for s in self._sources}
|
||||
gate_rejections = 0
|
||||
ordered = _demote_call_failures(
|
||||
self._selector.order(self._sources, stats), attempt_fails, self._health_view
|
||||
)
|
||||
for cand in ordered:
|
||||
if exclude and cand.name in exclude:
|
||||
continue # 对冲排除在途源: 不是拒绝, 不计数不写原因(见 docstring)
|
||||
if self._memo.active(cand.name):
|
||||
# 冷却备忘跳过也计入拒绝数,保住 circuit_open 判据(CHS 同款)
|
||||
gate_rejections += 1
|
||||
|
||||
@@ -208,6 +208,10 @@ class CacheMW:
|
||||
max_inter_token_ms=None,
|
||||
call_id=str(uuid.uuid4()),
|
||||
structured_data=structured_data,
|
||||
# 显式覆盖: 历史条目里的 `call_stats` 是个 dict,而 `_RESPONSE_FIELDS`
|
||||
# 过滤**会放行它**——不覆盖就会有 dict 冒充 `CallStats` 漏给调用方。
|
||||
# 本次调用的真实统计由公开边界在返回前追加(设计 §3)
|
||||
call_stats=None,
|
||||
)
|
||||
return LLMResponse(**fields)
|
||||
except Exception as exc:
|
||||
@@ -230,6 +234,9 @@ class CacheMW:
|
||||
def _serialize(self, response: LLMResponse) -> str:
|
||||
data = dataclasses.asdict(response)
|
||||
data.pop("structured_data", None) # pydantic 实例不可 JSON 往返(设计 §2.1)
|
||||
# 统计描述**本次**调用,存进去再放出来等于向下一个调用方谎称
|
||||
# 它重试了 N 次;`asdict` 会把 `CallStats` 摊成 dict,故必须显式剔除
|
||||
data.pop("call_stats", None)
|
||||
return json.dumps(data, ensure_ascii=False)
|
||||
|
||||
async def _safe_get(self, key: str) -> str | None:
|
||||
|
||||
@@ -42,6 +42,7 @@ from polygateway.types import LLMResponse
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import AsyncIterator, Awaitable, Callable
|
||||
|
||||
from polygateway.middleware.telemetry import TelemetryEmitter
|
||||
from polygateway.ports import (
|
||||
GateDecision,
|
||||
Permit,
|
||||
@@ -163,6 +164,17 @@ def _is_rate_limited(outcome: LLMResponse | _Failed) -> bool:
|
||||
return isinstance(outcome, _Failed) and _failure_reason(outcome.exc) == "rate_limited"
|
||||
|
||||
|
||||
_HEDGE_LOSER_ATTR = "_polygateway_hedge_loser"
|
||||
"""编排在 cancel() 之前给输家任务置位的标记;_attempt 读它选遥测标签。"""
|
||||
|
||||
|
||||
def _combine_failures(primary_f: _Failed, hedge_f: _Failed) -> _Failed:
|
||||
"""任一非 429 优先(计预算);两路皆 429 才按 429 免预算退还 stall 账;同类取原路。"""
|
||||
if _failure_reason(primary_f.exc) == "rate_limited" != _failure_reason(hedge_f.exc):
|
||||
return hedge_f
|
||||
return primary_f
|
||||
|
||||
|
||||
class RetryMW:
|
||||
"""尝试编排器;时钟/睡眠/随机全部注入,纯确定性可测(P6)。"""
|
||||
|
||||
@@ -181,7 +193,8 @@ class RetryMW:
|
||||
circuit_open: str = "fail_fast",
|
||||
cooldown_memo: SourceCooldownMemo | None = None,
|
||||
pacer: AdaptivePacer | None = None,
|
||||
emitter: object | None = None,
|
||||
emitter: TelemetryEmitter | None = None,
|
||||
hedge_after_s: float | None = None,
|
||||
now: Callable[[], float] = time.monotonic,
|
||||
sleep: Callable[[float], Awaitable[None]] = asyncio.sleep,
|
||||
rng: Callable[[], float] = random.random,
|
||||
@@ -199,6 +212,10 @@ class RetryMW:
|
||||
# M2.5 §3.35: AIMD 自适应并发——429 收紧、成功回涨,超限调用排队不烧预算
|
||||
self._pacer = pacer or AdaptivePacer(ceiling=64.0)
|
||||
self._emitter = emitter
|
||||
# 对冲触发阈值(issue #24): None = 关闭(__call__ 逐字走 1.3.6 单路路径)。
|
||||
# 值域/交叉守卫在装配层(config.check_hedge_assembly),本类不重复校验;
|
||||
# hedge_max_extra 不下传——v1 编排固定单路对冲(H5)
|
||||
self._hedge_after_s = hedge_after_s
|
||||
self._now = now
|
||||
self._sleep = sleep
|
||||
self._rng = rng
|
||||
@@ -245,12 +262,31 @@ class RetryMW:
|
||||
await self._admission.on_no_runnable(gate_rejections, reasons, clock)
|
||||
continue
|
||||
async with clock.attempting() as attempt:
|
||||
outcome = await self._attempt(request, *picked, reasons, attempt_fails)
|
||||
# 每轮一个 sink: 裸生成时间由编排裁定归属(T2 单路 = 成功那轮;
|
||||
# T3 对冲 = 赢家那一路),`_attempt` 只负责把本次 transport 耗时投进来
|
||||
generation_sink: list[int] = []
|
||||
if self._hedge_after_s is None:
|
||||
# 默认关闭: 逐字 1.3.6 单路路径(默认关闭回归门据此成立)
|
||||
outcome = await self._attempt(
|
||||
request,
|
||||
*picked,
|
||||
reasons,
|
||||
attempt_fails,
|
||||
first_token_event=None,
|
||||
generation_sink=generation_sink,
|
||||
)
|
||||
else:
|
||||
# 对冲轮 = 一次"超级尝试": 编排内部自建 sink 并按赢家归属登记
|
||||
outcome = await self._attempt_hedged(request, picked, reasons, attempt_fails)
|
||||
rate_limited = _is_rate_limited(outcome)
|
||||
if rate_limited:
|
||||
# 免了重试预算就得进 stall 账,否则这段耗时无人治理(见 StallClock)
|
||||
attempt.refund()
|
||||
if isinstance(outcome, LLMResponse):
|
||||
# 与 register_attempt 同款 None 守卫: 库内现场构造的请求跳过登记;
|
||||
# 对冲轮由 _attempt_hedged 按赢家归属登记,此处外层 sink 恒为空
|
||||
if request.call_context is not None and generation_sink:
|
||||
request.call_context.record_generation(generation_sink[0], accumulate=False)
|
||||
return outcome
|
||||
if not rate_limited:
|
||||
fails += 1
|
||||
@@ -274,11 +310,26 @@ class RetryMW:
|
||||
entry: GateDecision,
|
||||
reasons: dict[str, str],
|
||||
attempt_fails: dict[str, int],
|
||||
*,
|
||||
first_token_event: asyncio.Event | None,
|
||||
generation_sink: list[int],
|
||||
) -> LLMResponse | _Failed:
|
||||
call_id = str(uuid.uuid4())
|
||||
started = self._now()
|
||||
actual = 0
|
||||
# 局部阶段变量: 该刻库是否已算出**确定**结算。只服务于取消分支的兜底取值,
|
||||
# 不进任何签名/配置/遥测; 未分类异常逃逸时它无人读取, 故仍逐字走旧的全额退还。
|
||||
settlement_known = False
|
||||
# 登记在 transport 调用**之前**(1.3.5 设计 §4): 失败与取消的尝试同样
|
||||
# "真的打出去了",挪到成功之后会让诊断最需要看见的那几次从计数里消失。
|
||||
# 上下文为 None = 库内现场构造的请求,跳过而不是报错
|
||||
if request.call_context is not None:
|
||||
request.call_context.register_attempt()
|
||||
try:
|
||||
# 裸生成时间(1.3.7 H8): 计时只包 transport 调用本身,起点紧贴调用前、
|
||||
# 终点为返回后首句(中间无 await);取消落进来时 transport 未返回,不计。
|
||||
# 与该链路 total_latency_ms 同一只注入钟,差值(波动开销)才有意义
|
||||
gen_started = self._now()
|
||||
result = await self._transport.complete(
|
||||
messages=request.messages,
|
||||
source=source,
|
||||
@@ -288,13 +339,19 @@ class RetryMW:
|
||||
# 逐次尝试原样重传: 换源不改变调用方要的档位(源级默认由 transport
|
||||
# 自己按选中的源解析,两者在 effective_effort 里汇合)
|
||||
reasoning_effort=request.reasoning_effort,
|
||||
# 对冲编排(计划 §3.4): 原路携带首 token 事件,对冲路恒 None(v1 单路,
|
||||
# 不再梯次);未启用对冲时 __call__ 传 None = 调用方不观测首 token
|
||||
first_token_event=first_token_event,
|
||||
)
|
||||
generation_sink.append(int((self._now() - gen_started) * 1000))
|
||||
if result.usage_source == "unavailable":
|
||||
# 用量不可得时按入场预扣量结算(delta==0),否则押金会被整笔退回,
|
||||
# 对"从不返回 usage 帧"的源等于 TPM 闸失效(设计 §3.2 #9)
|
||||
actual = source.effective_est_tokens()
|
||||
else:
|
||||
actual = result.prompt_tokens + result.completion_tokens
|
||||
# 真实 usage 恰为 0 也是**已知事实**, 后续取消不得把它改写成 est
|
||||
settlement_known = True
|
||||
await self._record_quietly(self._breaker.record_success(entry))
|
||||
await self._record_quietly(self._quota.mark_progress())
|
||||
self._feed_outcome(source.name, ok=True)
|
||||
@@ -303,18 +360,31 @@ class RetryMW:
|
||||
await self._emit(request, source, call_id, started, response=response)
|
||||
return response
|
||||
except RequestRejectedError as exc:
|
||||
actual, settlement_known = 0, True # 逐字保住 1.3.5: 坏请求全额退还
|
||||
await self._on_rejected(exc, source, entry)
|
||||
await self._emit(request, source, call_id, started, error=exc)
|
||||
raise
|
||||
except ResultInvalidError as exc:
|
||||
actual, settlement_known = 0, True # 同上, 本版不改这一族记账口径
|
||||
# 坏结果 ≠ 坏服务: 熔断记成功但不计窗口样本,亦不喂健康分(M2.5 §3.1)
|
||||
await self._record_quietly(self._breaker.record_success(entry, count_attempt=False))
|
||||
await self._emit(request, source, call_id, started, error=exc)
|
||||
raise
|
||||
except asyncio.CancelledError:
|
||||
if not settlement_known:
|
||||
# 端口已开始、结算未定: 保守保留预扣(宁多扣不凭空退款, 见设计 §6.3 S3)
|
||||
actual = source.effective_est_tokens()
|
||||
if entry.is_probe:
|
||||
await self._record_quietly(self._breaker.release_probe(entry))
|
||||
await self._emit(request, source, call_id, started, error="cancelled")
|
||||
# 对冲输家在 cancel() 前被编排置位标记(计划 §3.4): 据此区分"被对冲
|
||||
# 淘汰"与"外部取消",零新遥测列(设计 §4.5);外部取消与对冲取消同时
|
||||
# 到达的竞速可能误贴——记账方向一致(est 保留),属已批准的可接受残留
|
||||
label = (
|
||||
"hedge_cancelled"
|
||||
if getattr(asyncio.current_task(), _HEDGE_LOSER_ATTR, False)
|
||||
else "cancelled"
|
||||
)
|
||||
await self._emit(request, source, call_id, started, error=label)
|
||||
raise
|
||||
except (SourceDeadError, TransientError) as exc:
|
||||
dead = isinstance(exc, SourceDeadError)
|
||||
@@ -324,16 +394,205 @@ class RetryMW:
|
||||
self._feed_outcome(source.name, ok=False)
|
||||
if reason == "rate_limited":
|
||||
self._pacer.on_backpressure(source.name)
|
||||
# 结算决定必须在本分支**第一个 await 之前**定死: 同级的 except CancelledError
|
||||
# 接不住落在本块 await 上的取消,它会直穿 finally——那一刻 actual 是什么就结什么。
|
||||
# 值与 1.3.5 逐字相同(dead 全额退、瞬时保留预扣),只是算得更早。
|
||||
actual = 0 if dead else source.effective_est_tokens()
|
||||
settlement_known = True
|
||||
await self._record_quietly(self._breaker.record_failure(entry, reason, dead))
|
||||
if not dead:
|
||||
# 保守: 失败请求可能已被网关计费(CHS 同款);与入场预扣同源取值
|
||||
actual = source.effective_est_tokens()
|
||||
await self._emit(request, source, call_id, started, error=exc)
|
||||
return _Failed(exc, immediate=dead)
|
||||
finally:
|
||||
self._pacer.leave(source.name)
|
||||
await settle_and_release(permit, actual)
|
||||
|
||||
# —— 对冲编排(issue #24 设计 §4.6;默认关闭,`hedge_after_s is None` 不进这里)——
|
||||
|
||||
async def _attempt_hedged(
|
||||
self,
|
||||
request: ChatRequest,
|
||||
picked: tuple[SourceConfig, Permit, GateDecision],
|
||||
reasons: dict[str, str],
|
||||
attempt_fails: dict[str, int],
|
||||
) -> LLMResponse | _Failed:
|
||||
"""一次"超级尝试": 原路 + (触发后)异源对冲路,FIRST_COMPLETED 竞速。
|
||||
|
||||
计时只用事件循环相对时长(`asyncio.wait` timeout),绝不读注入 `now`
|
||||
(设计 §4.1 时钟纪律);两路各持各的 permit,结算/遥测/熔断写回全部沿用
|
||||
`_attempt` 既有路径,本方法只做编排与赢家裁定。
|
||||
"""
|
||||
# Phase 1 启动原路: 事件与 sink 每轮新建(局部状态,严禁实例属性)
|
||||
source, permit, entry = picked
|
||||
first_token: asyncio.Event = asyncio.Event()
|
||||
sink_p: list[int] = []
|
||||
primary = asyncio.create_task(
|
||||
self._attempt(
|
||||
request,
|
||||
source,
|
||||
permit,
|
||||
entry,
|
||||
reasons,
|
||||
attempt_fails,
|
||||
first_token_event=first_token,
|
||||
generation_sink=sink_p,
|
||||
)
|
||||
)
|
||||
hedge: asyncio.Task[LLMResponse | _Failed] | None = None
|
||||
try:
|
||||
# Phase 2 触发窗: 只认"阈值到 + 首 token 未至 + 原路在途"(loop 相对时长)
|
||||
if not await self._past_hedge_window(primary, first_token):
|
||||
# 原路已了结/首 token 已至: 等价于未配置对冲
|
||||
outcome = await primary
|
||||
if isinstance(outcome, LLMResponse):
|
||||
self._record_generation(request, sink_p)
|
||||
return outcome
|
||||
# Phase 3 异源准入(完整 pick 路径,无旁路): 拿不到候选 = 静默等原路
|
||||
# (对冲是优化不是权利;严禁对这个 None 调 on_no_runnable,见 §3.5)
|
||||
hedge_picked, _ = await self._admission.pick(
|
||||
reasons, attempt_fails, exclude=frozenset({source.name})
|
||||
)
|
||||
if hedge_picked is None:
|
||||
outcome = await primary
|
||||
if isinstance(outcome, LLMResponse):
|
||||
self._record_generation(request, sink_p)
|
||||
return outcome
|
||||
# Phase 3.5 准入后复查: 原路可能在对冲准入的 await 期间已了结——此时
|
||||
# 一个对冲请求都不发(那是白付一次真实计费请求 + 一份 est 滞留 + 一条
|
||||
# hedge_cancelled 行)。按既有语义释放刚拿到的对冲准入(probe 须
|
||||
# release_probe,顺序同 _attempt 取消分支),直接裁定原路结果
|
||||
if primary.done():
|
||||
hedge_source, hedge_permit, hedge_entry = hedge_picked
|
||||
try:
|
||||
if hedge_entry.is_probe:
|
||||
await self._record_quietly(self._breaker.release_probe(hedge_entry))
|
||||
finally:
|
||||
self._pacer.leave(hedge_source.name)
|
||||
await settle_and_release(hedge_permit, 0)
|
||||
outcome = await primary
|
||||
if isinstance(outcome, LLMResponse):
|
||||
self._record_generation(request, sink_p)
|
||||
return outcome
|
||||
# Phase 4 启动对冲路(v1 单路,H5): 对冲路恒传 first_token_event=None,
|
||||
# 不再触发梯次对冲
|
||||
sink_h: list[int] = []
|
||||
hedge = asyncio.create_task(
|
||||
self._attempt(
|
||||
request,
|
||||
*hedge_picked,
|
||||
reasons,
|
||||
attempt_fails,
|
||||
first_token_event=None,
|
||||
generation_sink=sink_h,
|
||||
)
|
||||
)
|
||||
# Phase 5 赢家裁定与收口
|
||||
done, pending = await asyncio.wait(
|
||||
{primary, hedge}, return_when=asyncio.FIRST_COMPLETED
|
||||
)
|
||||
if pending and not any(self._succeeded(t, done) for t in done):
|
||||
# 先了结的是失败: 等另一路的结论再裁定(它可能后发先至;
|
||||
# 原路先败、对冲在途时不重试,设计 §4.6)
|
||||
more, pending = await asyncio.wait(pending)
|
||||
done |= more
|
||||
winner = (
|
||||
primary
|
||||
if self._succeeded(primary, done)
|
||||
else hedge
|
||||
if self._succeeded(hedge, done)
|
||||
else None
|
||||
)
|
||||
if winner is None:
|
||||
primary_exc = primary.exception()
|
||||
hedge_exc = hedge.exception() # 两路都取回,不留 never-retrieved 告警
|
||||
# 非可重试族(RequestRejected/ResultInvalid)穿透,与单路同口径
|
||||
if primary_exc is not None:
|
||||
raise primary_exc
|
||||
if hedge_exc is not None:
|
||||
raise hedge_exc
|
||||
# 两败(H6): 汇合为一次失败,重试预算只计一次;路数照登(无赢家)
|
||||
outcome = _combine_failures(primary.result(), hedge.result())
|
||||
self._register_hedge(request, hedge_won=False, winner_sink=None)
|
||||
return outcome
|
||||
# 两路同时成功的竞速 → 原路优先(保守不弃原路成果),由上面 primary
|
||||
# 先判实现;竞速落选者已完成,不置标记——它没被取消,行是正常行
|
||||
loser = hedge if winner is primary else primary
|
||||
if not loser.done():
|
||||
# 先置标记再 cancel: _attempt 的取消分支据标记选 hedge_cancelled
|
||||
setattr(loser, _HEDGE_LOSER_ATTR, True)
|
||||
loser.cancel()
|
||||
# 收口: gather 等输家 finally 的结算/遥测跑完才返回(快照含输家,
|
||||
# attempts==2);对已完成的输家顺带取回结果,不留 never-retrieved 告警
|
||||
await asyncio.gather(loser, return_exceptions=True)
|
||||
self._register_hedge(
|
||||
request,
|
||||
hedge_won=winner is hedge,
|
||||
winner_sink=sink_h if winner is hedge else sink_p,
|
||||
)
|
||||
return winner.result()
|
||||
except BaseException:
|
||||
# 取消穿透与准入冒泡(GovernanceBackendError)同路: 两任务(存在且未完者)
|
||||
# 同消,尽力收口后原异常上抛;不 shield,收口 await 允许被再取消(136 清理纪律)
|
||||
tasks = [t for t in (primary, hedge) if t is not None and not t.done()]
|
||||
for t in tasks:
|
||||
t.cancel()
|
||||
if tasks:
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
raise
|
||||
|
||||
async def _past_hedge_window(self, primary: asyncio.Task, first_token: asyncio.Event) -> bool:
|
||||
"""对冲触发窗: 阈值到 + 首 token 未至 + 原路在途,三者齐备才放行对冲。
|
||||
|
||||
只用事件循环相对时长(`asyncio.wait` timeout),绝不读注入 `now`——测试
|
||||
伪造注入钟跳变不得触发对冲(设计 §4.1)。waiter 任务必须收口,且**不得吞
|
||||
外部取消**(finally 里 await 已取消的 waiter 会接住一个 CancelledError,
|
||||
须靠 cancelling() 区分它是 waiter 自己的还是外面打进来的)。
|
||||
"""
|
||||
waiter = asyncio.create_task(first_token.wait())
|
||||
try:
|
||||
await asyncio.wait(
|
||||
{primary, waiter}, timeout=self._hedge_after_s, return_when=asyncio.FIRST_COMPLETED
|
||||
)
|
||||
finally:
|
||||
waiter.cancel()
|
||||
try:
|
||||
await waiter
|
||||
except asyncio.CancelledError:
|
||||
if asyncio.current_task().cancelling(): # 外部取消,穿透
|
||||
raise
|
||||
return not primary.done() and not first_token.is_set()
|
||||
|
||||
@staticmethod
|
||||
def _succeeded(task: asyncio.Task, done: set[asyncio.Task]) -> bool:
|
||||
"""赢家裁定: 已完成、未被取消、无异常且结果为 LLMResponse。"""
|
||||
return (
|
||||
task in done
|
||||
and not task.cancelled()
|
||||
and task.exception() is None
|
||||
and isinstance(task.result(), LLMResponse)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _record_generation(request: ChatRequest, sink: list[int]) -> None:
|
||||
"""record_generation 的共享守卫: 上下文缺失(库内现场构造)或空 sink 均跳过。"""
|
||||
if request.call_context is not None and sink:
|
||||
request.call_context.record_generation(sink[0], accumulate=False)
|
||||
|
||||
@staticmethod
|
||||
def _register_hedge(
|
||||
request: ChatRequest, *, hedge_won: bool, winner_sink: list[int] | None
|
||||
) -> None:
|
||||
"""对冲登记(赢家裁定后一次性;同 register_attempt 的 None 守卫)。
|
||||
|
||||
hedges 计"实际并发发出的对冲路数"——触发但准入失败的静默不计(没走到
|
||||
这里);两败轮次无赢家: 路数照登(hedge_won=False),裸生成时间无归属不记。
|
||||
"""
|
||||
context = request.call_context
|
||||
if context is None:
|
||||
return
|
||||
if winner_sink:
|
||||
context.record_generation(winner_sink[0], accumulate=False)
|
||||
context.register_hedge(hedge_won=hedge_won)
|
||||
|
||||
async def _on_rejected(
|
||||
self, exc: RequestRejectedError, source: SourceConfig, entry: GateDecision
|
||||
) -> None:
|
||||
@@ -408,9 +667,13 @@ class RetryMW:
|
||||
started: float,
|
||||
*,
|
||||
response: LLMResponse | None = None,
|
||||
error: object | None = None,
|
||||
error: PolyGatewayError | str | None = None,
|
||||
) -> None:
|
||||
"""逐次遥测(经注入的单一 Emitter);遥测失败不得影响调用(铁律)。"""
|
||||
"""逐次遥测(经注入的单一 Emitter);遥测失败不得影响调用(铁律)。
|
||||
|
||||
异常**对象原样下传**而非先 `str()` 压平(1.3.5 设计 §5): 状态码、底层异常
|
||||
类型与网关响应体已经在异常上了,在这里压平就是把它们丢掉。
|
||||
"""
|
||||
if self._emitter is None:
|
||||
return
|
||||
try:
|
||||
@@ -420,9 +683,11 @@ class RetryMW:
|
||||
call_id=call_id,
|
||||
latency_ms=int((self._now() - started) * 1000),
|
||||
response=response,
|
||||
error=None if error is None else str(error),
|
||||
error=error,
|
||||
# chat 路径是唯一带推理参数的路径,故实发档由这里的响应说了算
|
||||
reasoning_applies=True,
|
||||
# 公开方法四值之一;本中间件只服务 chat 洋葱
|
||||
operation="chat",
|
||||
)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
|
||||
@@ -14,6 +14,8 @@ from typing import TYPE_CHECKING
|
||||
from polygateway.errors import ResultInvalidError
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
|
||||
from polygateway.ports import CallNext, StructuredOutputStrategy
|
||||
from polygateway.types import ChatRequest, LLMResponse
|
||||
|
||||
@@ -22,12 +24,18 @@ _FEEDBACK_TEMPLATE = (
|
||||
"Your previous reply was not valid JSON matching the required schema. "
|
||||
"Errors: {errors}. Reply with ONLY the corrected JSON object."
|
||||
)
|
||||
_MAX_FEEDBACK_ERRORS = 3
|
||||
_MAX_ERROR_CHARS = 200
|
||||
MAX_FEEDBACK_ERRORS = 3
|
||||
MAX_ERROR_CHARS = 200
|
||||
|
||||
|
||||
def _format_errors(errors: list[str]) -> str:
|
||||
clipped = [e[:_MAX_ERROR_CHARS] for e in errors[:_MAX_FEEDBACK_ERRORS]]
|
||||
def format_bounded_errors(errors: Sequence[str]) -> str:
|
||||
"""校验错误的有界拼装: 至多 3 条 × 每条 200 字符。
|
||||
|
||||
**本模块是这条规则的所有者**: 重问反馈文案与 1.3.5 终态行的结构化说明
|
||||
两个消费者共用同一份实现与同一组数值——数值复制成两份必然漂移,而漂移后
|
||||
"模型看到的错误"与"台账里记的错误"就不再是同一件事。行为与重命名前逐字相同。
|
||||
"""
|
||||
clipped = [e[:MAX_ERROR_CHARS] for e in list(errors)[:MAX_FEEDBACK_ERRORS]]
|
||||
return "; ".join(clipped) if clipped else "output could not be parsed"
|
||||
|
||||
|
||||
@@ -104,7 +112,10 @@ class StructuredMW:
|
||||
messages = [
|
||||
*current.messages,
|
||||
{"role": "assistant", "content": bad_content},
|
||||
{"role": "user", "content": _FEEDBACK_TEMPLATE.format(errors=_format_errors(errors))},
|
||||
{
|
||||
"role": "user",
|
||||
"content": _FEEDBACK_TEMPLATE.format(errors=format_bounded_errors(errors)),
|
||||
},
|
||||
]
|
||||
reask = dataclasses.replace(current, messages=messages)
|
||||
if self._escalation is not None:
|
||||
|
||||
@@ -2,13 +2,19 @@
|
||||
|
||||
Emitter 是全库**唯一**调用 `record_llm_call` 的地方(三项目 4 处逐字复制
|
||||
15 参调用的教训)。分工: RetryMW 经 Emitter 逐次记录每次尝试;TelemetryMW
|
||||
(最外层)只记尝试层看不见的事件——缓存命中、scope 级失败、取消;
|
||||
RequestRejected/ResultInvalid 已被尝试层记录,最外层放行不重复记。
|
||||
(最外层)只记尝试层看不见的缓存命中;而**终态失败行**由三个 client 的公开
|
||||
边界经 `emit_terminal_once` 统一写出(1.3.5)——两处同时写就会双计。
|
||||
|
||||
一行遥测属于三类事件之一(`event_kind`): `attempt`(一次尝试)、`cache_hit`
|
||||
(未产生网关调用)、`terminal_failure`(一次**逻辑调用**的失败终态)。后两者与
|
||||
前者**不是重复事实**,故统计失败调用次数只能取 `terminal_failure`,
|
||||
不得按 `error IS NOT NULL` 跨两类直接计数(设计 §6/§8)。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import inspect
|
||||
import json
|
||||
import time
|
||||
import uuid
|
||||
@@ -17,12 +23,10 @@ from typing import TYPE_CHECKING
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from polygateway.errors import (
|
||||
GatewayUnavailableError,
|
||||
GovernanceBackendError,
|
||||
SourceNotConfiguredError,
|
||||
)
|
||||
from polygateway.errors import PolyGatewayError, ResultInvalidError
|
||||
from polygateway.middleware.cache import digest_messages
|
||||
from polygateway.middleware.structured import MAX_ERROR_CHARS, format_bounded_errors
|
||||
from polygateway.ports import TelemetryRecorder
|
||||
from polygateway.thinking import effective_effort
|
||||
from polygateway.types import Effort, ThinkingObservation, canonical_sampling_json, merge_sampling
|
||||
|
||||
@@ -30,9 +34,17 @@ if TYPE_CHECKING:
|
||||
from collections.abc import Callable, Mapping
|
||||
from typing import Any
|
||||
|
||||
from polygateway.ports import CallNext, TelemetryRecorder
|
||||
from polygateway.ports import CallNext
|
||||
from polygateway.pricing import PricingTable
|
||||
from polygateway.types import ChatRequest, LLMResponse, SourceConfig
|
||||
from polygateway.types import (
|
||||
CallOperation,
|
||||
CallStats,
|
||||
ChatRequest,
|
||||
EventKind,
|
||||
LLMResponse,
|
||||
SourceConfig,
|
||||
_CallContext,
|
||||
)
|
||||
|
||||
|
||||
def _canonical_meta_json(meta: Mapping[str, Any]) -> str:
|
||||
@@ -225,26 +237,159 @@ class _AttemptUsage:
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _ErrorFields:
|
||||
"""一行遥测的错误列;未知一律 `None`。
|
||||
|
||||
存在的理由是把"三种入参形态 × 两类行"的定型规则收敛到**一处**:
|
||||
改前调用方先 `str(exc)` 压平,状态码、底层异常类型与网关正文全部丢失。
|
||||
"""
|
||||
|
||||
error: str | None = None
|
||||
error_type: str | None = None
|
||||
cause_type: str | None = None
|
||||
http_status_code: int | None = None
|
||||
error_body: str | None = None
|
||||
|
||||
|
||||
def _structured_detail(exc: ResultInvalidError) -> str:
|
||||
"""结构化阶梯耗尽的**有界**说明(设计 §5 C2)。
|
||||
|
||||
`ResultInvalidError("结构化输出阶梯耗尽")` 的 message 不含校验与修复错误,而该
|
||||
失败发生在 StructuredMW 之上——RetryMW 侧的 attempt 行全是**成功行**,终态行是
|
||||
唯一记录。故把说明并入现有 `error` 串。
|
||||
|
||||
**不含 `raw_text`**: 它是模型正文,attempt 行的 `response` 列已按 `text_cap` 记过
|
||||
一份;再存一份等于绕过既有的正文预算。条数与限长复用 `structured.py` 的同一
|
||||
套常量(重问反馈与本说明同一口径),数值只有一份。
|
||||
"""
|
||||
parts: list[str] = []
|
||||
if exc.repair_error:
|
||||
parts.append(f"repair={exc.repair_error[:MAX_ERROR_CHARS]}")
|
||||
if exc.validation_errors:
|
||||
parts.append(f"validation={format_bounded_errors(exc.validation_errors)}")
|
||||
return " | ".join(parts)
|
||||
|
||||
|
||||
def _error_fields(
|
||||
error: PolyGatewayError | str | None,
|
||||
*,
|
||||
event_kind: EventKind,
|
||||
class_prefixed: bool,
|
||||
) -> _ErrorFields:
|
||||
"""三种入参形态的唯一定型点(设计 §5/§6)。
|
||||
|
||||
- `None` → 全 None(成功行不统一填 200: 那会让"有状态码"不再等价于"失败了")。
|
||||
- `str`(取消路径的 `"cancelled"`)→ 原样落 `error`,**不解析字符串猜诊断**。
|
||||
- 领域异常 → 只读它既有的属性,不遍历任意对象、不猜正文。
|
||||
|
||||
**终态行的三列恒为 NULL(C1 红线)**: `GatewayUnavailableError` 家族从不携带
|
||||
状态码与响应体,NULL 正是它自身的真实状态——把最后一次 attempt 的状态码与正文
|
||||
搬上来,就是拿最后一个源冒充整池归因。逐源现场由同一 `logical_call_id` 的
|
||||
attempt 行给出。
|
||||
"""
|
||||
if error is None:
|
||||
return _ErrorFields()
|
||||
if isinstance(error, str):
|
||||
return _ErrorFields(error=error)
|
||||
name = type(error).__name__
|
||||
# 空 `str()` 退回类名(httpx 的 Connect/Read/Write/PoolTimeout 文案就是空的);
|
||||
# `class_prefixed` 是 OCR 的既有口径(按类名归组的 metric),故逐字保留它的拼法
|
||||
text = f"{name}: {error}" if class_prefixed else (str(error) or name)
|
||||
if event_kind == "terminal_failure":
|
||||
if isinstance(error, ResultInvalidError):
|
||||
detail = _structured_detail(error)
|
||||
if detail:
|
||||
text = f"{text} | {detail}"
|
||||
return _ErrorFields(error=text, error_type=name)
|
||||
cause = error.__cause__
|
||||
return _ErrorFields(
|
||||
error=text,
|
||||
error_type=name,
|
||||
cause_type=type(cause).__name__ if cause is not None else None,
|
||||
# getattr 而非直读: 本函数在 `_record` 的降级 try **之外**求值,
|
||||
# 一个非领域异常误传进来不得把一次真实失败换成 AttributeError
|
||||
http_status_code=getattr(error, "status_code", None),
|
||||
# 空串归 None: 既有 `body_text` 的缺省就是空串,而本列的语义是"未知"
|
||||
error_body=getattr(error, "body_text", "") or None,
|
||||
)
|
||||
|
||||
|
||||
def _assert_recorder_shape(recorder: TelemetryRecorder) -> None:
|
||||
"""装配期一次 `signature.bind` 形状校验: 不执行写入,只证明该形状能被接受。
|
||||
|
||||
`_record` 的 `except Exception` 会把旧 recorder 的 `TypeError` 吞成 warning,
|
||||
后果是自定义 recorder 在下游升级后**100% 丢遥测且调用照常成功**——正是
|
||||
"遥测必录"要防的形态,而文档级迁移清单挡不住它。故在装配期当场报错
|
||||
(不是 warning: 降级方向的铁律管的是**运行期写失败**,不是装配错误)。
|
||||
|
||||
参数名从 `TelemetryRecorder.record_llm_call` 的协议签名**派生**(不手抄第四份
|
||||
字段清单),绑定用哨兵 `None`,不读任何真实请求数据;`**kwargs`
|
||||
(VAR_KEYWORD)自动通过。不可 inspect(C 实现等)同样按配置错误报错——宁可
|
||||
装配不起来,不进入"运行期静默丢行"。
|
||||
|
||||
边界诚实声明: 它只证明该形状能被接受,**不能证明函数体真的落这些列**。
|
||||
|
||||
Raises:
|
||||
ValueError: 签名不符、不可 inspect,或协议本身不可 inspect。
|
||||
"""
|
||||
try:
|
||||
# 模块全局查找而非常量快照: 协议改了,闸就跟着改(测试可据此机械验证)
|
||||
protocol = inspect.signature(TelemetryRecorder.record_llm_call).parameters
|
||||
except (TypeError, ValueError) as exc: # pragma: no cover - 协议一向可 inspect
|
||||
raise ValueError(f"TelemetryRecorder.record_llm_call 签名不可读取: {exc}") from exc
|
||||
sentinels = {name: None for name in protocol if name != "self"}
|
||||
label = type(recorder).__name__
|
||||
method = getattr(recorder, "record_llm_call", None)
|
||||
if method is None:
|
||||
# 连方法都没有: 比旧签名更明确的配置错误。不让它以裸 AttributeError
|
||||
# 逆流而上——那不属错误四分类,且现场离"注错了东西"这个真因很远
|
||||
raise ValueError(
|
||||
f"注入的遥测 recorder {label} 没有 record_llm_call 方法,不满足 TelemetryRecorder 端口"
|
||||
)
|
||||
try:
|
||||
signature = inspect.signature(method)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(
|
||||
f"遥测 recorder {label} 的 record_llm_call 不可 inspect(如 C 实现),"
|
||||
"无法在装配期确认它接受当前字段形状;请换成 Python 实现或包一层"
|
||||
) from exc
|
||||
try:
|
||||
signature.bind(**sentinels)
|
||||
except TypeError as exc:
|
||||
raise ValueError(
|
||||
f"遥测 recorder {label} 的 record_llm_call 签名与 TelemetryRecorder 不符"
|
||||
f"(当前 {len(sentinels)} 个字段): {exc}。"
|
||||
"这一条故意在装配期报错——放行的后果是每行遥测都被降级成 warning 后丢弃"
|
||||
) from exc
|
||||
|
||||
|
||||
class TelemetryEmitter:
|
||||
"""从请求与结果组装 26 字段并写入 recorder;一切写失败降级 warning。"""
|
||||
"""从请求与结果组装 36 字段并写入 recorder;一切写失败降级 warning。"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
recorder: TelemetryRecorder,
|
||||
*,
|
||||
scope: str,
|
||||
pricing: PricingTable | None = None,
|
||||
text_cap: int | None,
|
||||
) -> None:
|
||||
"""`text_cap` 无默认值是有意的: 它是关键行为参数,漏传即静默改变落库正文。
|
||||
"""`text_cap` 与 `scope` 无默认值是有意的: 两者都是关键行为参数。
|
||||
|
||||
`text_cap` 漏传即静默改变落库正文;`scope` 漏传则三类行都失去池名
|
||||
——终态失败可能根本没选出源,但 scope 始终已知,不拿 `source_name` 顶替。
|
||||
本类是库内部类,唯一构造者是三个公共 Client,必填能保证没有一处漏传。
|
||||
同理,值域校验也放在这一处: 三个 Client 的 `text_cap` 全部汇流到这里,
|
||||
|
||||
同理,值域校验与**装配闸**都放在这一处: 三个 Client 全部汇流到这里,
|
||||
`GatewaySettings` 那道只管 env 一条路,而直接构造 Client 是库承诺的另一
|
||||
条公共装配路——`text_cap=0` 会让每条正文只剩一个省略标记(P5 不得静默)。
|
||||
条公共装配路(`text_cap=0` 会让每条正文只剩一个省略标记;P5 不得静默)。
|
||||
"""
|
||||
if text_cap is not None and text_cap <= 0:
|
||||
raise ValueError(f"text_cap 必须 > 0(不截断请传 None): {text_cap}")
|
||||
_assert_recorder_shape(recorder)
|
||||
self._recorder = recorder
|
||||
self._scope = scope
|
||||
self._pricing = pricing
|
||||
self._text_cap = text_cap
|
||||
|
||||
@@ -256,8 +401,10 @@ class TelemetryEmitter:
|
||||
call_id: str,
|
||||
latency_ms: int,
|
||||
response: LLMResponse | None,
|
||||
error: str | None,
|
||||
error: PolyGatewayError | str | None,
|
||||
reasoning_applies: bool,
|
||||
operation: CallOperation,
|
||||
class_prefixed_error: bool = False,
|
||||
) -> None:
|
||||
"""逐次尝试记录(三个 Client 的重试层调用);失败尝试无用量可言,记 0 并标 unavailable。
|
||||
|
||||
@@ -266,7 +413,11 @@ class TelemetryEmitter:
|
||||
共用同一个 `SourceConfig` 类型,一个误配了 `ENABLE_THINKING` 的 embedding 源
|
||||
会让下面的回落算出 `auto`,给一次从来不带推理参数的调用挂上一个从未发出过的
|
||||
档。**不设默认值**: 与 `TelemetryRecorder` 同一约定,库外无第三方调用者,漏传
|
||||
当场 TypeError,好过被静默当成"没表态"。
|
||||
当场 TypeError,好过被静默当成"没表态"。`operation` 同理且另有一层:
|
||||
它只能由调用点给定,**绝不读 `exc.operation`**(后者是 HTTP 子操作)。
|
||||
|
||||
`error` 收**领域异常对象**而非预先 `str()` 压平的文本: 状态码/底层异常类型/
|
||||
网关正文在此提取成四列(设计 §5)。取消路径仍传既有字符串 `"cancelled"`。
|
||||
"""
|
||||
usage = _AttemptUsage.of(response)
|
||||
await self._record(
|
||||
@@ -284,7 +435,7 @@ class TelemetryEmitter:
|
||||
ttft_ms=usage.ttft_ms,
|
||||
max_inter_token_ms=usage.max_inter_token_ms,
|
||||
cache_hit=False,
|
||||
error=error,
|
||||
errors=_error_fields(error, event_kind="attempt", class_prefixed=class_prefixed_error),
|
||||
cached_prompt_tokens=usage.cached_prompt_tokens,
|
||||
model_reported=usage.model_reported,
|
||||
reasoning_tokens=usage.reasoning_tokens,
|
||||
@@ -296,10 +447,19 @@ class TelemetryEmitter:
|
||||
reasoning_effort=_attempt_effort(
|
||||
request=request, source=source, response=response, applies=reasoning_applies
|
||||
),
|
||||
operation=operation,
|
||||
event_kind="attempt",
|
||||
attempts=None,
|
||||
total_latency_ms=None,
|
||||
)
|
||||
|
||||
async def emit_cache_hit(self, *, request: ChatRequest, response: LLMResponse) -> None:
|
||||
"""缓存命中记录: cache_hit=True、latency_ms=0(VT 同款)。"""
|
||||
async def emit_cache_hit(
|
||||
self, *, request: ChatRequest, response: LLMResponse, operation: CallOperation
|
||||
) -> None:
|
||||
"""缓存命中记录: cache_hit=True、latency_ms=0(VT 同款)。
|
||||
|
||||
逻辑计数两列恒 NULL: 本行描述的是"一次命中",不是一次逻辑调用的终态。
|
||||
"""
|
||||
await self._record(
|
||||
request=request,
|
||||
call_id=response.call_id,
|
||||
@@ -315,7 +475,7 @@ class TelemetryEmitter:
|
||||
ttft_ms=None,
|
||||
max_inter_token_ms=None,
|
||||
cache_hit=True,
|
||||
error=None,
|
||||
errors=_ErrorFields(),
|
||||
# 决策 B1: 与 model/prompt_tokens 同一口径,原样回放历史值。
|
||||
# 统计供应商缓存命中率必须带 WHERE cache_hit = false,否则重复计数。
|
||||
cached_prompt_tokens=response.cached_prompt_tokens,
|
||||
@@ -334,12 +494,28 @@ class TelemetryEmitter:
|
||||
# 与 sampling 同一口径: 命中行没有选中源,源级档位与 `nearest` 映射
|
||||
# 都无从谈起,只记调用方这次要的档(response 里那个是历史那次实发的)
|
||||
reasoning_effort=_normalize_effort(request.reasoning_effort),
|
||||
operation=operation,
|
||||
event_kind="cache_hit",
|
||||
attempts=None,
|
||||
total_latency_ms=None,
|
||||
)
|
||||
|
||||
async def emit_terminal_failure(
|
||||
self, *, request: ChatRequest, call_id: str, latency_ms: int, error: str
|
||||
self,
|
||||
*,
|
||||
request: ChatRequest,
|
||||
call_id: str,
|
||||
error: PolyGatewayError | str,
|
||||
operation: CallOperation,
|
||||
stats: CallStats,
|
||||
class_prefixed_error: bool = False,
|
||||
) -> None:
|
||||
"""scope 级失败/取消记录: 无具体源,溯源字段置空标记。"""
|
||||
"""一次**逻辑调用**的失败终态: 无具体源,溯源字段置空标记。
|
||||
|
||||
`latency_ms` 与 `total_latency_ms` 同取**同一份冻结快照**,避免双时钟微差;
|
||||
故本方法不再收 `latency_ms`。token 与 cost 一律不从 attempt 行复制
|
||||
(费用聚合仍只由 attempt / cache_hit 行决定,口径不变)。
|
||||
"""
|
||||
await self._record(
|
||||
request=request,
|
||||
call_id=call_id,
|
||||
@@ -351,11 +527,13 @@ class TelemetryEmitter:
|
||||
prompt_tokens=0,
|
||||
completion_tokens=0,
|
||||
usage_source="unavailable",
|
||||
latency_ms=latency_ms,
|
||||
latency_ms=stats.total_latency_ms,
|
||||
ttft_ms=None,
|
||||
max_inter_token_ms=None,
|
||||
cache_hit=False,
|
||||
error=error,
|
||||
errors=_error_fields(
|
||||
error, event_kind="terminal_failure", class_prefixed=class_prefixed_error
|
||||
),
|
||||
cached_prompt_tokens=None,
|
||||
model_reported=None,
|
||||
reasoning_tokens=None,
|
||||
@@ -368,6 +546,10 @@ class TelemetryEmitter:
|
||||
meta=request.meta,
|
||||
# 可能根本没选出源,故与 sampling 同样只取请求档
|
||||
reasoning_effort=_normalize_effort(request.reasoning_effort),
|
||||
operation=operation,
|
||||
event_kind="terminal_failure",
|
||||
attempts=stats.attempts,
|
||||
total_latency_ms=stats.total_latency_ms,
|
||||
)
|
||||
|
||||
async def _record(
|
||||
@@ -387,7 +569,9 @@ class TelemetryEmitter:
|
||||
ttft_ms: float | None,
|
||||
max_inter_token_ms: float | None,
|
||||
cache_hit: bool,
|
||||
error: str | None,
|
||||
# 1.3.5: 错误四列已由 `_error_fields` 定型(三种入参形态 × 两类行的唯一规则所有者),
|
||||
# 本方法只搬运——拆成五个平铺参数就是把"一处定型"换回"三处各自拼"
|
||||
errors: _ErrorFields,
|
||||
cached_prompt_tokens: int | None,
|
||||
model_reported: str | None,
|
||||
sampling: str | None,
|
||||
@@ -403,6 +587,11 @@ class TelemetryEmitter:
|
||||
# 注释),本方法只搬运——把定型放这里就得再传一遍 response/source,等于把
|
||||
# "唯一 record_llm_call 调用点"换成"两处口径判断",那正是要避免的复制
|
||||
reasoning_effort: str | None,
|
||||
# —— 1.3.5: 行形态与逻辑调用快照 ——
|
||||
operation: CallOperation,
|
||||
event_kind: EventKind,
|
||||
attempts: int | None,
|
||||
total_latency_ms: int | None,
|
||||
) -> None:
|
||||
try:
|
||||
# 成本换算(M2 §6): 成功行按单价换算;缓存命中 0.0(未产生新调用);
|
||||
@@ -413,7 +602,7 @@ class TelemetryEmitter:
|
||||
# 用量不可得: 宁可算不出成本,也不算错成本(解耦设计 §3.1 不变式)。
|
||||
# 必须排在 cache_hit 之后——缓存命中未产生新调用,0.0 是事实而非未知
|
||||
cost = None
|
||||
elif error is None and model and self._pricing is not None:
|
||||
elif errors.error is None and model and self._pricing is not None:
|
||||
cost = self._pricing.cost(
|
||||
model, prompt_tokens, completion_tokens, cached_prompt_tokens
|
||||
)
|
||||
@@ -425,6 +614,7 @@ class TelemetryEmitter:
|
||||
_cap_messages(digest_messages(request.messages), self._text_cap),
|
||||
ensure_ascii=False,
|
||||
)
|
||||
context = request.call_context
|
||||
await self._recorder.record_llm_call(
|
||||
call_id=call_id,
|
||||
parent_call_id=request.parent_call_id,
|
||||
@@ -442,7 +632,7 @@ class TelemetryEmitter:
|
||||
ttft_ms=ttft_ms,
|
||||
max_inter_token_ms=max_inter_token_ms,
|
||||
cache_hit=cache_hit,
|
||||
error=error,
|
||||
error=errors.error,
|
||||
cost=cost,
|
||||
cached_prompt_tokens=cached_prompt_tokens,
|
||||
model_reported=model_reported,
|
||||
@@ -457,6 +647,19 @@ class TelemetryEmitter:
|
||||
# Postgres 那一路悄悄少一列数据
|
||||
thinking_observation=_normalize_observation(thinking_observation),
|
||||
reasoning_effort=reasoning_effort,
|
||||
# —— 1.3.5 十列 ——
|
||||
scope=self._scope,
|
||||
# 调用点给定的公开方法四值,**绝不读 `exc.operation`**(设计 §5 I1/I2)
|
||||
operation=operation,
|
||||
# 上下文缺席(库内现场构造的 ChatRequest)→ NULL,**不造 ID**(I5)
|
||||
logical_call_id=None if context is None else context.logical_call_id,
|
||||
event_kind=event_kind,
|
||||
http_status_code=errors.http_status_code,
|
||||
error_type=errors.error_type,
|
||||
cause_type=errors.cause_type,
|
||||
error_body=errors.error_body,
|
||||
attempts=attempts,
|
||||
total_latency_ms=total_latency_ms,
|
||||
)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
@@ -464,36 +667,69 @@ class TelemetryEmitter:
|
||||
logger.warning("遥测记录失败(降级不冒泡): {}", exc)
|
||||
|
||||
|
||||
async def emit_terminal_once(
|
||||
emitter: TelemetryEmitter | None,
|
||||
*,
|
||||
request: ChatRequest,
|
||||
context: _CallContext,
|
||||
error: PolyGatewayError | str,
|
||||
operation: CallOperation,
|
||||
class_prefixed_error: bool = False,
|
||||
) -> None:
|
||||
"""三个 client 共用的**终态唯一出口**: 去重 + 同步冻结快照 + best effort 写入。
|
||||
|
||||
去重由 `claim_terminal()` 承担(每逻辑调用至多一条终态行);`emitter is None`
|
||||
或已写过 → 直接返回。
|
||||
|
||||
**降级范围包含诊断字段的提取与构建**,不只是写入那一步: `_record` 内的
|
||||
`except Exception` 只兜住落库,而 `_error_fields` / `canonical_sampling_json`
|
||||
在它**之外**求值——下游经公共端口(自实现 `StructuredOutputStrategy` 或
|
||||
transport)构造出的 `ResultInvalidError(validation_errors=(非 str,))` 会让提取期
|
||||
抛 `TypeError` 顶替调用方本该收到的领域异常,错误四分类被击穿且终态行照样丢。
|
||||
故在此整段兜底,与 RetryMW 的 attempt 出口(`retry.py::_emit`)同款写法。
|
||||
终态行按 best effort: 兜底命中时本次逻辑调用**没有**终态行(`claim_terminal()`
|
||||
已消耗,不补写、不重试——重写一遍只会把同一个提取期异常再抛一次)。
|
||||
|
||||
**`CancelledError` 原样传播**(取消优先,不 shield、不开后台任务): 这一次
|
||||
`await` 本身就是新的取消点,外部取消落在它上时调用方会看到 `CancelledError`
|
||||
而非领域错误——与 TelemetryMW 的历史行为同款,已经人类批准(设计 §6/§10)。
|
||||
快照冻结是**同步**动作,故终态行不含它自身的写入耗时。
|
||||
"""
|
||||
if emitter is None or not context.claim_terminal():
|
||||
return
|
||||
try:
|
||||
stats = context.snapshot()
|
||||
await emitter.emit_terminal_failure(
|
||||
request=request,
|
||||
call_id=str(uuid.uuid4()),
|
||||
error=error,
|
||||
operation=operation,
|
||||
stats=stats,
|
||||
class_prefixed_error=class_prefixed_error,
|
||||
)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception as exc:
|
||||
logger.warning("终态遥测记录失败(降级不冒泡): {}", exc)
|
||||
|
||||
|
||||
class TelemetryMW:
|
||||
"""洋葱最外层: 观测尝试层看不见的路径,任何路径都留痕(遥测必录)。"""
|
||||
"""洋葱最外层: 只观测尝试层看不见的**缓存命中**。
|
||||
|
||||
1.3.5 起不再在此写终态失败行: 终态由 `GatewayClient.chat` 的公开边界经
|
||||
`emit_terminal_once` 统一写出。两处同时写会让同一次失败出两条终态行,
|
||||
而下游正是按 `event_kind = 'terminal_failure'` 计失败调用次数的。
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self, emitter: TelemetryEmitter, now: Callable[[], float] = time.monotonic
|
||||
) -> None:
|
||||
self._emitter = emitter
|
||||
self._now = now
|
||||
# `now` 自 1.3.5 起本类不再读取(终态行迁到公开边界后无耗时可测),但形参保留:
|
||||
# 删它会平白打断 `TelemetryMW(emitter, now=...)` 这一既有装配写法
|
||||
|
||||
async def __call__(self, request: ChatRequest, call_next: CallNext) -> LLMResponse:
|
||||
started = self._now()
|
||||
try:
|
||||
response = await call_next(request)
|
||||
except (GatewayUnavailableError, GovernanceBackendError, SourceNotConfiguredError) as exc:
|
||||
await self._emitter.emit_terminal_failure(
|
||||
request=request,
|
||||
call_id=str(uuid.uuid4()),
|
||||
latency_ms=int((self._now() - started) * 1000),
|
||||
error=str(exc),
|
||||
)
|
||||
raise
|
||||
except asyncio.CancelledError:
|
||||
# 尽力而为: 取消也留痕(§5.1 约定④);随后立即重抛
|
||||
await self._emitter.emit_terminal_failure(
|
||||
request=request,
|
||||
call_id=str(uuid.uuid4()),
|
||||
latency_ms=int((self._now() - started) * 1000),
|
||||
error="cancelled",
|
||||
)
|
||||
raise
|
||||
response = await call_next(request)
|
||||
if response.cache_hit:
|
||||
await self._emitter.emit_cache_hit(request=request, response=response)
|
||||
await self._emitter.emit_cache_hit(request=request, response=response, operation="chat")
|
||||
return response
|
||||
|
||||
+187
-24
@@ -23,6 +23,7 @@ from typing import TYPE_CHECKING, Any, Literal
|
||||
from loguru import logger
|
||||
|
||||
from polygateway.client import _aclose_component, _telemetry_status_of
|
||||
from polygateway.deadline import ensure_call_deadline, with_call_deadline
|
||||
from polygateway.errors import (
|
||||
AllSourcesExhausted,
|
||||
GovernanceBackendError,
|
||||
@@ -37,15 +38,17 @@ from polygateway.middleware.admission import SourceAdmission, settle_and_release
|
||||
from polygateway.middleware.breaker import BreakerGate
|
||||
from polygateway.middleware.ratelimit import QuotaGate
|
||||
from polygateway.middleware.retry import StallClock, _failure_reason, backoff_delay
|
||||
from polygateway.middleware.telemetry import TelemetryEmitter
|
||||
from polygateway.middleware.telemetry import TelemetryEmitter, emit_terminal_once
|
||||
from polygateway.ports import OutcomeAwareSelector
|
||||
from polygateway.types import (
|
||||
CallStats,
|
||||
ChatRequest,
|
||||
LLMResponse,
|
||||
OcrLayoutResult,
|
||||
OcrTextResult,
|
||||
TelemetryStatus,
|
||||
Usage,
|
||||
_CallContext,
|
||||
strip_unsupported_extra_body,
|
||||
validate_caller_dimensions,
|
||||
)
|
||||
@@ -65,6 +68,7 @@ if TYPE_CHECKING:
|
||||
)
|
||||
from polygateway.types import (
|
||||
BackpressurePolicy,
|
||||
CallOperation,
|
||||
OcrLayoutTransportResult,
|
||||
OcrTextTransportResult,
|
||||
RetryPolicy,
|
||||
@@ -112,10 +116,15 @@ class OcrClient:
|
||||
circuit_open: str = "fail_fast",
|
||||
telemetry: TelemetryRecorder | None = None,
|
||||
text_cap: int | None = None,
|
||||
call_deadline_s: float | None = None,
|
||||
now: Callable[[], float] = time.monotonic,
|
||||
sleep: Callable[[float], Awaitable[None]] = asyncio.sleep,
|
||||
rng: Callable[[], float] = random.random,
|
||||
) -> None:
|
||||
# 入口即校: 装配错误当场报,不等到第一次调用才炸
|
||||
self._call_deadline_s = ensure_call_deadline(
|
||||
call_deadline_s, "OcrClient(call_deadline_s=...)"
|
||||
)
|
||||
self._scope = scope
|
||||
# MonkeyOCR 只发 multipart 表单,带 extra_body 的源必须先剥离,否则
|
||||
# 遥测会记录一个从未发出的采样参数(issue #4 决策 G)
|
||||
@@ -126,7 +135,9 @@ class OcrClient:
|
||||
self._breaker = BreakerGate(breaker, scope=self._scope)
|
||||
self._transport = transport
|
||||
self._retry = retry
|
||||
self._emitter = TelemetryEmitter(telemetry, text_cap=text_cap) if telemetry else None
|
||||
self._emitter = (
|
||||
TelemetryEmitter(telemetry, scope=self._scope, text_cap=text_cap) if telemetry else None
|
||||
)
|
||||
self._telemetry = telemetry
|
||||
# 限流/熔断后端在此之外只以 QuotaGate/BreakerGate 的形态存在,自持一份
|
||||
# 引用才关得到自建的 redis 客户端(设计 §3.4)
|
||||
@@ -166,18 +177,29 @@ class OcrClient:
|
||||
parent_call_id: str | None = None,
|
||||
tenant_id: str | None = None,
|
||||
meta: Mapping[str, Any] | None = None,
|
||||
call_deadline_s: float | None = None,
|
||||
) -> OcrTextResult:
|
||||
"""一次治理文本转录(/ocr/text);text 空串 = 合法"无文字"。
|
||||
|
||||
`tenant_id` 与 `meta` 是调用方自定义维度,只进遥测(issue #11)。
|
||||
|
||||
`call_deadline_s` 是本次调用的墙钟硬边界(issue #22): `None` = 继承装配值。
|
||||
它治理的是**等待**——到期不等于未产出,返回时刻 = 期限 + 清理耗时。
|
||||
"""
|
||||
# 必须在进链路之前校验: 链路内的一切失败都被遥测层降级成 warning
|
||||
# (库铁律「遥测写失败降级不冒泡」),校验放下游等于没有校验
|
||||
dimension_tenant_id, dimensions = validate_caller_dimensions(
|
||||
tenant_id, meta, origin="recognize_text(tenant_id=..., meta=...)"
|
||||
)
|
||||
outcome = await self._call(
|
||||
"text", image, session_id, parent_call_id, dimension_tenant_id, dimensions
|
||||
outcome, call_stats = await self._call(
|
||||
"text",
|
||||
"recognize_text",
|
||||
image,
|
||||
session_id,
|
||||
parent_call_id,
|
||||
dimension_tenant_id,
|
||||
dimensions,
|
||||
call_deadline_s,
|
||||
)
|
||||
result = outcome.result
|
||||
return OcrTextResult(
|
||||
@@ -187,6 +209,7 @@ class OcrClient:
|
||||
latency_ms=outcome.latency_ms,
|
||||
call_id=outcome.call_id,
|
||||
raw=result.raw,
|
||||
call_stats=call_stats,
|
||||
)
|
||||
|
||||
async def parse_layout(
|
||||
@@ -197,17 +220,27 @@ class OcrClient:
|
||||
parent_call_id: str | None = None,
|
||||
tenant_id: str | None = None,
|
||||
meta: Mapping[str, Any] | None = None,
|
||||
call_deadline_s: float | None = None,
|
||||
) -> OcrLayoutResult:
|
||||
"""一次治理版面解析(/parse → ZIP);elements 空 = 合法"无元素"。
|
||||
|
||||
`tenant_id` 与 `meta` 是调用方自定义维度,只进遥测(issue #11)。
|
||||
|
||||
`call_deadline_s` 同 `recognize_text`(issue #22): `None` = 继承装配值。
|
||||
"""
|
||||
# 校验早于链路,理由同 recognize_text;origin 标明方法名以便定位入口
|
||||
dimension_tenant_id, dimensions = validate_caller_dimensions(
|
||||
tenant_id, meta, origin="parse_layout(tenant_id=..., meta=...)"
|
||||
)
|
||||
outcome = await self._call(
|
||||
"layout", image, session_id, parent_call_id, dimension_tenant_id, dimensions
|
||||
outcome, call_stats = await self._call(
|
||||
"layout",
|
||||
"parse_layout",
|
||||
image,
|
||||
session_id,
|
||||
parent_call_id,
|
||||
dimension_tenant_id,
|
||||
dimensions,
|
||||
call_deadline_s,
|
||||
)
|
||||
result = outcome.result
|
||||
return OcrLayoutResult(
|
||||
@@ -218,6 +251,7 @@ class OcrClient:
|
||||
latency_ms=outcome.latency_ms,
|
||||
call_id=outcome.call_id,
|
||||
raw=result.raw,
|
||||
call_stats=call_stats,
|
||||
)
|
||||
|
||||
async def check_health(self) -> dict[str, bool]:
|
||||
@@ -237,16 +271,71 @@ class OcrClient:
|
||||
async def _call(
|
||||
self,
|
||||
kind: _OcrKind,
|
||||
operation: CallOperation,
|
||||
image: bytes,
|
||||
session_id: str | None,
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
) -> _AttemptOutcome:
|
||||
call_deadline_s: float | None = None,
|
||||
) -> tuple[_AttemptOutcome, CallStats]:
|
||||
if not isinstance(image, bytes):
|
||||
raise TypeError("image 必须是 bytes(路径读取/批量拼帧留业务侧,D9)")
|
||||
if not image:
|
||||
raise ValueError("image 不能为空")
|
||||
# 期限校验与 `image` 校验同列(仍在 `_CallContext` 之前、创建 awaitable 之前)
|
||||
deadline = (
|
||||
self._call_deadline_s
|
||||
if call_deadline_s is None
|
||||
else ensure_call_deadline(call_deadline_s, f"{operation}(call_deadline_s=...)")
|
||||
)
|
||||
# M1 例外: `image` 校验在 `_call` 内而非公开方法,故上下文在该校验
|
||||
# **通过之后**创建——这样设计 §3 的"校验在统计边界外"对 OCR 才成立
|
||||
context = _CallContext(now=self._now)
|
||||
try:
|
||||
return await with_call_deadline(
|
||||
self._run(
|
||||
kind, operation, image, session_id, parent_call_id, tenant_id, meta, context
|
||||
),
|
||||
deadline_s=deadline,
|
||||
scope=self._scope,
|
||||
)
|
||||
except PolyGatewayError as exc:
|
||||
await self._emit_terminal(
|
||||
kind, operation, image, session_id, parent_call_id, tenant_id, meta, context, exc
|
||||
)
|
||||
raise
|
||||
except asyncio.CancelledError:
|
||||
# 三条链路同一口径尽力写一条(允许 0 条);取消优先,不 shield
|
||||
await self._emit_terminal(
|
||||
kind,
|
||||
operation,
|
||||
image,
|
||||
session_id,
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
"cancelled",
|
||||
)
|
||||
raise
|
||||
|
||||
async def _run(
|
||||
self,
|
||||
kind: _OcrKind,
|
||||
operation: CallOperation,
|
||||
image: bytes,
|
||||
session_id: str | None,
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
) -> tuple[_AttemptOutcome, CallStats]:
|
||||
"""选源与重试循环。
|
||||
|
||||
无源的 raise 必须在本方法内(而非循环之前的调用方): 它得被 `_call` 的
|
||||
`try` 包住,否则无源终态行根本写不出来(设计 §3.5)。
|
||||
"""
|
||||
if not self._sources:
|
||||
raise AllSourcesExhausted(scope=self._scope, reason="no_sources", retry_after_s=0.0)
|
||||
fails = 0
|
||||
@@ -260,10 +349,19 @@ class OcrClient:
|
||||
continue
|
||||
async with clock.attempting():
|
||||
outcome = await self._attempt(
|
||||
kind, image, *picked, reasons, session_id, parent_call_id, tenant_id, meta
|
||||
kind,
|
||||
operation,
|
||||
image,
|
||||
*picked,
|
||||
reasons,
|
||||
session_id,
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
)
|
||||
if isinstance(outcome, _AttemptOutcome):
|
||||
return outcome
|
||||
return outcome, context.snapshot()
|
||||
fails += 1
|
||||
if fails >= self._retry.max_attempts:
|
||||
raise AllSourcesExhausted(
|
||||
@@ -278,6 +376,7 @@ class OcrClient:
|
||||
async def _attempt(
|
||||
self,
|
||||
kind: _OcrKind,
|
||||
operation: CallOperation,
|
||||
image: bytes,
|
||||
source: SourceConfig,
|
||||
permit: Permit,
|
||||
@@ -287,19 +386,28 @@ class OcrClient:
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
) -> _AttemptOutcome | _FailedAttempt:
|
||||
call_id = str(uuid.uuid4())
|
||||
started = self._now()
|
||||
# 四个 emit 分支(成功/终态拒绝/取消/可重试失败)都必须带调用方维度:
|
||||
# 失败行与取消行同样需要租户归属,漏掉任一分支就会写出无归属的行
|
||||
# layout 的 POST + ZIP GET 在同一次 `_invoke` 内,故这里只登记 **1** 次
|
||||
context.register_attempt()
|
||||
try:
|
||||
# 裸生成时间(1.3.7 H8): 计时只包 transport 调用本身(同 RetryMW 口径);
|
||||
# layout 的 POST + ZIP GET 在同一次 `_invoke` 内,属同一段生成耗时
|
||||
gen_started = self._now()
|
||||
result = await self._invoke(kind, image, source, call_id)
|
||||
gen_ms = int((self._now() - gen_started) * 1000)
|
||||
await self._record_quietly(self._breaker.record_success(entry))
|
||||
await self._record_quietly(self._quota.mark_progress())
|
||||
self._feed_outcome(source.name, ok=True)
|
||||
latency_ms = int((self._now() - started) * 1000)
|
||||
context.record_generation(gen_ms, accumulate=False)
|
||||
await self._emit(
|
||||
kind,
|
||||
operation,
|
||||
image,
|
||||
source,
|
||||
call_id,
|
||||
@@ -308,6 +416,7 @@ class OcrClient:
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
result,
|
||||
)
|
||||
return _AttemptOutcome(result, source, call_id, latency_ms)
|
||||
@@ -315,6 +424,7 @@ class OcrClient:
|
||||
await self._gate_on_terminal(exc, entry)
|
||||
await self._emit(
|
||||
kind,
|
||||
operation,
|
||||
image,
|
||||
source,
|
||||
call_id,
|
||||
@@ -323,6 +433,7 @@ class OcrClient:
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
error=exc,
|
||||
)
|
||||
raise
|
||||
@@ -331,6 +442,7 @@ class OcrClient:
|
||||
await self._record_quietly(self._breaker.release_probe(entry))
|
||||
await self._emit(
|
||||
kind,
|
||||
operation,
|
||||
image,
|
||||
source,
|
||||
call_id,
|
||||
@@ -339,6 +451,7 @@ class OcrClient:
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
error="cancelled",
|
||||
)
|
||||
raise
|
||||
@@ -350,6 +463,7 @@ class OcrClient:
|
||||
self._feed_outcome(source.name, ok=False)
|
||||
await self._emit(
|
||||
kind,
|
||||
operation,
|
||||
image,
|
||||
source,
|
||||
call_id,
|
||||
@@ -358,10 +472,13 @@ class OcrClient:
|
||||
parent_call_id,
|
||||
tenant_id,
|
||||
meta,
|
||||
context,
|
||||
error=exc,
|
||||
)
|
||||
return _FailedAttempt(exc, immediate=dead)
|
||||
finally:
|
||||
# OCR 的 0 token 是**事实**而非"未知"(设计 §6.3 S6): 故取消也恰恰结 0,
|
||||
# 不引入 chat/embedding 那套 settlement_known 兜底。
|
||||
await settle_and_release(permit, 0)
|
||||
|
||||
async def _invoke(
|
||||
@@ -399,9 +516,60 @@ class OcrClient:
|
||||
except (GovernanceBackendError, SourceNotConfiguredError) as exc:
|
||||
logger.warning("OCR 治理记账写回降级(不冒泡): {}", exc)
|
||||
|
||||
def _request_for(
|
||||
self,
|
||||
kind: _OcrKind,
|
||||
image: bytes,
|
||||
session_id: str | None,
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
) -> ChatRequest:
|
||||
"""OCR 行的现场 ChatRequest: 占位摘要,**图像 bytes 永不入库**。
|
||||
|
||||
尝试行与终态行共用同一个构造点: 占位字面量复制成两份就会漂移。
|
||||
调用方维度与上下文必须显式填回(OCR 不走 chat 洋葱),否则 OCR 行的
|
||||
维度恒为空、`logical_call_id` 恒为 NULL。
|
||||
"""
|
||||
return ChatRequest(
|
||||
messages=[{"role": "user", "content": f"<ocr:{kind} image_bytes={len(image)}>"}],
|
||||
session_id=session_id,
|
||||
parent_call_id=parent_call_id,
|
||||
tenant_id=tenant_id,
|
||||
meta=meta,
|
||||
call_context=context,
|
||||
)
|
||||
|
||||
async def _emit_terminal(
|
||||
self,
|
||||
kind: _OcrKind,
|
||||
operation: CallOperation,
|
||||
image: bytes,
|
||||
session_id: str | None,
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
error: PolyGatewayError | str,
|
||||
) -> None:
|
||||
"""终态行: 沿用 `<ocr:{kind} image_bytes=…>` 占位,错误文本保留类名前缀。"""
|
||||
await emit_terminal_once(
|
||||
self._emitter,
|
||||
request=self._request_for(
|
||||
kind, image, session_id, parent_call_id, tenant_id, meta, context
|
||||
),
|
||||
context=context,
|
||||
error=error,
|
||||
operation=operation,
|
||||
# metric ocr-call-success 的注册口径是按类名归组,终态行同款保留
|
||||
class_prefixed_error=True,
|
||||
)
|
||||
|
||||
async def _emit(
|
||||
self,
|
||||
kind: _OcrKind,
|
||||
operation: CallOperation,
|
||||
image: bytes,
|
||||
source: SourceConfig,
|
||||
call_id: str,
|
||||
@@ -410,20 +578,15 @@ class OcrClient:
|
||||
parent_call_id: str | None,
|
||||
tenant_id: str | None,
|
||||
meta: dict[str, Any],
|
||||
context: _CallContext,
|
||||
result: OcrTextTransportResult | OcrLayoutTransportResult | None = None,
|
||||
error: object | None = None,
|
||||
error: PolyGatewayError | str | None = None,
|
||||
) -> None:
|
||||
"""逐尝试遥测(单一 Emitter): messages 占位摘要,图像 bytes 绝不入库。"""
|
||||
if self._emitter is None:
|
||||
return
|
||||
# 这个 ChatRequest 只为复用同一个 Emitter 而现场构造(OCR 不走 chat 洋葱),
|
||||
# 故调用方维度必须在这里显式填回,否则 OCR 行的维度恒为空
|
||||
request = ChatRequest(
|
||||
messages=[{"role": "user", "content": f"<ocr:{kind} image_bytes={len(image)}>"}],
|
||||
session_id=session_id,
|
||||
parent_call_id=parent_call_id,
|
||||
tenant_id=tenant_id,
|
||||
meta=meta,
|
||||
request = self._request_for(
|
||||
kind, image, session_id, parent_call_id, tenant_id, meta, context
|
||||
)
|
||||
latency_ms = int((self._now() - started) * 1000)
|
||||
response = None
|
||||
@@ -443,20 +606,19 @@ class OcrClient:
|
||||
source_name=source.name,
|
||||
usage_source="measured",
|
||||
)
|
||||
# 错误带异常类名前缀(metric ocr-call-success 注册口径: 按类名归组)
|
||||
if error is None or isinstance(error, str):
|
||||
error_text = error
|
||||
else:
|
||||
error_text = f"{type(error).__name__}: {error}"
|
||||
# 错误文本的类名前缀现由出口的显式策略参数承担(设计 §6 I7):
|
||||
# 三处各拼一遍才是下一次漂移的种子,而取消行传的是字符串,不受前缀影响
|
||||
await self._emitter.emit_attempt(
|
||||
request=request,
|
||||
source=source,
|
||||
call_id=call_id,
|
||||
latency_ms=latency_ms,
|
||||
response=response,
|
||||
error=error_text,
|
||||
error=error,
|
||||
# OCR 走 MonkeyOCR 自有端点,没有推理参数可言(理由同 embedding)
|
||||
reasoning_applies=False,
|
||||
operation=operation,
|
||||
class_prefixed_error=True,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
@@ -539,6 +701,7 @@ class OcrClient:
|
||||
# OCR 行与 chat 行写同一张 llm_calls;漏传这一条,同表内就一半受控
|
||||
# 一半不受控(issue #12)
|
||||
text_cap=gw.telemetry_text_cap,
|
||||
call_deadline_s=gw.call_deadline_s,
|
||||
)
|
||||
_mark_owned_components(client, limiter=limiter, breaker=breaker, telemetry=telemetry)
|
||||
return client
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
时间量纲一律**秒**(CHS Redis 实现内部的毫秒换算是后端私事,不进契约)。
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from collections.abc import Awaitable, Callable
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
@@ -46,6 +47,10 @@ class Transport(Protocol):
|
||||
该参数**不设默认值**,与 `TelemetryRecorder.record_llm_call` 同一既有约定:
|
||||
库外无第三方实现者,写全签名的成本为零,而默认值会把"某一层漏传"变成静默的
|
||||
"调用方没表态"——一次本该报错的漏配就此变成一次悄悄涨价的调用。
|
||||
|
||||
`first_token_event` 同一约定(1.3.7 对冲 H2): `None` = 调用方不观测首 token
|
||||
(未启用对冲);非流式实现**永不置位**(物理上无中途信号,事件自然退化为纯时间
|
||||
阈值),流式实现在首个增量(内容或思考)到达时置位。
|
||||
"""
|
||||
|
||||
async def complete(
|
||||
@@ -57,6 +62,7 @@ class Transport(Protocol):
|
||||
overlay: dict[str, Any],
|
||||
call_id: str,
|
||||
reasoning_effort: Effort | None,
|
||||
first_token_event: asyncio.Event | None,
|
||||
) -> TransportResult: ...
|
||||
|
||||
|
||||
@@ -271,7 +277,7 @@ class TelemetryStatusProvider(Protocol):
|
||||
|
||||
@runtime_checkable
|
||||
class TelemetryRecorder(Protocol):
|
||||
"""遥测后端;26 字段冻结(M1 设计 §4.4 + issue #3/#4/#11/#16/#20),唯一调用点是 TelemetryEmitter。
|
||||
"""遥测后端;36 字段冻结(M1 设计 §4.4 + issue #3/#4/#11/#16/#20 + 1.3.5),唯一调用点是 TelemetryEmitter。
|
||||
|
||||
新增参数不设默认值: 库外无第三方实现者(三项目迁移时删除了各自的同名
|
||||
Protocol),完整签名的成本为零,而少写一列会被 emitter 的降级吞成 warning。
|
||||
@@ -286,6 +292,15 @@ class TelemetryRecorder(Protocol):
|
||||
|
||||
recorder 只负责落库,不做任何语义判断,与 `sampling` 列由
|
||||
`canonical_sampling_json()` 在 emitter 侧定型是同一先例。
|
||||
|
||||
1.3.5 新增十列同理已在 emitter 侧定型: `operation` 是**公开方法**四值之一
|
||||
(与 `PolyGatewayError.operation` 这个 HTTP 子操作是两个语义);`event_kind` 区分
|
||||
attempt / cache_hit / terminal_failure 三类行;`attempts` 与 `total_latency_ms`
|
||||
只在终态行非空;`error_body` 沿用 transport 侧 `summarize_body` 的上限,
|
||||
**不进 `PGW_TELEMETRY_TEXT_CAP` 的覆盖面**。
|
||||
|
||||
**本签名是装配闸的唯一事实源**: `TelemetryEmitter.__init__` 按它派生参数名做
|
||||
一次 `signature.bind` 形状校验(设计 §7),改本签名即改闸的判据。
|
||||
"""
|
||||
|
||||
async def record_llm_call(
|
||||
@@ -317,4 +332,14 @@ class TelemetryRecorder(Protocol):
|
||||
meta: str,
|
||||
thinking_observation: str,
|
||||
reasoning_effort: str | None,
|
||||
scope: str,
|
||||
operation: str,
|
||||
logical_call_id: str | None,
|
||||
event_kind: str,
|
||||
http_status_code: int | None,
|
||||
error_type: str | None,
|
||||
cause_type: str | None,
|
||||
error_body: str | None,
|
||||
attempts: int | None,
|
||||
total_latency_ms: int | None,
|
||||
) -> None: ...
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
多处各存一份必然漂移,而漂移的表现是"下游照打印的 SQL 建完表,库仍报缺列"。
|
||||
|
||||
**`COLUMNS` 是 INSERT 字段序,不是物理列序**: 数据库自填的 `created_at` 不在其中(它带
|
||||
`DEFAULT now()` / `datetime('now')`,库从不显式写它)。物理表列 = 26 个 INSERT 字段 +
|
||||
`created_at` = 27;列数断言一律按物理列数写,两套口径混用是最易错处。
|
||||
`DEFAULT now()` / `datetime('now')`,库从不显式写它)。物理表列 = 36 个 INSERT 字段 +
|
||||
`created_at` = 37;列数断言一律按物理列数写,两套口径混用是最易错处。
|
||||
|
||||
本模块只依赖标准库: `telemetry/` 与 `backends/`、`transports/`、`structured/` 同层且
|
||||
互不依赖(import-linter 契约执法)。
|
||||
@@ -52,7 +52,17 @@ CREATE TABLE IF NOT EXISTS llm_calls (
|
||||
tenant_id TEXT NOT NULL DEFAULT '',
|
||||
meta TEXT NOT NULL DEFAULT '{}',
|
||||
thinking_observation TEXT,
|
||||
reasoning_effort TEXT
|
||||
reasoning_effort TEXT,
|
||||
scope TEXT,
|
||||
operation TEXT,
|
||||
logical_call_id TEXT,
|
||||
event_kind TEXT,
|
||||
http_status_code INTEGER,
|
||||
error_type TEXT,
|
||||
cause_type TEXT,
|
||||
error_body TEXT,
|
||||
attempts INTEGER,
|
||||
total_latency_ms INTEGER
|
||||
);
|
||||
"""
|
||||
|
||||
@@ -84,7 +94,17 @@ CREATE TABLE IF NOT EXISTS llm_calls (
|
||||
tenant_id TEXT NOT NULL DEFAULT '',
|
||||
meta JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
thinking_observation TEXT,
|
||||
reasoning_effort TEXT
|
||||
reasoning_effort TEXT,
|
||||
scope TEXT,
|
||||
operation TEXT,
|
||||
logical_call_id TEXT,
|
||||
event_kind TEXT,
|
||||
http_status_code INTEGER,
|
||||
error_type TEXT,
|
||||
cause_type TEXT,
|
||||
error_body TEXT,
|
||||
attempts INTEGER,
|
||||
total_latency_ms INTEGER
|
||||
);
|
||||
"""
|
||||
|
||||
@@ -105,6 +125,18 @@ SQLITE_BACKFILL = (
|
||||
# 同样可空,但这里 NULL 表达的是"调用方没表态"(issue #20): 它与 'none'
|
||||
# (明确要求不推理)是两回事,折叠成任一档都等于替上游声称了它没说过的事
|
||||
("reasoning_effort", "TEXT"),
|
||||
# 1.3.5 十列: 全部可空且无默认值——旧行的 NULL 表达的是"补列之前根本没记过
|
||||
# 这件事",与任何哨兵值都不是一回事,故不回填(设计 §5)
|
||||
("scope", "TEXT"),
|
||||
("operation", "TEXT"),
|
||||
("logical_call_id", "TEXT"),
|
||||
("event_kind", "TEXT"),
|
||||
("http_status_code", "INTEGER"),
|
||||
("error_type", "TEXT"),
|
||||
("cause_type", "TEXT"),
|
||||
("error_body", "TEXT"),
|
||||
("attempts", "INTEGER"),
|
||||
("total_latency_ms", "INTEGER"),
|
||||
)
|
||||
|
||||
# PG 补列的列定义。语句由此派生成两份文本(见下),使"库内执行的那份"与"打印给
|
||||
@@ -120,6 +152,17 @@ _PG_BACKFILL_DECLS = (
|
||||
# 可空,理由同 SQLITE_BACKFILL 同名项
|
||||
("thinking_observation", "TEXT"),
|
||||
("reasoning_effort", "TEXT"),
|
||||
# 1.3.5 十列,列序与 SQLITE_BACKFILL 逐项对齐(两条路径的物理列序不许分叉)
|
||||
("scope", "TEXT"),
|
||||
("operation", "TEXT"),
|
||||
("logical_call_id", "TEXT"),
|
||||
("event_kind", "TEXT"),
|
||||
("http_status_code", "INTEGER"),
|
||||
("error_type", "TEXT"),
|
||||
("cause_type", "TEXT"),
|
||||
("error_body", "TEXT"),
|
||||
("attempts", "INTEGER"),
|
||||
("total_latency_ms", "INTEGER"),
|
||||
)
|
||||
|
||||
# 新列排在 created_at 之后: 与旧表 ALTER 追加的位置一致(见 SQLITE_BACKFILL 同款注释)。
|
||||
@@ -158,6 +201,17 @@ COLUMNS = (
|
||||
"meta",
|
||||
"thinking_observation",
|
||||
"reasoning_effort",
|
||||
# —— 1.3.5 逻辑调用统计与结构化失败诊断(issue #19/#23)——
|
||||
"scope",
|
||||
"operation",
|
||||
"logical_call_id",
|
||||
"event_kind",
|
||||
"http_status_code",
|
||||
"error_type",
|
||||
"cause_type",
|
||||
"error_body",
|
||||
"attempts",
|
||||
"total_latency_ms",
|
||||
)
|
||||
|
||||
_COLUMN_SET = frozenset(COLUMNS)
|
||||
|
||||
@@ -143,7 +143,7 @@ class SQLiteRecorder:
|
||||
logger.warning("SQLite 遥测补列失败(写入将逐行降级): {}", exc)
|
||||
|
||||
async def record_llm_call(self, **fields: object) -> None:
|
||||
"""写一行遥测;字段集合即 26 字段冻结签名(ports.TelemetryRecorder)。
|
||||
"""写一行遥测;字段集合即 36 字段冻结签名(ports.TelemetryRecorder)。
|
||||
|
||||
取值按 `self._columns`(manual 档可能已被裁剪),与 `self._insert` 的
|
||||
占位符同序——两者必须一起改,分开改就是把值写进错位的列。
|
||||
|
||||
@@ -9,6 +9,7 @@ SSE 纯函数移植 VT `adapters/llm.py:51-124`;错误翻译移植 CHS
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
import time
|
||||
from dataclasses import replace
|
||||
@@ -45,6 +46,7 @@ from polygateway.types import (
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import asyncio
|
||||
from collections.abc import AsyncIterator, Callable, Mapping
|
||||
|
||||
_THINK_PATTERN = re.compile(r"<think>(.*?)</think>", re.DOTALL)
|
||||
@@ -108,14 +110,30 @@ async def _iter_sse_deltas(
|
||||
# —— 错误翻译(CHS invokers.py 同款)——
|
||||
|
||||
|
||||
def _parse_retry_after(raw: str | None) -> float | None:
|
||||
"""解析 Retry-After 头;仅支持秒数形态,HTTP-date 返回 None(CHS 同款)。"""
|
||||
def _parse_retry_after(raw: str | None, *, source_name: str) -> float | None:
|
||||
"""解析 Retry-After 头;仅支持秒数形态,HTTP-date 返回 None(CHS 同款)。
|
||||
|
||||
**非有限值必须当作"无提示"**(issue F1): `"inf"` / `"1e999"` 能被 `float()`
|
||||
成功解析,又能通过 `seconds > 0`,于是一路变成 `retry_after_s=inf`——而
|
||||
`backoff_delay` 的 `max(delay, retry_after)` 取大之后就是一次**永不醒来**的
|
||||
退避 sleep(库刻意不拿 `backoff_max_s` 去夹它,见设计 §6.2)。
|
||||
|
||||
`source_name` 是**必填** keyword-only 参数: 本函数是私有的,不给默认值,
|
||||
漏传即 `TypeError`,免得将来新增调用点静默丢掉源标识(告警定位不到是哪个源)。
|
||||
`nan` 不新增分支,沿用既有 `seconds > 0` 恒假的值语义。
|
||||
"""
|
||||
if raw is None:
|
||||
return None
|
||||
try:
|
||||
seconds = float(raw.strip())
|
||||
except ValueError:
|
||||
return None
|
||||
if math.isinf(seconds):
|
||||
# 只写源名与判据词: 429 风暴下回显原始头会把日志淹掉,也无助于定位
|
||||
logger.warning(
|
||||
"{} 的 Retry-After 非有限值,按无提示处理(retry_after_not_finite)", source_name
|
||||
)
|
||||
return None
|
||||
return seconds if seconds > 0 else None
|
||||
|
||||
|
||||
@@ -137,7 +155,7 @@ def _translate_429(
|
||||
)
|
||||
return TransientError(
|
||||
compose_message(f"{source.name} 限速: 429", summary),
|
||||
retry_after_s=_parse_retry_after(headers.get("retry-after")),
|
||||
retry_after_s=_parse_retry_after(headers.get("retry-after"), source_name=source.name),
|
||||
**ctx,
|
||||
)
|
||||
|
||||
@@ -154,18 +172,28 @@ def _classify(status: int) -> tuple[type[PolyGatewayError], str]:
|
||||
|
||||
|
||||
def _status_to_error(
|
||||
source: SourceConfig, status: int, body_text: str, headers: Mapping[str, str]
|
||||
source: SourceConfig,
|
||||
status: int,
|
||||
body_text: str,
|
||||
headers: Mapping[str, str],
|
||||
*,
|
||||
operation: str,
|
||||
) -> Exception:
|
||||
"""非 2xx → 领域错误,**全部分支**携带响应体摘要(issue #10)。
|
||||
|
||||
摘要只算一次,message 与 `body_text` 共用同一份串: 两份不同长度会让"遥测里
|
||||
看到的"与"下游 catch 到的"对不上,排查时反而多一层困惑。
|
||||
|
||||
`operation` 是 **HTTP 子操作**词表(`chat` / `embedding` / `ocr_text` / ...),由
|
||||
调用点显式给定。它曾被硬编码成 `"chat"`,而 `embed()` 的非 200 分支也走它
|
||||
——于是现存所有 embedding HTTP 失败的 `exc.operation` 都是错的(1.3.5 设计 §5 I1)。
|
||||
注意它与遥测新列 `operation`(公开方法四值)是**两个语义**,不做自动转换。
|
||||
"""
|
||||
summary = summarize_body(body_text)
|
||||
ctx: dict[str, Any] = {
|
||||
"source_name": source.name,
|
||||
"status_code": status,
|
||||
"operation": "chat",
|
||||
"operation": operation,
|
||||
"body_text": summary,
|
||||
}
|
||||
if status == 429:
|
||||
@@ -405,11 +433,14 @@ class OpenAICompatTransport:
|
||||
overlay: dict[str, Any],
|
||||
call_id: str,
|
||||
reasoning_effort: Effort | None,
|
||||
first_token_event: asyncio.Event | None,
|
||||
) -> TransportResult:
|
||||
"""一次原始调用;HTTP/线路/流式异常按 ARCH §6.2 翻译为领域错误。
|
||||
|
||||
`reasoning_effort` 是**请求级**档位(`None` = 不表态);它与源级配置的优先级
|
||||
在 `_build_payload` 里由 `effective_effort` 裁定,本层只负责把它送到。
|
||||
`first_token_event` 为对冲处置位: `None` = 不观测首 token;仅流式路径
|
||||
(`_complete_stream`)在首个增量到达时置位,非流式路径收它但永不置位。
|
||||
"""
|
||||
profile = get_provider(source.provider, registry=self._registry)
|
||||
try:
|
||||
@@ -434,9 +465,13 @@ class OpenAICompatTransport:
|
||||
ctx: dict[str, Any] = {"source_name": source.name, "operation": "chat"}
|
||||
try:
|
||||
if stream:
|
||||
result = await self._complete_stream(client, url, payload, source, profile)
|
||||
result = await self._complete_stream(
|
||||
client, url, payload, source, profile, first_token_event
|
||||
)
|
||||
else:
|
||||
result = await self._complete_once(client, url, payload, source, profile)
|
||||
result = await self._complete_once(
|
||||
client, url, payload, source, profile, first_token_event
|
||||
)
|
||||
except StreamLivenessTimeout as exc:
|
||||
raise TransientError(f"{source.name} 流活性超时({exc.kind})", **ctx) from exc
|
||||
except httpx.TimeoutException as exc:
|
||||
@@ -509,7 +544,10 @@ class OpenAICompatTransport:
|
||||
except httpx.TransportError as exc:
|
||||
raise TransientError(f"{source.name} 网络错误: {exc}", **ctx) from exc
|
||||
if resp.status_code != 200:
|
||||
raise _status_to_error(source, resp.status_code, resp.text, resp.headers)
|
||||
# 历史误标修正: 本分支属 `embed()`,与上方 ctx 同为 `"embedding"`
|
||||
raise _status_to_error(
|
||||
source, resp.status_code, resp.text, resp.headers, operation="embedding"
|
||||
)
|
||||
return _parse_embedding_payload(resp, source, len(texts))
|
||||
|
||||
async def _complete_stream(
|
||||
@@ -519,12 +557,15 @@ class OpenAICompatTransport:
|
||||
payload: dict[str, Any],
|
||||
source: SourceConfig,
|
||||
profile: ProviderProfile,
|
||||
first_token_event: asyncio.Event | None,
|
||||
) -> TransportResult:
|
||||
started = time.monotonic()
|
||||
async with client.stream("POST", url, json=payload) as resp:
|
||||
if resp.status_code != 200:
|
||||
body = (await resp.aread()).decode("utf-8", errors="replace")
|
||||
raise _status_to_error(source, resp.status_code, body, resp.headers)
|
||||
raise _status_to_error(
|
||||
source, resp.status_code, body, resp.headers, operation="chat"
|
||||
)
|
||||
sink: dict[str, Any] = {}
|
||||
guarded = stream_with_liveness_timeouts(
|
||||
_iter_sse_deltas(resp.aiter_lines(), sink),
|
||||
@@ -541,6 +582,10 @@ class OpenAICompatTransport:
|
||||
now = time.monotonic()
|
||||
if ttft_ms is None:
|
||||
ttft_ms = (now - started) * 1000
|
||||
# 首个增量即对冲语义上的"首 token"(思考增量同样是存活证据,
|
||||
# 与看门狗活性口径一致);None = 调用方未启用对冲,零分支成本
|
||||
if first_token_event is not None:
|
||||
first_token_event.set()
|
||||
else:
|
||||
max_gap = max(max_gap, (now - last) * 1000)
|
||||
last = now
|
||||
@@ -618,11 +663,18 @@ class OpenAICompatTransport:
|
||||
payload: dict[str, Any],
|
||||
source: SourceConfig,
|
||||
profile: ProviderProfile,
|
||||
first_token_event: asyncio.Event | None,
|
||||
) -> TransportResult:
|
||||
"""非流式快路径(三项目均无,库新增): 单 JSON 响应,仅 total 超时。"""
|
||||
"""非流式快路径(三项目均无,库新增): 单 JSON 响应,仅 total 超时。
|
||||
|
||||
接收 `first_token_event` 但**永不置位**: 非流式无中途信号,事件自然退化
|
||||
为纯时间阈值(对冲只能靠 `hedge_after_s` 触发)。
|
||||
"""
|
||||
resp = await client.post(url, json=payload)
|
||||
if resp.status_code != 200:
|
||||
raise _status_to_error(source, resp.status_code, resp.text, resp.headers)
|
||||
raise _status_to_error(
|
||||
source, resp.status_code, resp.text, resp.headers, operation="chat"
|
||||
)
|
||||
try:
|
||||
body = resp.json()
|
||||
except json.JSONDecodeError as exc:
|
||||
|
||||
+138
-2
@@ -8,11 +8,12 @@ import dataclasses
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
from collections.abc import Mapping
|
||||
import uuid
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from enum import StrEnum
|
||||
from types import MappingProxyType
|
||||
from typing import Any
|
||||
from typing import Any, Literal
|
||||
|
||||
from loguru import logger
|
||||
|
||||
@@ -274,6 +275,122 @@ class ThinkingObservation(StrEnum):
|
||||
UNKNOWN = "unknown"
|
||||
|
||||
|
||||
CallOperation = Literal["chat", "embed", "recognize_text", "parse_layout"]
|
||||
"""遥测 `operation` 列的值域: **公开方法**四值,由调用点给定。
|
||||
|
||||
与 `PolyGatewayError.operation`(HTTP 子操作,如 `download_result`)是**两个语义**,
|
||||
不做自动转换;链路上任何位置都不得读 `exc.operation` 来填本列(设计 §5 I1/I2)。"""
|
||||
|
||||
CALL_OPERATIONS: tuple[CallOperation, ...] = ("chat", "embed", "recognize_text", "parse_layout")
|
||||
|
||||
EventKind = Literal["attempt", "cache_hit", "terminal_failure"]
|
||||
"""一行遥测描述的事件形态;旧行 NULL,不回填。
|
||||
|
||||
终态行与 attempt 行**不是重复事实**(前者描述逻辑终态,后者描述单次尝试),
|
||||
故禁止按 `error IS NOT NULL` 跨两类直接计失败调用次数(设计 §6/§8)。"""
|
||||
|
||||
EVENT_KINDS: tuple[EventKind, ...] = ("attempt", "cache_hit", "terminal_failure")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CallStats:
|
||||
"""一次**公开调用**(而非单次尝试)的统计快照(设计 §3)。
|
||||
|
||||
四种响应各平铺三字段会立刻漂移,故收敛成单一对象并由包根导出。
|
||||
第三方合成响应的 `None` 表示**未知**,不得伪造 0。
|
||||
"""
|
||||
|
||||
logical_call_id: str
|
||||
"""每次公开调用一个 UUID;重试、结构化重问、embedding 分批共享同一个。
|
||||
|
||||
不占用既有 `parent_call_id`(后者是调用方的业务关联,语义不变)。"""
|
||||
|
||||
attempts: int
|
||||
"""准入后实际调用 transport 端口的次数;含免预算 429 与端口本地拒绝。
|
||||
|
||||
**不是 HTTP 请求条数**: OCR layout 的 POST + ZIP GET 在同一次 transport
|
||||
调用内,计 1 次。缓存命中与空输入是合法的零尝试。"""
|
||||
|
||||
total_latency_ms: int
|
||||
"""从输入校验通过到返回/异常传播前的单调时钟快照。
|
||||
|
||||
含缓存 IO、退避等待、准入等待、重问、分批与内联记账。
|
||||
"总耗时减最后一次尝试耗时"**不等于**纯等待(含其他本地工作)。"""
|
||||
|
||||
hedges: int = 0
|
||||
"""本次逻辑调用实际并发发出的对冲路数(触发但准入失败静默不计);1.3.6 及以前恒 0。"""
|
||||
generation_ms: int = 0
|
||||
"""裸生成时间: 赢家/成功那次 transport 调用的墙钟时长(口径见设计 §4.5 H8)。"""
|
||||
hedge_won: bool = False
|
||||
"""赢家是否为对冲路;无对冲恒 False。"""
|
||||
|
||||
|
||||
class _CallContext:
|
||||
"""私有可变逻辑调用上下文: 只持计数、单调时钟与终态去重位,不做 I/O。
|
||||
|
||||
**每逻辑调用一个实例**,可多任务并发登记(对冲);全部方法无 await,
|
||||
事件循环内任务安全。**严禁提升为 client 实例属性**
|
||||
——那会让同一 client 的并发调用互相串掉计数与逻辑 ID(库铁律"纯 asyncio 中立"、
|
||||
VT `evolve_llm = llm` 教训的同一形态)。
|
||||
"""
|
||||
|
||||
__slots__ = (
|
||||
"_attempts",
|
||||
"_generation_ms",
|
||||
"_hedge_won",
|
||||
"_hedges",
|
||||
"_now",
|
||||
"_started",
|
||||
"_terminal_claimed",
|
||||
"logical_call_id",
|
||||
)
|
||||
|
||||
def __init__(self, *, now: Callable[[], float]) -> None:
|
||||
self.logical_call_id = str(uuid.uuid4())
|
||||
self._now = now
|
||||
self._started = now()
|
||||
self._attempts = 0
|
||||
self._generation_ms = 0
|
||||
self._hedges = 0
|
||||
self._hedge_won = False
|
||||
self._terminal_claimed = False
|
||||
|
||||
def register_attempt(self) -> None:
|
||||
"""transport 调用**前**登记一次尝试(含免预算 429 与端口本地拒绝)。
|
||||
|
||||
登记点在调用前而非成功后: 否则失败与取消的尝试会从计数里消失,
|
||||
而那正是诊断时最需要看见的那几次。
|
||||
"""
|
||||
self._attempts += 1
|
||||
|
||||
def record_generation(self, elapsed_ms: int, *, accumulate: bool) -> None:
|
||||
"""chat/OCR 覆盖(结构化重问最后一轮为准);embedding 分批累加。"""
|
||||
self._generation_ms = self._generation_ms + elapsed_ms if accumulate else elapsed_ms
|
||||
|
||||
def register_hedge(self, *, hedge_won: bool) -> None:
|
||||
"""对冲路实际发出即计数;赢家裁定后一次性登记。"""
|
||||
self._hedges += 1
|
||||
self._hedge_won = hedge_won
|
||||
|
||||
def snapshot(self) -> CallStats:
|
||||
"""同步冻结当前快照;**绝不 await**,可多次调用。"""
|
||||
return CallStats(
|
||||
logical_call_id=self.logical_call_id,
|
||||
attempts=self._attempts,
|
||||
total_latency_ms=int((self._now() - self._started) * 1000),
|
||||
hedges=self._hedges,
|
||||
generation_ms=self._generation_ms,
|
||||
hedge_won=self._hedge_won,
|
||||
)
|
||||
|
||||
def claim_terminal(self) -> bool:
|
||||
"""首次 `True`、其后 `False`: 保证每逻辑调用至多写一条终态行。"""
|
||||
if self._terminal_claimed:
|
||||
return False
|
||||
self._terminal_claimed = True
|
||||
return True
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LLMResponse:
|
||||
"""一次治理调用的统一响应(与三项目超集兼容,ARCH §5.1)。"""
|
||||
@@ -336,6 +453,9 @@ class LLMResponse:
|
||||
`None` 不是"没推理": 库不表态时也不推定模型自己的默认档——"没看见"不许说成
|
||||
"发生了"(同 `thinking_observation` 的 `UNKNOWN` 一脉)。"""
|
||||
|
||||
call_stats: CallStats | None = None
|
||||
"""本次**逻辑调用**的统计快照(1.3.5);`None` = 未知,不得读成 0。"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ChatRequest:
|
||||
@@ -382,6 +502,16 @@ class ChatRequest:
|
||||
而档位要经能力表校验、要进缓存 key、要落遥测——混进直通层等于放弃这三样,
|
||||
正是 issue #20 里下游手写 `extra_body` 绕过全部治理的那条路。"""
|
||||
|
||||
# —— 库内部逻辑调用上下文(1.3.5;追加在末尾,不扰动既有字段的位置构造)——
|
||||
call_context: _CallContext | None = field(default=None, compare=False, repr=False)
|
||||
"""库内部逻辑调用上下文;`None` = 库内现场构造的请求,遥测 `logical_call_id` 落 NULL。
|
||||
|
||||
`compare=False, repr=False` 不是洁癖: 进 `compare` 会让两个内容相同的请求因
|
||||
"不是同一次调用"而不相等,进 `repr` 则把库内部件泄进调用方的日志。
|
||||
|
||||
洋葱各层经 `dataclasses.replace` 派生请求时保留**同一引用**(不是拷贝),
|
||||
重试/重问/分批才能共享同一个逻辑 ID 与计数。"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Usage:
|
||||
@@ -722,6 +852,8 @@ class OcrTextResult:
|
||||
latency_ms: int
|
||||
call_id: str
|
||||
raw: dict[str, Any]
|
||||
call_stats: CallStats | None = None
|
||||
"""本次逻辑调用的统计快照(1.3.5);`None` = 未知。"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -739,6 +871,8 @@ class OcrLayoutResult:
|
||||
latency_ms: int
|
||||
call_id: str
|
||||
raw: dict[str, Any]
|
||||
call_stats: CallStats | None = None
|
||||
"""本次逻辑调用的统计快照(1.3.5);`None` = 未知。"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -783,3 +917,5 @@ class EmbeddingResponse:
|
||||
call_id: str
|
||||
source_name: str
|
||||
cost: float | None = None
|
||||
call_stats: CallStats | None = None
|
||||
"""本次逻辑调用(含全部分批)的统计快照(1.3.5);`None` = 未知。"""
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""测试侧独立 HTTP 取证装配;无环境自读取或成功 SSE 预读。"""
|
||||
|
||||
import asyncio
|
||||
from collections.abc import AsyncIterator, Iterator, Mapping
|
||||
from contextlib import AsyncExitStack, asynccontextmanager, contextmanager
|
||||
from contextvars import ContextVar
|
||||
@@ -290,6 +291,7 @@ class ObservedTransport:
|
||||
overlay: dict[str, Any],
|
||||
call_id: str,
|
||||
reasoning_effort: Effort | None,
|
||||
first_token_event: asyncio.Event | None,
|
||||
) -> TransportResult:
|
||||
"""与生产端口逐参数同签名。"""
|
||||
with self._capture.attempt_context(call_id):
|
||||
@@ -301,6 +303,7 @@ class ObservedTransport:
|
||||
overlay=overlay,
|
||||
call_id=call_id,
|
||||
reasoning_effort=reasoning_effort,
|
||||
first_token_event=first_token_event,
|
||||
)
|
||||
|
||||
async def embed(
|
||||
|
||||
@@ -28,7 +28,7 @@ import pytest
|
||||
from dotenv import dotenv_values
|
||||
|
||||
from polygateway.telemetry.postgres import PostgresRecorder
|
||||
from polygateway.telemetry.schema import COLUMNS, telemetry_schema_sql
|
||||
from polygateway.telemetry.schema import COLUMNS, insert_sql, telemetry_schema_sql
|
||||
|
||||
_EXPECTED_COLUMNS = [
|
||||
"call_id",
|
||||
@@ -58,8 +58,31 @@ _EXPECTED_COLUMNS = [
|
||||
"meta",
|
||||
"thinking_observation",
|
||||
"reasoning_effort",
|
||||
# —— 1.3.5 逻辑调用统计与结构化失败诊断的十列(issue #19/#23)——
|
||||
"scope",
|
||||
"operation",
|
||||
"logical_call_id",
|
||||
"event_kind",
|
||||
"http_status_code",
|
||||
"error_type",
|
||||
"cause_type",
|
||||
"error_body",
|
||||
"attempts",
|
||||
"total_latency_ms",
|
||||
]
|
||||
|
||||
# 1.3.5 之前那张表的 27 个物理列(26 个 INSERT 字段 + created_at)。写成固定切片
|
||||
# 而非 `[:-10]`: 后者会随下一次补列静默漂移到另一张表上,而漂移的表现是
|
||||
# "旧表补列"用例悄悄改测了别的形态。
|
||||
_PRE_135_COLUMNS = _EXPECTED_COLUMNS[:27]
|
||||
|
||||
# 1.3.4 版本的 recorder 实际写入的列(物理列去掉库从不显式写的 created_at)
|
||||
_PRE_135_WRITTEN_COLUMNS = [c for c in _PRE_135_COLUMNS if c != "created_at"]
|
||||
|
||||
# 1.3.5 新增的十列,按 `COLUMNS`(即 INSERT 字段序)排列: manual 档告警逐字比对与
|
||||
# "旧行新列为 NULL"两处共用同一份,免得两边各抄一份后各自漂移。
|
||||
_CALL_OBSERVABILITY_COLUMNS = [c for c in COLUMNS if c not in _PRE_135_WRITTEN_COLUMNS]
|
||||
|
||||
|
||||
def _dsn() -> str | None:
|
||||
"""读 `.env` 的 DSN 并剥掉 SQLAlchemy 风格的 `+driver` 后缀;未配置返回 None。"""
|
||||
@@ -92,13 +115,13 @@ async def template_admin_dsn() -> str:
|
||||
return value
|
||||
|
||||
|
||||
async def _record_minimal(
|
||||
recorder: PostgresRecorder, call_id: str | None = None, **overrides
|
||||
) -> dict[str, object]:
|
||||
"""记一行最小遥测,并**返回实际提交的字段**供调用方逐列比对回读结果。
|
||||
def _minimal_fields(call_id: str | None = None, **overrides) -> dict[str, object]:
|
||||
"""一行最小遥测的**完整字段字典**(不写库),供 recorder 写入与旧版本进程模拟共用。
|
||||
|
||||
返回值不是顺手加的: 逐列断言若在测试里另抄一份期望值,抄错的那一列会以
|
||||
"库写错列位"的形态误报,而漏抄的列则悄悄不被验证。
|
||||
独立出来不是顺手拆的: "新旧进程混写"那条用例要以 1.3.4 的列集直接发 INSERT,
|
||||
若它另抄一份取值,抄错的那一列会以"库写错列位"的形态误报。
|
||||
|
||||
调用方随后逐列比对回读结果,故返回的就是实际提交的那一份。
|
||||
"""
|
||||
fields: dict[str, object] = {
|
||||
"call_id": call_id if call_id is not None else "c1",
|
||||
@@ -130,8 +153,30 @@ async def _record_minimal(
|
||||
"thinking_observation": "unknown",
|
||||
# 同理: `Effort` 归一成裸 str,不表态则是 None(与 'low' 必须分得开)
|
||||
"reasoning_effort": None,
|
||||
# —— 1.3.5 十列: 默认形态即"一次普通尝试行"(与单测 `_record_minimal` 同款)——
|
||||
"scope": "LLM",
|
||||
"operation": "chat",
|
||||
# 库内现场构造的请求没有上下文 → NULL,不造 ID
|
||||
"logical_call_id": None,
|
||||
"event_kind": "attempt",
|
||||
# 诊断四列只在失败的 attempt 行上非空;成功行不统一填 200
|
||||
"http_status_code": None,
|
||||
"error_type": None,
|
||||
"cause_type": None,
|
||||
"error_body": None,
|
||||
# 逻辑快照两列只属终态行
|
||||
"attempts": None,
|
||||
"total_latency_ms": None,
|
||||
}
|
||||
fields.update(overrides)
|
||||
return fields
|
||||
|
||||
|
||||
async def _record_minimal(
|
||||
recorder: PostgresRecorder, call_id: str | None = None, **overrides
|
||||
) -> dict[str, object]:
|
||||
"""记一行最小遥测,并**返回实际提交的字段**供调用方逐列比对回读结果。"""
|
||||
fields = _minimal_fields(call_id, **overrides)
|
||||
await recorder.record_llm_call(**fields)
|
||||
return fields
|
||||
|
||||
@@ -167,6 +212,17 @@ async def _fetch(dsn: str, sql: str, *args):
|
||||
await conn.close()
|
||||
|
||||
|
||||
async def _execute_args(dsn: str, sql: str, *args) -> None:
|
||||
"""带参数执行单条语句(扩展协议);用于模拟旧版本进程按旧列集发出的 INSERT。"""
|
||||
import asyncpg
|
||||
|
||||
conn = await asyncpg.connect(dsn, timeout=10)
|
||||
try:
|
||||
await conn.execute(sql, *args)
|
||||
finally:
|
||||
await conn.close()
|
||||
|
||||
|
||||
async def _execute_script(dsn: str, sql: str) -> None:
|
||||
"""整段执行多语句脚本(不带参数,走简单查询协议)——模拟下游把脚本贴进 psql。"""
|
||||
import asyncpg
|
||||
@@ -588,14 +644,24 @@ _PRE_TENANT_INSERT = (
|
||||
)
|
||||
|
||||
|
||||
# `_PRE_TENANT_DDL` 的物理列(23 个): 由 `_EXPECTED_COLUMNS` 去掉此后新增的四列
|
||||
# `_PRE_TENANT_DDL` 的物理列(23 个): 由 `_EXPECTED_COLUMNS` 去掉此后新增的列
|
||||
# 派生而非另抄一份——两份常量必然漂移,而漂移的表现是"manual 档没补列"这条断言假绿。
|
||||
# 去掉后的顺序与 DDL 逐字一致(这四列在 DDL 里本就排在末尾)。
|
||||
_PRE_TENANT_COLUMNS = [
|
||||
c
|
||||
for c in _EXPECTED_COLUMNS
|
||||
if c not in ("tenant_id", "meta", "thinking_observation", "reasoning_effort")
|
||||
]
|
||||
# 去掉后的顺序与 DDL 逐字一致(这些列在 DDL 里本就排在末尾)。
|
||||
_PRE_TENANT_ABSENT = (
|
||||
"tenant_id",
|
||||
"meta",
|
||||
"thinking_observation",
|
||||
"reasoning_effort",
|
||||
*_CALL_OBSERVABILITY_COLUMNS,
|
||||
)
|
||||
|
||||
_PRE_TENANT_COLUMNS = [c for c in _EXPECTED_COLUMNS if c not in _PRE_TENANT_ABSENT]
|
||||
|
||||
# 这张表缺的 14 个维度,按 `COLUMNS`(即告警的排列序)列出: manual 档告警逐字比对用。
|
||||
# 逐字而非前缀断言,是为了让"将来漏进告警的新列"当场红(设计 §4.2 的纪律)。
|
||||
_PRE_TENANT_MISSING_NOTICE = (
|
||||
f"以下维度不会被记录: {', '.join(c for c in COLUMNS if c in _PRE_TENANT_ABSENT)}。"
|
||||
)
|
||||
|
||||
# 回读要逐列比对的字段: 物理列去掉库从不显式写的 created_at,恰好 22 个
|
||||
_PRE_TENANT_WRITTEN_COLUMNS = [c for c in _PRE_TENANT_COLUMNS if c != "created_at"]
|
||||
@@ -703,7 +769,7 @@ class TestCallerDimensionsAcceptance:
|
||||
"WHERE table_schema = $1 AND table_name = 'llm_calls' ORDER BY ordinal_position",
|
||||
schema,
|
||||
)
|
||||
# 22 → 26 个 recorder 字段(加 created_at 共 27 个物理列),且新列追加在末尾
|
||||
# 22 → 36 个 recorder 字段(加 created_at 共 37 个物理列),且新列追加在末尾
|
||||
assert [r["column_name"] for r in cols] == _EXPECTED_COLUMNS
|
||||
rows = await _fetch(
|
||||
schema_dsn,
|
||||
@@ -908,11 +974,8 @@ class TestManualSchemaModeAcceptance:
|
||||
assert [m for m in captured_warnings if "补列失败" in m] == []
|
||||
notices = [m for m in captured_warnings if "auto_migrate=False" in m]
|
||||
assert len(notices) == 1 # 准备期一次讲清,不逐行刷屏
|
||||
# 逐字钉住四个维度: 前缀断言会让将来漏进告警的新列照样绿
|
||||
assert (
|
||||
"以下维度不会被记录: tenant_id, meta, thinking_observation, reasoning_effort。"
|
||||
in notices[0]
|
||||
)
|
||||
# 逐字钉住缺的每一个维度: 前缀断言会让将来漏进告警的新列照样绿
|
||||
assert _PRE_TENANT_MISSING_NOTICE in notices[0]
|
||||
finally:
|
||||
await recorder.aclose()
|
||||
|
||||
@@ -938,11 +1001,8 @@ class TestManualSchemaModeAcceptance:
|
||||
assert recorder.telemetry_status.degraded is False
|
||||
notices = [m for m in captured_warnings if "auto_migrate=False" in m]
|
||||
assert len(notices) == 1 # 准备期一次,第二行不再重复
|
||||
# 逐字钉住四个维度: 前缀断言会让将来漏进告警的新列照样绿
|
||||
assert (
|
||||
"以下维度不会被记录: tenant_id, meta, thinking_observation, reasoning_effort。"
|
||||
in notices[0]
|
||||
)
|
||||
# 逐字钉住缺的每一个维度: 前缀断言会让将来漏进告警的新列照样绿
|
||||
assert _PRE_TENANT_MISSING_NOTICE in notices[0]
|
||||
# 提示里的 SQL 必须可直接粘贴执行,而不是只报个列名
|
||||
assert (
|
||||
"ALTER TABLE llm_calls ADD COLUMN tenant_id TEXT NOT NULL DEFAULT '';" in notices[0]
|
||||
@@ -974,6 +1034,283 @@ class TestManualSchemaModeAcceptance:
|
||||
await recorder.aclose()
|
||||
|
||||
|
||||
# 1.3.5 之前(1.3.4 发布形态)的表: 26 个 recorder 字段 + created_at = 27 个物理列,
|
||||
# 没有本版新增的任何一列。裸表名: 由 `pg_sandbox` 在沙箱 schema 的 search_path 下执行。
|
||||
_PRE_135_DDL = """
|
||||
CREATE TABLE llm_calls (
|
||||
call_id TEXT PRIMARY KEY,
|
||||
parent_call_id TEXT,
|
||||
session_id TEXT,
|
||||
model TEXT NOT NULL,
|
||||
provider TEXT NOT NULL,
|
||||
source_name TEXT NOT NULL,
|
||||
messages TEXT NOT NULL,
|
||||
response TEXT NOT NULL,
|
||||
thinking TEXT NOT NULL DEFAULT '',
|
||||
prompt_tokens INTEGER NOT NULL,
|
||||
completion_tokens INTEGER NOT NULL,
|
||||
usage_source TEXT NOT NULL,
|
||||
latency_ms INTEGER NOT NULL,
|
||||
ttft_ms DOUBLE PRECISION,
|
||||
max_inter_token_ms DOUBLE PRECISION,
|
||||
cache_hit BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
error TEXT,
|
||||
cost DOUBLE PRECISION,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
cached_prompt_tokens INTEGER,
|
||||
model_reported TEXT,
|
||||
sampling TEXT,
|
||||
reasoning_tokens INTEGER,
|
||||
tenant_id TEXT NOT NULL DEFAULT '',
|
||||
meta JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
thinking_observation TEXT,
|
||||
reasoning_effort TEXT
|
||||
)
|
||||
"""
|
||||
|
||||
# 一行 1.3.5 之前写下的历史数据(只列 NOT NULL 列,与当年 recorder 的写入等价)。
|
||||
# 工厂的 `extra` 逐条裸执行、不接受查询参数,故 call_id 内联成字面量。
|
||||
_PRE_135_INSERT = (
|
||||
"INSERT INTO llm_calls (call_id, model, provider, source_name, messages, response, "
|
||||
"prompt_tokens, completion_tokens, usage_source, latency_ms) "
|
||||
"VALUES ('pre135', 'm', 'p', 's1', '[]', 'old body', 1, 2, 'measured', 10)"
|
||||
)
|
||||
|
||||
# 这张表缺的正是本版十列;manual 档告警要逐字比对的那句。
|
||||
_PRE_135_MISSING_NOTICE = f"以下维度不会被记录: {', '.join(_CALL_OBSERVABILITY_COLUMNS)}。"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def pre_135_schema(pg_sandbox) -> tuple[str, str]:
|
||||
"""一次性沙箱里造一张 **1.3.4 形态的 27 列表**,并留一行本版之前的历史数据。
|
||||
|
||||
共享表 `llm_calls` 一个字节都不碰: 本机那张表升级一次就再也回不到旧形态,
|
||||
指望它还是旧形态的测试第二次跑就会空转(与 `pre_tenant_schema` 同款理由)。
|
||||
"""
|
||||
sandbox = await pg_sandbox(ddl=_PRE_135_DDL, extra=(_PRE_135_INSERT,))
|
||||
return sandbox.dsn, sandbox.schema
|
||||
|
||||
|
||||
class TestCallObservabilityColumnsAcceptance:
|
||||
"""1.3.5(issue #19/#23)的 PG 存储兼容验收: auto 追加 / manual 裁剪 / 旧行 NULL / 混写。
|
||||
|
||||
单元层在 SQLite 上断的是同一族语义,这里断的是**真实 PG 上确实如此**——
|
||||
两端的 DDL、补列语句与列序是两份文本(`SQLITE_BACKFILL` 与 `_PG_BACKFILL_DECLS`),
|
||||
只有真表能证明它们没有分叉。
|
||||
"""
|
||||
|
||||
async def test_pre_135_table_gains_the_ten_columns_and_old_rows_stay_null(self, pre_135_schema):
|
||||
"""27 列旧表 auto 补齐到 37 列,新行两类取值读得回,**历史行十列一律 NULL**。
|
||||
|
||||
旧行不回填是本版的明示决策(设计 §5): NULL 表达的是"补列之前根本没记过
|
||||
这件事",与任何哨兵值都不是一回事。若哪天有人给这十列加了 DEFAULT,历史行
|
||||
会被就地改写成"记过且值为 X",归因查询从此分不清真实缺口——故这条断言的
|
||||
方向是 NULL,不是空串也不是 0。
|
||||
"""
|
||||
schema_dsn, schema = pre_135_schema
|
||||
recorder = _recorder(schema_dsn, auto_migrate=True)
|
||||
try:
|
||||
# attempt 行: 诊断四列非空、逻辑快照两列 NULL
|
||||
await _record_minimal(
|
||||
recorder,
|
||||
call_id="att",
|
||||
scope="LLM",
|
||||
operation="chat",
|
||||
logical_call_id="lcid-1",
|
||||
event_kind="attempt",
|
||||
http_status_code=503,
|
||||
error_type="TransientError",
|
||||
cause_type="ReadTimeout",
|
||||
error_body="upstream said 503",
|
||||
)
|
||||
# 终态行: 逻辑快照两列非空、诊断三列 NULL(不搬最后一次 attempt 的现场)
|
||||
await _record_minimal(
|
||||
recorder,
|
||||
call_id="term",
|
||||
scope="LLM",
|
||||
operation="chat",
|
||||
logical_call_id="lcid-1",
|
||||
event_kind="terminal_failure",
|
||||
error="AllSourcesExhausted: 全部源已耗尽",
|
||||
error_type="AllSourcesExhausted",
|
||||
attempts=3,
|
||||
total_latency_ms=4200,
|
||||
)
|
||||
cols = await _fetch(
|
||||
schema_dsn,
|
||||
"SELECT column_name FROM information_schema.columns "
|
||||
"WHERE table_schema = $1 AND table_name = 'llm_calls' ORDER BY ordinal_position",
|
||||
schema,
|
||||
)
|
||||
# 26 → 36 个 recorder 字段(加 created_at 共 37 个物理列),新列追加在末尾:
|
||||
# 列序与新建库一致才不会让两条升级路径分叉
|
||||
assert [r["column_name"] for r in cols] == _EXPECTED_COLUMNS
|
||||
|
||||
names = ", ".join(_CALL_OBSERVABILITY_COLUMNS)
|
||||
rows = await _fetch(
|
||||
schema_dsn,
|
||||
f"SELECT call_id, {names} FROM llm_calls "
|
||||
"WHERE call_id = ANY($1::text[]) ORDER BY call_id",
|
||||
["att", "pre135", "term"],
|
||||
)
|
||||
by_id = {r["call_id"]: r for r in rows}
|
||||
|
||||
assert dict(by_id["att"]) == {
|
||||
"call_id": "att",
|
||||
"scope": "LLM",
|
||||
"operation": "chat",
|
||||
"logical_call_id": "lcid-1",
|
||||
"event_kind": "attempt",
|
||||
"http_status_code": 503,
|
||||
"error_type": "TransientError",
|
||||
"cause_type": "ReadTimeout",
|
||||
"error_body": "upstream said 503",
|
||||
"attempts": None,
|
||||
"total_latency_ms": None,
|
||||
}
|
||||
assert dict(by_id["term"]) == {
|
||||
"call_id": "term",
|
||||
"scope": "LLM",
|
||||
"operation": "chat",
|
||||
"logical_call_id": "lcid-1",
|
||||
"event_kind": "terminal_failure",
|
||||
"http_status_code": None,
|
||||
"error_type": "AllSourcesExhausted",
|
||||
"cause_type": None,
|
||||
"error_body": None,
|
||||
"attempts": 3,
|
||||
"total_latency_ms": 4200,
|
||||
}
|
||||
# 历史行: 十列逐列 NULL(整体比对,漏掉任一列都红)
|
||||
assert dict(by_id["pre135"]) == {
|
||||
"call_id": "pre135",
|
||||
**dict.fromkeys(_CALL_OBSERVABILITY_COLUMNS),
|
||||
}
|
||||
finally:
|
||||
await recorder.aclose()
|
||||
|
||||
async def test_manual_trims_the_insert_on_a_pre_135_table(
|
||||
self, pre_135_schema, captured_warnings
|
||||
):
|
||||
"""27 列旧表 + manual: 一条 DDL 都不发,写入按现有列裁剪后照样落库。
|
||||
|
||||
与上一条恰成对照: 同一张表、同一份负载,只有 `auto_migrate` 不同,列数就必须是
|
||||
27 与 37 之别。裁剪是关掉 ALTER 的前提——不裁剪的话每行 INSERT 都撞缺列
|
||||
(SQLSTATE 42703)而被整行丢弃,那是把自动补列换成遥测静默全失。
|
||||
"""
|
||||
schema_dsn, schema = pre_135_schema
|
||||
recorder = _recorder(schema_dsn, auto_migrate=False)
|
||||
try:
|
||||
recorded = await _record_minimal(
|
||||
recorder, call_id="man135", scope="LLM", logical_call_id="lcid-x"
|
||||
)
|
||||
cols = await _fetch(
|
||||
schema_dsn,
|
||||
"SELECT column_name FROM information_schema.columns "
|
||||
"WHERE table_schema = $1 AND table_name = 'llm_calls' ORDER BY ordinal_position",
|
||||
schema,
|
||||
)
|
||||
# 表结构逐字不动: 既没多出十列,也没被顺手改了列序
|
||||
assert [r["column_name"] for r in cols] == _PRE_135_COLUMNS
|
||||
|
||||
names = ", ".join(_PRE_135_WRITTEN_COLUMNS)
|
||||
rows = await _fetch(
|
||||
schema_dsn, f"SELECT {names} FROM llm_calls WHERE call_id = $1", "man135"
|
||||
)
|
||||
assert len(rows) == 1 # 裁剪后的 INSERT 真写进去了,不是被 PG 拒收
|
||||
# 其余 26 列逐列与提交值相等: 少写十列最容易引发的错是剩下的值整体错位
|
||||
assert dict(rows[0]) == {c: recorded[c] for c in _PRE_135_WRITTEN_COLUMNS}
|
||||
|
||||
assert [m for m in captured_warnings if "写入失败" in m] == []
|
||||
assert [m for m in captured_warnings if "补列失败" in m] == []
|
||||
assert recorder.telemetry_status.degraded is False
|
||||
notices = [m for m in captured_warnings if "auto_migrate=False" in m]
|
||||
assert len(notices) == 1 # 准备期一次讲清,不逐行刷屏
|
||||
# 逐字钉住十个维度: 前缀断言会让将来漏进告警的新列照样绿
|
||||
assert _PRE_135_MISSING_NOTICE in notices[0]
|
||||
# 提示里的 SQL 必须可直接粘贴执行(首列与末列各验一条,含类型)
|
||||
assert "ALTER TABLE llm_calls ADD COLUMN scope TEXT;" in notices[0]
|
||||
assert "ALTER TABLE llm_calls ADD COLUMN total_latency_ms INTEGER;" in notices[0]
|
||||
finally:
|
||||
await recorder.aclose()
|
||||
|
||||
async def test_old_and_new_writers_share_one_table(self, fresh_schema, captured_warnings):
|
||||
"""滚动升级期的混写: 已补列的表上,旧版本进程按 26 列写、新版本按 36 列写。
|
||||
|
||||
这是升级窗口里必然出现的形态(先升一个 worker,其余仍是 1.3.4),而它的失败
|
||||
方式是静默的: 若新列带了 NOT NULL 或旧列集的 INSERT 被拒,旧 worker 的遥测
|
||||
会整段消失而只留逐行 warning。故这里既断三行都在、也断没有写入失败 warning。
|
||||
|
||||
旧进程用 `insert_sql("postgres", 旧列集)` 而不是另抄一条 SQL: 1.3.4 的
|
||||
recorder 发出的正是同一函数按当年列集拼出的语句,另抄一份只会各自漂移。
|
||||
"""
|
||||
fresh_dsn, schema = fresh_schema
|
||||
recorder = _recorder(fresh_dsn, auto_migrate=True)
|
||||
try:
|
||||
# 新版本进程: 建表(37 列)并写一条带完整新列的终态行
|
||||
await _record_minimal(
|
||||
recorder,
|
||||
call_id="new-1",
|
||||
scope="LLM",
|
||||
operation="chat",
|
||||
logical_call_id="lcid-mix",
|
||||
event_kind="terminal_failure",
|
||||
error_type="AllSourcesExhausted",
|
||||
attempts=2,
|
||||
total_latency_ms=1500,
|
||||
)
|
||||
|
||||
# 旧版本进程: 同一张表,按 1.3.4 的 26 列集写入
|
||||
legacy_fields = _minimal_fields("old-1", response="from a 1.3.4 worker")
|
||||
await _execute_args(
|
||||
fresh_dsn,
|
||||
insert_sql("postgres", _PRE_135_WRITTEN_COLUMNS),
|
||||
*(legacy_fields[c] for c in _PRE_135_WRITTEN_COLUMNS),
|
||||
)
|
||||
|
||||
# 新版本进程继续写: 旧进程的写入不得污染后续(列集是每进程各自探测的)
|
||||
await _record_minimal(
|
||||
recorder, call_id="new-2", scope="LLM", operation="embed", event_kind="attempt"
|
||||
)
|
||||
|
||||
cols = await _fetch(
|
||||
fresh_dsn,
|
||||
"SELECT column_name FROM information_schema.columns "
|
||||
"WHERE table_schema = $1 AND table_name = 'llm_calls' ORDER BY ordinal_position",
|
||||
schema,
|
||||
)
|
||||
assert [r["column_name"] for r in cols] == _EXPECTED_COLUMNS # 旧进程不改表结构
|
||||
|
||||
rows = await _fetch(
|
||||
fresh_dsn,
|
||||
"SELECT call_id, response, scope, operation, event_kind, attempts, "
|
||||
"total_latency_ms FROM llm_calls ORDER BY call_id",
|
||||
)
|
||||
assert [r["call_id"] for r in rows] == ["new-1", "new-2", "old-1"]
|
||||
by_id = {r["call_id"]: r for r in rows}
|
||||
assert by_id["old-1"]["response"] == "from a 1.3.4 worker"
|
||||
# 旧进程写下的行,新列一律 NULL——它没写,也不该被谁替它填
|
||||
assert (by_id["old-1"]["scope"], by_id["old-1"]["event_kind"]) == (None, None)
|
||||
assert (by_id["old-1"]["attempts"], by_id["old-1"]["total_latency_ms"]) == (None, None)
|
||||
assert (by_id["new-1"]["attempts"], by_id["new-1"]["total_latency_ms"]) == (2, 1500)
|
||||
assert by_id["new-2"]["operation"] == "embed"
|
||||
assert [m for m in captured_warnings if "写入失败" in m] == []
|
||||
|
||||
# 下游可见变化的机械化依据: 新口径"计失败调用"按 event_kind 过滤,
|
||||
# 混写期旧行(event_kind 为 NULL)既不会被误计成失败,也不会被误计成成功
|
||||
terminal = await _fetch(
|
||||
fresh_dsn,
|
||||
"SELECT count(*) AS n FROM llm_calls WHERE event_kind = 'terminal_failure'",
|
||||
)
|
||||
assert terminal[0]["n"] == 1
|
||||
unclassified = await _fetch(
|
||||
fresh_dsn, "SELECT count(*) AS n FROM llm_calls WHERE event_kind IS NULL"
|
||||
)
|
||||
assert unclassified[0]["n"] == 1
|
||||
finally:
|
||||
await recorder.aclose()
|
||||
|
||||
|
||||
_PHYSICAL_COLUMNS_SQL = (
|
||||
"SELECT column_name FROM information_schema.columns "
|
||||
"WHERE table_schema = $1 AND table_name = 'llm_calls' ORDER BY ordinal_position"
|
||||
@@ -1000,7 +1337,7 @@ class TestPublishedSchemaScript:
|
||||
|
||||
await _execute_script(fresh_dsn, script)
|
||||
actual = [r["column_name"] for r in await _fetch(fresh_dsn, _PHYSICAL_COLUMNS_SQL, schema)]
|
||||
# 物理列 = 26 个 INSERT 字段 + 库从不显式写的 created_at;对着库常量比,不另抄一份
|
||||
# 物理列 = 36 个 INSERT 字段 + 库从不显式写的 created_at;对着库常量比,不另抄一份
|
||||
assert set(actual) == set(COLUMNS) | {"created_at"}
|
||||
# 列序也不许漂: 新列必须排在 created_at 之后,否则新建库与 ALTER 升级的列序分叉
|
||||
assert actual == _EXPECTED_COLUMNS
|
||||
|
||||
@@ -72,10 +72,15 @@ class ScriptedTransport:
|
||||
def __init__(self, hang: bool = False):
|
||||
self.hang = hang
|
||||
self.calls: list[str] = []
|
||||
# 取消用例的确定性窗口(同 test_retry FakeTransport): 进入挂起即置位
|
||||
self.entered = asyncio.Event()
|
||||
|
||||
async def complete(self, *, messages, source, stream, overlay, call_id, reasoning_effort):
|
||||
async def complete(
|
||||
self, *, messages, source, stream, overlay, call_id, reasoning_effort, first_token_event
|
||||
):
|
||||
self.calls.append(source.name)
|
||||
if self.hang:
|
||||
self.entered.set()
|
||||
await asyncio.Event().wait()
|
||||
return TransportResult(
|
||||
content="ok",
|
||||
@@ -225,7 +230,32 @@ async def test_cancel_in_flight_releases_lease(clients):
|
||||
assert (await limiter.source_stats("s1")).inflight == 0
|
||||
|
||||
|
||||
# —— 掉线方向(fail-closed 集成证据)——
|
||||
async def test_cancel_in_flight_keeps_the_reservation(clients):
|
||||
"""1.3.6 §6.3 S3 在**真实 Redis** 上: 端口已开始、用量未知 → 保留 est 预扣。
|
||||
|
||||
内存后端与 Lua 后端的 `settle(delta)` 算术必须同口径——取消时凭空退款
|
||||
在分布式部署下就是几个 worker 一起击穿 TPM 闸。本用例不改 Lua、不改契约套件。
|
||||
"""
|
||||
a_cli, _ = clients
|
||||
scope = f"t{uuid4().hex[:8]}"
|
||||
sources = [make_source(max_concurrency=1, tpm=1000, est_tokens=400)]
|
||||
limiter = _limiter(a_cli, scope, sources, GlobalLimits(0, 0, 0))
|
||||
transport = ScriptedTransport(hang=True)
|
||||
client = _client(
|
||||
scope,
|
||||
sources,
|
||||
limiter,
|
||||
RedisGate(config=_CFG, redis=a_cli, scope=scope),
|
||||
transport,
|
||||
)
|
||||
task = asyncio.create_task(client.chat([{"role": "user", "content": "hi"}]))
|
||||
await transport.entered.wait()
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
stats = await limiter.source_stats("s1")
|
||||
assert stats.tpm_used == 400 # 预扣保留, 不回退到 0
|
||||
assert stats.inflight == 0
|
||||
|
||||
|
||||
async def test_redis_down_admission_fails_closed():
|
||||
|
||||
@@ -210,7 +210,9 @@ class ClockAdvancingTransport:
|
||||
self.clock = clock
|
||||
self.calls = []
|
||||
|
||||
async def complete(self, *, messages, source, stream, overlay, call_id, reasoning_effort):
|
||||
async def complete(
|
||||
self, *, messages, source, stream, overlay, call_id, reasoning_effort, first_token_event
|
||||
):
|
||||
self.calls.append((source.name, call_id))
|
||||
advance, action = self.script.pop(0)
|
||||
self.clock.advance(advance)
|
||||
|
||||
@@ -570,7 +570,7 @@ class TestTelemetryCapDoesNotPoisonTheCacheKey:
|
||||
before = build_cache_key("m", messages, "proj", None)
|
||||
|
||||
rec = self._Rows()
|
||||
await TelemetryEmitter(rec, text_cap=8).emit_attempt(
|
||||
await TelemetryEmitter(rec, text_cap=8, scope="LLM").emit_attempt(
|
||||
request=ChatRequest(messages=messages),
|
||||
source=SourceConfig(
|
||||
name="s1",
|
||||
@@ -585,6 +585,7 @@ class TestTelemetryCapDoesNotPoisonTheCacheKey:
|
||||
response=_resp(),
|
||||
error=None,
|
||||
reasoning_applies=True,
|
||||
operation="chat",
|
||||
)
|
||||
# 截断确实发生了(否则本用例恒真)
|
||||
logged = json.loads(rec.rows[0]["messages"])
|
||||
@@ -818,3 +819,75 @@ class TestExplicitCacheMigration:
|
||||
finally:
|
||||
for transport in transports:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
class TestCallStatsNotPoisoned:
|
||||
"""缓存不得回放历史统计(1.3.5 设计 §3)。
|
||||
|
||||
统计描述**本次**调用;把上次那条存进去再放出来,等于对调用方谎称这次
|
||||
重试了 N 次、耗了 M 毫秒。
|
||||
"""
|
||||
|
||||
async def test_serialized_payload_carries_no_call_stats_key(self):
|
||||
from polygateway.types import CallStats
|
||||
|
||||
backend = InMemoryCache()
|
||||
mw = _mw(backend)
|
||||
stats = CallStats(logical_call_id="lc-1", attempts=3, total_latency_ms=900)
|
||||
terminal = _Terminal(_resp(call_stats=stats))
|
||||
await mw(ChatRequest(messages=_MSGS), terminal)
|
||||
key = build_cache_key("m", _MSGS, "proj", None)
|
||||
stored = json.loads(await backend.get(key))
|
||||
assert "call_stats" not in stored # asdict 会把它摊成 dict,必须显式剔除
|
||||
|
||||
async def test_historic_dict_never_impersonates_call_stats(self):
|
||||
"""旧条目里的 `call_stats` dict 会被 `_RESPONSE_FIELDS` 放行,必须显式覆盖。
|
||||
|
||||
不覆盖就会有一个 dict 冒充 `CallStats` 从公共 API 漏给调用方,
|
||||
`resp.call_stats.attempts` 当场 `AttributeError`。
|
||||
"""
|
||||
backend = InMemoryCache()
|
||||
mw = _mw(backend)
|
||||
key = build_cache_key("m", _MSGS, "proj", None)
|
||||
poisoned = {
|
||||
"content": "legacy",
|
||||
"thinking": "",
|
||||
"model": "m",
|
||||
"provider": "p",
|
||||
"prompt_tokens": 1,
|
||||
"completion_tokens": 2,
|
||||
"latency_ms": 30,
|
||||
"ttft_ms": 5.0,
|
||||
"max_inter_token_ms": 2.0,
|
||||
"cache_hit": False,
|
||||
"call_id": "orig",
|
||||
"source_name": "s1",
|
||||
"usage_source": "measured",
|
||||
"call_stats": {
|
||||
"logical_call_id": "stale-lc",
|
||||
"attempts": 7,
|
||||
"total_latency_ms": 9999,
|
||||
},
|
||||
}
|
||||
await backend.set(key, json.dumps(poisoned), ttl_s=100)
|
||||
terminal = _Terminal(_resp())
|
||||
hit = await mw(ChatRequest(messages=_MSGS), terminal)
|
||||
assert hit.content == "legacy" and terminal.calls == 0 # 真的走了缓存
|
||||
assert hit.call_stats is None # dict 不得冒充 CallStats
|
||||
|
||||
async def test_cache_key_is_unchanged_by_the_new_field(self):
|
||||
"""新增内部字段不得扰动 key 公式,否则存量缓存全量冷启动(黄金值)。"""
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
class _Clock:
|
||||
def __call__(self):
|
||||
return 1000.0
|
||||
|
||||
ctx = _CallContext(now=_Clock())
|
||||
bare = build_cache_key("m", _MSGS, "proj", None)
|
||||
assert bare == build_cache_key("m", _MSGS, "proj", None)
|
||||
# 带上下文的请求与不带的请求必须落在同一个 key 上
|
||||
with_ctx = ChatRequest(messages=_MSGS, call_context=ctx)
|
||||
without = ChatRequest(messages=_MSGS)
|
||||
assert with_ctx.cache_namespace == without.cache_namespace
|
||||
assert digest_messages(with_ctx.messages) == digest_messages(without.messages)
|
||||
|
||||
+656
-1
@@ -1,8 +1,10 @@
|
||||
"""GatewayClient 装配与端到端(fake 后端 + MockTransport)测试(设计 §2.4)。"""
|
||||
|
||||
import asyncio
|
||||
import gc
|
||||
import json
|
||||
import sys
|
||||
import warnings
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
@@ -10,9 +12,12 @@ import pytest
|
||||
|
||||
from polygateway import (
|
||||
AllSourcesExhausted,
|
||||
CallDeadlineExceeded,
|
||||
GatewayClient,
|
||||
GatewaySettings,
|
||||
RequestRejectedError,
|
||||
ResultInvalidError,
|
||||
TransientError,
|
||||
gather_bounded,
|
||||
)
|
||||
from polygateway.backends.memory.breaker import InMemoryGate
|
||||
@@ -25,12 +30,17 @@ from polygateway.transports.openai_compat import OpenAICompatTransport
|
||||
from polygateway.types import (
|
||||
BackpressurePolicy,
|
||||
BreakerConfig,
|
||||
ChatRequest,
|
||||
Effort,
|
||||
GlobalLimits,
|
||||
RetryPolicy,
|
||||
SourceConfig,
|
||||
)
|
||||
|
||||
# 复用 RetryMW 那份可编程 fake transport(含确定性 `entered` 窗口),不再造第二份;
|
||||
# `tests/unit/test_backpressure.py:34` 已是同款复用
|
||||
from tests.unit.test_retry import FakeTransport, _ok
|
||||
|
||||
_REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
_ENV = {
|
||||
@@ -134,6 +144,92 @@ class TestChatEndToEnd:
|
||||
await client.chat([{"role": "user", "content": "hi"}], structured="json")
|
||||
|
||||
|
||||
class TestHedgeParams:
|
||||
"""对冲两个 keyword-only 参数的 client 入口校验(issue #24 H4;计划批次 H)。
|
||||
|
||||
编排行为本身见 tests/unit/test_hedge.py;这里只钉装配面: 入口即校、
|
||||
值域/交叉守卫与 settings 共用同一份 `check_hedge_assembly`。
|
||||
"""
|
||||
|
||||
def test_client_hedge_params_entry_validation(self):
|
||||
# 值域: 0/负/非有限当场 ValueError(不经 settings 那道守卫)
|
||||
with pytest.raises(ValueError, match=r"GatewayClient\(hedge_after_s"):
|
||||
_client(hedge_after_s=0)
|
||||
with pytest.raises(ValueError, match="hedge_max_extra"):
|
||||
_client(hedge_max_extra=0)
|
||||
with pytest.raises(ValueError, match="hedge_max_extra"):
|
||||
_client(hedge_max_extra=True) # bool 不是 int 档位
|
||||
# 交叉: 阈值 ≥ 源 timeout_s(10)= 对冲永不可能触发
|
||||
with pytest.raises(ValueError, match="timeout_s"):
|
||||
_client(hedge_after_s=99.0)
|
||||
# 合法值透传到 RetryMW(单源 scope 的装配 warning 是预期噪音,不断言)
|
||||
client = _client(hedge_after_s=0.05)
|
||||
assert client._hedge_after_s == 0.05
|
||||
assert client._terminal._hedge_after_s == 0.05
|
||||
# 未启用(缺省): 行为逐字等于 1.3.6
|
||||
assert _client()._terminal._hedge_after_s is None
|
||||
|
||||
|
||||
class TestGenerationMsClient:
|
||||
"""裸生成时间的 client 级口径(1.3.7 批次 C2/F)。
|
||||
|
||||
`_ScriptedGenClockTransport` 在每次 transport 调用内推进注入钟,
|
||||
使"时间花在哪"可断言(同 test_backpressure.ClockAdvancingTransport 范式)。
|
||||
"""
|
||||
|
||||
_MSG = [{"role": "user", "content": "hi"}]
|
||||
|
||||
async def test_generation_ms_structured_last_round_wins(self):
|
||||
"""结构化重问覆盖而非累加: 首轮坏 JSON 推进 1s,重问轮推进 0.25s → 250。
|
||||
|
||||
推进量取二进制可精确表示值: int 截断下非精确值会因浮点误差少 1ms。
|
||||
"""
|
||||
from pydantic import BaseModel
|
||||
|
||||
class Answer(BaseModel):
|
||||
answer: int
|
||||
|
||||
clock = _StatsClock()
|
||||
transport = _ScriptedGenClockTransport(
|
||||
[_ok("not json at all"), _ok('{"answer": 1}')], [1.0, 0.25], clock
|
||||
)
|
||||
async with _client(transport=transport, structured_max_retries=1, now=clock) as client:
|
||||
resp = await client.chat(self._MSG, structured=Answer)
|
||||
assert resp.content == '{"answer": 1}' and len(transport.calls) == 2
|
||||
assert resp.call_stats is not None
|
||||
assert resp.call_stats.attempts == 2
|
||||
assert resp.call_stats.generation_ms == 250
|
||||
|
||||
async def test_cache_hit_generation_ms_zero(self):
|
||||
"""缓存命中不产生 transport 调用: generation_ms 恒 0(0 是实测,非"未知")。"""
|
||||
client = _client(cache=InMemoryCache(), cache_namespace="proj", cache_ttl_s=3600)
|
||||
async with client:
|
||||
first = await client.chat(self._MSG)
|
||||
second = await client.chat(self._MSG)
|
||||
assert first.cache_hit is False and second.cache_hit is True
|
||||
assert second.call_stats is not None
|
||||
assert second.call_stats.generation_ms == 0
|
||||
assert second.call_stats.attempts == 0
|
||||
|
||||
|
||||
class _ScriptedGenClockTransport:
|
||||
"""脚本化假 transport: 每次成功调用在返回前按脚本推进注入钟(批次 C2/F)。"""
|
||||
|
||||
def __init__(self, results, advances, clock):
|
||||
self._results = list(results)
|
||||
self._advances = list(advances)
|
||||
self._clock = clock
|
||||
self.calls = []
|
||||
|
||||
async def complete(
|
||||
self, *, messages, source, stream, overlay, call_id, reasoning_effort, first_token_event
|
||||
):
|
||||
self.calls.append(call_id)
|
||||
result = self._results.pop(0)
|
||||
self._clock.advance(self._advances.pop(0))
|
||||
return result
|
||||
|
||||
|
||||
class TestSamplingOverlay:
|
||||
"""调用级采样参数入口(issue #4 Task 3)。"""
|
||||
|
||||
@@ -846,11 +942,19 @@ _CACHE_ENV = dict(
|
||||
|
||||
|
||||
class _Closable:
|
||||
"""记 close 次数的假组件;所有权纪律的唯一观测点。"""
|
||||
"""记 close 次数的假组件;所有权纪律的唯一观测点。
|
||||
|
||||
带 `record_llm_call(**fields)` 是因为它也被当作注入的 telemetry recorder 用:
|
||||
1.3.5 的装配闸在构造期就会拒掉不满足 `TelemetryRecorder` 的对象
|
||||
(否则下游升级后 100% 丢遥测而调用照常成功)。`**fields` 形态天然兼容。
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.closed = 0
|
||||
|
||||
async def record_llm_call(self, **fields):
|
||||
pass
|
||||
|
||||
async def aclose(self):
|
||||
self.closed += 1
|
||||
|
||||
@@ -861,6 +965,9 @@ class _SyncClosable:
|
||||
def __init__(self):
|
||||
self.closed = 0
|
||||
|
||||
async def record_llm_call(self, **fields):
|
||||
pass
|
||||
|
||||
def close(self):
|
||||
self.closed += 1
|
||||
|
||||
@@ -1362,3 +1469,551 @@ async def test_synthetic_runtime_protocol_and_legacy_call_signatures():
|
||||
assert client._transport._clients == {}
|
||||
finally:
|
||||
await client.aclose()
|
||||
|
||||
|
||||
class _StatsClock:
|
||||
"""确定性单调钟;测试主动推进以断言"哪些区段计入了总耗时"。"""
|
||||
|
||||
def __init__(self, start=1000.0):
|
||||
self.t = start
|
||||
|
||||
def __call__(self):
|
||||
return self.t
|
||||
|
||||
def advance(self, seconds):
|
||||
self.t += seconds
|
||||
|
||||
|
||||
class _TickingCache:
|
||||
"""假缓存后端: 每次 IO 推进注入钟。
|
||||
|
||||
不推进时钟的替身会让"缓存 IO 计入总耗时"的断言退化成恒等于 0 的空转绿
|
||||
(计划 §T1 替身构造要求)。
|
||||
"""
|
||||
|
||||
def __init__(self, clock, tick=0.25):
|
||||
self._clock = clock
|
||||
self._tick = tick
|
||||
self._data = {}
|
||||
|
||||
async def get(self, key):
|
||||
self._clock.advance(self._tick)
|
||||
return self._data.get(key)
|
||||
|
||||
async def set(self, key, value, ttl_s):
|
||||
self._clock.advance(self._tick)
|
||||
self._data[key] = value
|
||||
|
||||
|
||||
class _TickingRecorder:
|
||||
"""假 recorder: 写入时推进注入钟,用于断言内联遥测收尾计入总耗时。"""
|
||||
|
||||
def __init__(self, clock, tick=0.5):
|
||||
self._clock = clock
|
||||
self._tick = tick
|
||||
self.rows = []
|
||||
|
||||
async def record_llm_call(self, **fields):
|
||||
self._clock.advance(self._tick)
|
||||
self.rows.append(fields)
|
||||
|
||||
|
||||
class TestLogicalCallStats:
|
||||
"""一次公开 chat 调用的统计(1.3.5 设计 §3)。"""
|
||||
|
||||
_MSG = [{"role": "user", "content": "hi"}]
|
||||
|
||||
async def test_success_reports_one_attempt(self):
|
||||
async with _client() as client:
|
||||
resp = await client.chat(self._MSG)
|
||||
assert resp.call_stats is not None
|
||||
assert resp.call_stats.attempts == 1
|
||||
assert resp.call_stats.logical_call_id
|
||||
|
||||
async def test_concurrent_calls_do_not_share_counters_or_ids(self):
|
||||
"""同一 client 并发两路必须各自计数与各自 ID(库铁律「纯 asyncio 中立」)。
|
||||
|
||||
上下文若被提升成 client 实例属性,这条就会红——那正是 VT
|
||||
`evolve_llm = llm` 教训的同一形态。
|
||||
"""
|
||||
async with _client() as client:
|
||||
a, b = await asyncio.gather(client.chat(self._MSG), client.chat(self._MSG))
|
||||
assert a.call_stats.logical_call_id != b.call_stats.logical_call_id
|
||||
assert a.call_stats.attempts == b.call_stats.attempts == 1
|
||||
|
||||
async def test_cache_hit_is_zero_attempts_with_a_fresh_logical_id(self):
|
||||
"""命中不产生网关调用 → 0 尝试;且是**新**逻辑调用,不回放历史统计。"""
|
||||
clock = _StatsClock()
|
||||
cache = _TickingCache(clock)
|
||||
async with _client(
|
||||
cache=cache, cache_namespace="proj", cache_ttl_s=600, now=clock
|
||||
) as client:
|
||||
first = await client.chat(self._MSG)
|
||||
second = await client.chat(self._MSG)
|
||||
assert first.cache_hit is False and first.call_stats.attempts == 1
|
||||
assert second.cache_hit is True
|
||||
assert second.call_stats.attempts == 0
|
||||
assert second.call_stats.logical_call_id != first.call_stats.logical_call_id
|
||||
|
||||
async def test_cache_io_counts_into_total_latency(self):
|
||||
"""缓存读写是本次调用真实花掉的时间,必须进总耗时(设计 §3)。"""
|
||||
clock = _StatsClock()
|
||||
cache = _TickingCache(clock, tick=0.25)
|
||||
async with _client(
|
||||
cache=cache, cache_namespace="proj", cache_ttl_s=600, now=clock
|
||||
) as client:
|
||||
hit = (await client.chat(self._MSG), await client.chat(self._MSG))[1]
|
||||
# 命中路径只有一次 get(0.25s),无网关调用
|
||||
assert hit.call_stats.attempts == 0
|
||||
assert hit.call_stats.total_latency_ms == 250
|
||||
|
||||
async def test_inline_telemetry_teardown_counts_into_total_latency(self):
|
||||
"""成功响应的快照含返回前已完成的内联遥测耗时(设计 §6)。"""
|
||||
clock = _StatsClock()
|
||||
recorder = _TickingRecorder(clock, tick=0.5)
|
||||
async with _client(telemetry=recorder, now=clock) as client:
|
||||
resp = await client.chat(self._MSG)
|
||||
assert recorder.rows # 确实写了行,否则本断言空转
|
||||
assert resp.call_stats.total_latency_ms == 500
|
||||
|
||||
async def test_milliseconds_not_seconds(self):
|
||||
"""毫秒/秒不混用: 1.5s 必须是 1500 而不是 1 或 1.5。"""
|
||||
clock = _StatsClock()
|
||||
recorder = _TickingRecorder(clock, tick=1.5)
|
||||
async with _client(telemetry=recorder, now=clock) as client:
|
||||
resp = await client.chat(self._MSG)
|
||||
assert resp.call_stats.total_latency_ms == 1500
|
||||
|
||||
async def test_stats_work_without_any_telemetry(self):
|
||||
"""统计生效与否**不由 telemetry 是否启用决定**(设计 §3.5)。"""
|
||||
async with _client(telemetry=None) as client:
|
||||
resp = await client.chat(self._MSG)
|
||||
assert resp.call_stats is not None and resp.call_stats.attempts == 1
|
||||
|
||||
async def test_failure_exception_carries_no_stats_attribute(self):
|
||||
"""本版**不向异常对象附加统计**(设计 §3.1): 第三方可能复用同一异常实例。"""
|
||||
|
||||
def reject(request):
|
||||
return httpx.Response(400, json={"error": {"message": "bad"}})
|
||||
|
||||
async with _client(handler=reject) as client:
|
||||
with pytest.raises(RequestRejectedError) as exc:
|
||||
await client.chat(self._MSG)
|
||||
assert hasattr(exc.value, "call_stats") is False
|
||||
|
||||
async def test_input_validation_stays_outside_the_stats_boundary(self):
|
||||
"""校验异常保持原行为,发生在统计边界之外(设计 §3)。"""
|
||||
async with _client() as client:
|
||||
with pytest.raises(ValueError, match="meta"):
|
||||
await client.chat(self._MSG, meta={"BAD-KEY": 1})
|
||||
|
||||
|
||||
class TestChatTerminalFailureRows:
|
||||
"""chat 链路的**真实**终态行(1.3.5 设计 §6;补漏而非改口径)。
|
||||
|
||||
这些路径改前一条失败行都没有(结构化耗尽)或只有尝试行,
|
||||
"这次调用到底失败了几次"因此 SQL 答不出来。
|
||||
"""
|
||||
|
||||
_MSG = [{"role": "user", "content": "hi"}]
|
||||
|
||||
def _rows(self, recorder, kind):
|
||||
return [r for r in recorder.rows if r["event_kind"] == kind]
|
||||
|
||||
async def test_structured_exhaustion_writes_the_only_failure_row(self):
|
||||
"""结构化耗尽发生在 transport 成功之后: attempt 行全是成功行,终态是唯一记录。"""
|
||||
from pydantic import BaseModel
|
||||
|
||||
class Answer(BaseModel):
|
||||
answer: int
|
||||
|
||||
recorder = _MemoryRecorder()
|
||||
async with _client(
|
||||
handler=lambda request: _sse("not json at all"),
|
||||
telemetry=recorder,
|
||||
structured_max_retries=1,
|
||||
) as client:
|
||||
with pytest.raises(ResultInvalidError):
|
||||
await client.chat(self._MSG, structured=Answer)
|
||||
attempts = self._rows(recorder, "attempt")
|
||||
terminals = self._rows(recorder, "terminal_failure")
|
||||
assert [a["error"] for a in attempts] == [None, None] # 两次尝试都成功
|
||||
assert len(terminals) == 1
|
||||
row = terminals[0]
|
||||
assert row["error_type"] == "ResultInvalidError"
|
||||
# C2: 有界结构化说明并入 error,且不含 raw_text(正文预算已由 attempt 行承担)
|
||||
assert "validation=" in row["error"] or "repair=" in row["error"]
|
||||
assert len(row["error"]) < 1200
|
||||
|
||||
async def test_retry_exhaustion_writes_exactly_one_terminal_row(self):
|
||||
"""重试耗尽: 逐次 attempt 行之外只能有**一条**终态行。
|
||||
|
||||
终态行的 `attempts` 是整次逻辑调用的真实尝试数——这正是改前 SQL
|
||||
答不出的"这次调用到底重试了几次"。
|
||||
"""
|
||||
recorder = _MemoryRecorder()
|
||||
async with _client(
|
||||
handler=lambda request: httpx.Response(503),
|
||||
telemetry=recorder,
|
||||
retry=RetryPolicy(3, 0.001, 0.01),
|
||||
) as client:
|
||||
with pytest.raises(AllSourcesExhausted):
|
||||
await client.chat(self._MSG)
|
||||
attempts = self._rows(recorder, "attempt")
|
||||
terminals = self._rows(recorder, "terminal_failure")
|
||||
assert len(attempts) == 3
|
||||
assert len(terminals) == 1
|
||||
row = terminals[0]
|
||||
assert row["error_type"] == "AllSourcesExhausted"
|
||||
assert row["attempts"] == 3 # 整次逻辑调用的尝试数
|
||||
assert row["scope"] == "llm" and row["operation"] == "chat"
|
||||
# 终态行的 latency_ms 与 total_latency_ms 同取一份冻结快照
|
||||
assert row["latency_ms"] == row["total_latency_ms"]
|
||||
|
||||
async def test_request_rejected_now_has_both_an_attempt_and_a_terminal_row(self):
|
||||
"""已批准的下游可见变化: 400 密集负载的错误行翻倍,失败计数只能取终态。"""
|
||||
recorder = _MemoryRecorder()
|
||||
async with _client(
|
||||
handler=lambda request: httpx.Response(400, json={"error": {"message": "bad"}}),
|
||||
telemetry=recorder,
|
||||
) as client:
|
||||
with pytest.raises(RequestRejectedError):
|
||||
await client.chat(self._MSG)
|
||||
attempts = self._rows(recorder, "attempt")
|
||||
terminals = self._rows(recorder, "terminal_failure")
|
||||
assert len(attempts) == 1 and attempts[0]["http_status_code"] == 400
|
||||
assert len(terminals) == 1
|
||||
# C1: 终态不搬运最后一次 attempt 的状态码与正文
|
||||
assert terminals[0]["http_status_code"] is None
|
||||
assert terminals[0]["error_body"] is None
|
||||
# 两类行共享同一 logical_call_id,归因查询才连得起来
|
||||
assert attempts[0]["logical_call_id"] == terminals[0]["logical_call_id"]
|
||||
|
||||
async def test_non_domain_exception_writes_no_terminal_row(self):
|
||||
"""编程错原样传播,本版**不承诺**任何统计或终态行,也不偷偷改分类。"""
|
||||
recorder = _MemoryRecorder()
|
||||
|
||||
async def boom(request):
|
||||
raise KeyError("programming error")
|
||||
|
||||
async with _client(telemetry=recorder) as client:
|
||||
client._handler = boom
|
||||
with pytest.raises(KeyError):
|
||||
await client.chat(self._MSG)
|
||||
assert self._rows(recorder, "terminal_failure") == []
|
||||
|
||||
async def test_cancellation_writes_at_most_one_terminal_row(self):
|
||||
"""取消尽力写一条(允许 0),且 `CancelledError` 类型与语义不变。"""
|
||||
recorder = _MemoryRecorder()
|
||||
|
||||
async def hang(request):
|
||||
raise asyncio.CancelledError
|
||||
|
||||
async with _client(telemetry=recorder) as client:
|
||||
client._handler = hang
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await client.chat(self._MSG)
|
||||
terminals = self._rows(recorder, "terminal_failure")
|
||||
assert len(terminals) == 1
|
||||
assert terminals[0]["error"] == "cancelled"
|
||||
# 字符串不解析猜诊断
|
||||
assert terminals[0]["error_type"] is None
|
||||
|
||||
async def test_terminal_row_is_written_once_per_logical_call(self):
|
||||
"""`claim_terminal` 去重: 同一次调用即便出口被多次触达也只有一条。"""
|
||||
from polygateway.middleware.telemetry import emit_terminal_once
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
recorder = _MemoryRecorder()
|
||||
async with _client(telemetry=recorder) as client:
|
||||
context = _CallContext(now=client._now)
|
||||
request = ChatRequest(messages=self._MSG, call_context=context)
|
||||
for _ in range(3):
|
||||
await emit_terminal_once(
|
||||
client._emitter,
|
||||
request=request,
|
||||
context=context,
|
||||
error=AllSourcesExhausted(
|
||||
scope="llm", reason="retry_exhausted", retry_after_s=1.0
|
||||
),
|
||||
operation="chat",
|
||||
)
|
||||
assert len(self._rows(recorder, "terminal_failure")) == 1
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def captured_warnings():
|
||||
"""捕获库发出的 WARNING;loguru 不经标准 logging,pytest 的 caplog 抓不到。
|
||||
|
||||
名字避开裸 `warnings`: 那会遮蔽标准库模块名,本文件将来任何一次
|
||||
`import warnings` 都会与它静默互相顶掉,而报错点离真因很远。
|
||||
"""
|
||||
from loguru import logger
|
||||
|
||||
messages: list[str] = []
|
||||
sink_id = logger.add(messages.append, level="WARNING")
|
||||
yield messages
|
||||
logger.remove(sink_id)
|
||||
|
||||
|
||||
class TestTerminalEmitDegradation:
|
||||
"""终态出口的降级方向: 诊断字段的**提取**同样在降级范围内(铁律"遥测写失败降级不冒泡")。
|
||||
|
||||
改前 `_record` 的 except 只包住写入本身,而错误诊断列在它之外求值——
|
||||
经公共扩展点(自实现 `StructuredOutputStrategy`/transport)构造出的
|
||||
`ResultInvalidError(validation_errors=(非 str,))` 会让提取期抛 `TypeError`
|
||||
顶替调用方本该收到的领域异常,错误四分类当场被击穿。
|
||||
"""
|
||||
|
||||
_MSG = [{"role": "user", "content": "hi"}]
|
||||
|
||||
def _rows(self, recorder, kind):
|
||||
return [r for r in recorder.rows if r["event_kind"] == kind]
|
||||
|
||||
async def test_broken_validation_errors_keep_the_domain_error(self, captured_warnings):
|
||||
"""自实现策略给出非 str 的 `validation_errors`: 领域异常必须原样上抛。"""
|
||||
|
||||
class _BadStrategy:
|
||||
"""公共端口 `StructuredOutputStrategy` 的下游实现(库外没有类型执法)。"""
|
||||
|
||||
def request_overlay(self, schema):
|
||||
return {}
|
||||
|
||||
def parse(self, text):
|
||||
raise ResultInvalidError("模型返回不可解析", validation_errors=(object(),))
|
||||
|
||||
recorder = _MemoryRecorder()
|
||||
async with _client(telemetry=recorder, structured_strategy=_BadStrategy()) as client:
|
||||
with pytest.raises(ResultInvalidError): # 不是 TypeError
|
||||
await client.chat(self._MSG, structured="json")
|
||||
# 降级有声: 静默吞掉等于遥测缺口无人知道
|
||||
assert [m for m in captured_warnings if "终态遥测记录失败" in m]
|
||||
# 尝试行不受影响;终态行按 best effort 允许 0 条,但绝不能重复
|
||||
assert len(self._rows(recorder, "attempt")) == 1
|
||||
assert len(self._rows(recorder, "terminal_failure")) <= 1
|
||||
|
||||
async def test_cancellation_is_never_swallowed_by_the_degradation(self):
|
||||
"""降级不得吞取消: 写入那一次 await 上被取消,`CancelledError` 照常传播。"""
|
||||
from polygateway.middleware.telemetry import emit_terminal_once
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
class _CancellingEmitter:
|
||||
async def emit_terminal_failure(self, **kwargs):
|
||||
raise asyncio.CancelledError
|
||||
|
||||
context = _CallContext(now=asyncio.get_running_loop().time)
|
||||
request = ChatRequest(messages=self._MSG, call_context=context)
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await emit_terminal_once(
|
||||
_CancellingEmitter(),
|
||||
request=request,
|
||||
context=context,
|
||||
error=AllSourcesExhausted(scope="llm", reason="retry_exhausted", retry_after_s=1.0),
|
||||
operation="chat",
|
||||
)
|
||||
|
||||
|
||||
class _ClockJumpTransport:
|
||||
"""假 transport: 只推进**注入钟**,真实墙钟几乎不走。
|
||||
|
||||
用于把"期限读哪只钟"与"统计读哪只钟"两件事分开断言。
|
||||
"""
|
||||
|
||||
def __init__(self, clock, *, jump):
|
||||
self._clock = clock
|
||||
self._jump = jump
|
||||
self.calls = []
|
||||
|
||||
async def complete(
|
||||
self, *, messages, source, stream, overlay, call_id, reasoning_effort, first_token_event
|
||||
):
|
||||
self.calls.append(call_id)
|
||||
self._clock.advance(self._jump)
|
||||
return _ok()
|
||||
|
||||
|
||||
class _SlowRecorder:
|
||||
"""假 recorder: 每写一行真实等待一段,用于量化"清理不被期限截断"。"""
|
||||
|
||||
def __init__(self, delay=0.15):
|
||||
self._delay = delay
|
||||
self.rows = []
|
||||
|
||||
async def record_llm_call(self, **fields):
|
||||
await asyncio.sleep(self._delay)
|
||||
self.rows.append(fields)
|
||||
|
||||
|
||||
class _SlowSetCache:
|
||||
"""假缓存后端: `set` 慢于期限,用于构造"已产出、已计费的成功被丢弃"。"""
|
||||
|
||||
def __init__(self, delay=0.5):
|
||||
self._delay = delay
|
||||
self.data = {}
|
||||
self.sets = 0
|
||||
|
||||
async def get(self, key):
|
||||
return self.data.get(key)
|
||||
|
||||
async def set(self, key, value, ttl_s):
|
||||
self.sets += 1
|
||||
await asyncio.sleep(self._delay)
|
||||
self.data[key] = value
|
||||
|
||||
|
||||
class TestChatCallDeadline:
|
||||
"""chat 链路的期限覆盖面与到期代价(计划 §5 批次 D/D2)。
|
||||
|
||||
真实事件循环时钟: 期限 0.05s 对被治理的等待(退避 5s、轮询 300s、慢 IO 0.5s)
|
||||
有 10 倍以上余量,故不标 slow。
|
||||
"""
|
||||
|
||||
_MSG = [{"role": "user", "content": "hi"}]
|
||||
_DEADLINE = 0.05
|
||||
|
||||
def _rows(self, recorder, kind):
|
||||
return [r for r in recorder.rows if r["event_kind"] == kind]
|
||||
|
||||
# —— 批次 D: 期限落点覆盖面 ——
|
||||
|
||||
async def test_deadline_fires_during_backoff_sleep(self):
|
||||
"""退避 sleep 是等待的大头(429 序列可睡到小时级),期限必须能在它中间落地。"""
|
||||
transport = FakeTransport([TransientError("boom", operation="chat"), _ok()])
|
||||
async with _client(transport=transport, retry=RetryPolicy(3, 5.0, 30.0)) as client:
|
||||
with pytest.raises(CallDeadlineExceeded) as exc:
|
||||
await client.chat(self._MSG, call_deadline_s=self._DEADLINE)
|
||||
assert exc.value.scope == "llm" and exc.value.deadline_s == self._DEADLINE
|
||||
# 第二次尝试还压在 5s 退避里,期限确实落在 sleep 上而非 transport 上
|
||||
assert len(transport.calls) == 1
|
||||
|
||||
async def test_deadline_fires_while_queued_for_quota(self):
|
||||
"""准入排队(配额满轮询)是第二类长等待: 一次 transport 都没打出去也要能到期。"""
|
||||
source = _source(tpm=1, est_tokens=1000) # 预扣量恒超本源 TPM → 六闸永不放行
|
||||
limiter = InMemoryLimiter(
|
||||
scope="llm", sources={source.name: source}, global_limits=GlobalLimits(0, 0, 0)
|
||||
)
|
||||
transport = FakeTransport([_ok()])
|
||||
async with _client([source], transport=transport, limiter=limiter) as client:
|
||||
with pytest.raises(CallDeadlineExceeded):
|
||||
await client.chat(self._MSG, call_deadline_s=self._DEADLINE)
|
||||
assert transport.calls == [] # 期限落在轮询里,尝试从未开始
|
||||
|
||||
async def test_deadline_fires_during_structured_re_ask(self):
|
||||
"""结构化重问共享同一份期限: 阶梯不得按轮数各起一份,否则期限被放大 N 倍。
|
||||
|
||||
窗口用"第三轮挂起"构造而非 sleep 猜时长——前两轮瞬时返回坏 JSON,
|
||||
期限只可能落在第三轮上,断言因此与机器负载无关。
|
||||
"""
|
||||
from pydantic import BaseModel
|
||||
|
||||
class Answer(BaseModel):
|
||||
answer: int
|
||||
|
||||
transport = FakeTransport([_ok("not json at all"), _ok("not json at all"), "hang"])
|
||||
async with _client(
|
||||
transport=transport, structured_max_retries=5, structured_strategy=JsonRepairStrategy()
|
||||
) as client:
|
||||
with pytest.raises(CallDeadlineExceeded):
|
||||
await client.chat(self._MSG, structured=Answer, call_deadline_s=0.05)
|
||||
# 到期发生在第三轮: 期限确实跨过了两次重问,而不是在首轮就截断
|
||||
assert len(transport.calls) == 3
|
||||
|
||||
# —— 批次 D2: 到期代价 ——
|
||||
|
||||
async def test_expiry_writes_one_terminal_row_and_a_cancelled_attempt(self):
|
||||
"""到期恰好一条终态行 + 被取消的 attempt 行,两行同一 logical_call_id。"""
|
||||
recorder = _MemoryRecorder()
|
||||
transport = FakeTransport(["hang"])
|
||||
async with _client(transport=transport, telemetry=recorder) as client:
|
||||
with pytest.raises(CallDeadlineExceeded):
|
||||
await client.chat(self._MSG, call_deadline_s=self._DEADLINE)
|
||||
attempts = self._rows(recorder, "attempt")
|
||||
terminals = self._rows(recorder, "terminal_failure")
|
||||
assert len(terminals) == 1
|
||||
assert terminals[0]["error_type"] == "CallDeadlineExceeded"
|
||||
assert len(attempts) == 1 and attempts[0]["error"] == "cancelled"
|
||||
assert attempts[0]["logical_call_id"] == terminals[0]["logical_call_id"]
|
||||
|
||||
# 零新增遥测列: 期限终态行的列集合与既有失败路径的终态行逐字相同
|
||||
baseline = _MemoryRecorder()
|
||||
async with _client(
|
||||
handler=lambda request: httpx.Response(400, json={"error": {"message": "bad"}}),
|
||||
telemetry=baseline,
|
||||
) as client:
|
||||
with pytest.raises(RequestRejectedError):
|
||||
await client.chat(self._MSG)
|
||||
assert set(terminals[0]) == set(self._rows(baseline, "terminal_failure")[0])
|
||||
|
||||
async def test_cleanup_is_not_cut_short_by_the_expiry(self):
|
||||
"""返回时刻 = 期限 + 清理耗时: 只断下界(> 期限 × 2),不断上界。"""
|
||||
recorder = _SlowRecorder(delay=0.15) # attempt 行与终态行各付一次
|
||||
transport = FakeTransport(["hang"])
|
||||
loop = asyncio.get_running_loop()
|
||||
started = loop.time()
|
||||
async with _client(transport=transport, telemetry=recorder) as client:
|
||||
with pytest.raises(CallDeadlineExceeded):
|
||||
await client.chat(self._MSG, call_deadline_s=self._DEADLINE)
|
||||
elapsed = loop.time() - started
|
||||
assert len(recorder.rows) == 2 # 清理照常写完两行,没被期限截断
|
||||
assert elapsed > self._DEADLINE * 2, f"清理疑似被截断: {elapsed}s"
|
||||
|
||||
async def test_expiry_discards_a_success_that_was_already_billed(self):
|
||||
"""到期 ≠ 未产出、未计费: transport 已成功一次,结果仍被丢弃。"""
|
||||
transport = FakeTransport([_ok()])
|
||||
cache = _SlowSetCache(delay=0.5) # 写缓存慢于期限 → 到期落在成功之后
|
||||
async with _client(
|
||||
transport=transport, cache=cache, cache_namespace="proj", cache_ttl_s=600
|
||||
) as client:
|
||||
with pytest.raises(CallDeadlineExceeded):
|
||||
await client.chat(self._MSG, call_deadline_s=self._DEADLINE)
|
||||
assert len(transport.calls) == 1 # 上游已经产出并计费
|
||||
assert cache.sets == 1 and cache.data == {} # 结果既没回给调用方也没落缓存
|
||||
|
||||
# —— 批次 E: 注入钟与期限正交 ——
|
||||
|
||||
async def test_injected_clock_jump_does_not_trigger_the_deadline(self):
|
||||
"""期限只认真实墙钟: 注入钟跳 10^6 秒也不该凭空到期(不换算绝对截止时刻)。"""
|
||||
clock = _StatsClock()
|
||||
transport = _ClockJumpTransport(clock, jump=1_000_000.0)
|
||||
async with _client(transport=transport, now=clock) as client:
|
||||
resp = await client.chat(self._MSG, call_deadline_s=5.0)
|
||||
assert resp.content == "ok"
|
||||
# 而统计仍逐字读注入钟(10^6 s = 10^9 ms),两只钟各司其职
|
||||
assert resp.call_stats is not None
|
||||
assert resp.call_stats.total_latency_ms == 1_000_000_000
|
||||
|
||||
async def test_expiry_latency_still_reads_the_injected_clock(self):
|
||||
"""期限由真实钟触发,终态行的耗时仍取自注入钟(真实耗时只有几十毫秒)。"""
|
||||
clock = _StatsClock()
|
||||
recorder = _TickingRecorder(clock, tick=0.5)
|
||||
transport = FakeTransport(["hang"])
|
||||
async with _client(transport=transport, telemetry=recorder, now=clock) as client:
|
||||
with pytest.raises(CallDeadlineExceeded):
|
||||
await client.chat(self._MSG, call_deadline_s=self._DEADLINE)
|
||||
terminal = [r for r in recorder.rows if r["event_kind"] == "terminal_failure"][0]
|
||||
assert terminal["total_latency_ms"] == 500 # attempt 行那一次 tick,不是真实的 ~50ms
|
||||
|
||||
|
||||
class TestChatCallDeadlineEntryGuards:
|
||||
"""per-call 入口校验的两条硬红线(计划 §3.4/§5 批次 E)。"""
|
||||
|
||||
_MSG = [{"role": "user", "content": "hi"}]
|
||||
|
||||
async def test_illegal_per_call_value_leaves_no_un_awaited_coroutine(self):
|
||||
"""校验先于构造 awaitable: 否则非法值抛错时遗留未 await 的协程(资源不释放)。"""
|
||||
transport = FakeTransport([_ok()])
|
||||
async with _client(transport=transport) as client:
|
||||
with warnings.catch_warnings(record=True) as caught:
|
||||
warnings.simplefilter("always")
|
||||
with pytest.raises(ValueError, match=r"chat\(call_deadline_s"):
|
||||
await client.chat(self._MSG, call_deadline_s=0)
|
||||
gc.collect() # 未 await 的协程在回收时才发 RuntimeWarning
|
||||
assert [w for w in caught if "never awaited" in str(w.message)] == []
|
||||
assert transport.calls == []
|
||||
|
||||
async def test_per_call_none_inherits_the_assembled_deadline(self):
|
||||
"""`None` = 继承装配值(不提供"本次关闭"): 装配了期限就照样到期。"""
|
||||
transport = FakeTransport(["hang"])
|
||||
async with _client(transport=transport, call_deadline_s=0.05) as client:
|
||||
with pytest.raises(CallDeadlineExceeded):
|
||||
await client.chat(self._MSG)
|
||||
|
||||
@@ -1041,3 +1041,245 @@ def test_live_unknown_wire_assembly_is_local_only():
|
||||
GatewayClient.from_settings(
|
||||
dataclasses.replace(settings, sources=(source,)), registry=register_provider(mystery)
|
||||
)
|
||||
|
||||
|
||||
class TestCallDeadlineConfig:
|
||||
"""`{SCOPE}__CALL_DEADLINE_S` 与三个 client 入口参数的值域四条路(issue #22)。"""
|
||||
|
||||
def test_key_unset_means_disabled(self):
|
||||
assert GatewaySettings.from_env("LLM", env=_env()).call_deadline_s is None
|
||||
|
||||
def test_env_key_parsed(self):
|
||||
s = GatewaySettings.from_env("LLM", env=_env(**{"LLM__CALL_DEADLINE_S": "30"}))
|
||||
assert s.call_deadline_s == 30.0
|
||||
|
||||
@pytest.mark.parametrize("bad", ["0", "-1", "nan", "inf", "abc"])
|
||||
def test_env_illegal_value_reports_the_actual_key_name(self, bad):
|
||||
"""origin 必须是实际命中的 env 键名,多 scope 部署里才定位得到是哪个键。"""
|
||||
with pytest.raises(ValueError, match="LLM__CALL_DEADLINE_S"):
|
||||
GatewaySettings.from_env("LLM", env=_env(**{"LLM__CALL_DEADLINE_S": bad}))
|
||||
|
||||
def test_direct_construction_is_guarded(self):
|
||||
base = GatewaySettings.from_env("LLM", env=_env())
|
||||
with pytest.raises(ValueError, match="GatewaySettings.call_deadline_s"):
|
||||
dataclasses.replace(base, call_deadline_s=0)
|
||||
|
||||
def test_plain_constructor_call_is_guarded_too(self):
|
||||
"""`dataclasses.replace` 与直接构造是两条路: 守卫在 `__post_init__` 才两条都盖住。
|
||||
|
||||
只在 `from_env` 里校验的话,直接 `GatewaySettings(...)` 装配的下游(测试/高级
|
||||
注入路径,CLAUDE.md §4.5 的第二条装配路)会把非法期限一路带到第一次调用才炸。
|
||||
"""
|
||||
base = GatewaySettings.from_env("LLM", env=_env())
|
||||
fields = {f.name: getattr(base, f.name) for f in dataclasses.fields(base)}
|
||||
with pytest.raises(ValueError, match="GatewaySettings.call_deadline_s"):
|
||||
GatewaySettings(**{**fields, "call_deadline_s": float("inf")})
|
||||
# 合法值走同一条路不受影响(守卫对合法值是幂等空操作)
|
||||
assert GatewaySettings(**{**fields, "call_deadline_s": 7}).call_deadline_s == 7.0
|
||||
|
||||
def test_replace_with_legal_value_is_idempotent(self):
|
||||
base = GatewaySettings.from_env("LLM", env=_env())
|
||||
assert dataclasses.replace(base, call_deadline_s=5).call_deadline_s == 5.0
|
||||
|
||||
def test_deadline_shorter_than_timeout_is_legal(self):
|
||||
"""期限短于单次 timeout_s 是调用方的合法选择,不做跨字段耦合校验。"""
|
||||
s = GatewaySettings.from_env("LLM", env=_env(**{"LLM__CALL_DEADLINE_S": "1"}))
|
||||
assert s.call_deadline_s == 1.0 and s.sources[0].timeout_s == 120.0
|
||||
|
||||
def test_from_settings_propagates_to_client(self):
|
||||
s = GatewaySettings.from_env("LLM", env=_env(**{"LLM__CALL_DEADLINE_S": "12"}))
|
||||
assert GatewayClient.from_settings(s)._call_deadline_s == 12.0
|
||||
|
||||
def test_client_init_validates_at_entry(self):
|
||||
"""三个 client 的 `__init__` 直传非法值也当场报错(不经 settings 那道守卫)。"""
|
||||
from tests.unit.test_client import _client
|
||||
|
||||
with pytest.raises(ValueError, match=r"GatewayClient\(call_deadline_s"):
|
||||
_client(call_deadline_s=0)
|
||||
|
||||
|
||||
class TestHedgeConfig:
|
||||
"""`{SCOPE}__HEDGE__AFTER_S`/`{SCOPE}__HEDGE__MAX_EXTRA` 两键与装配守卫(issue #24 H4)。
|
||||
|
||||
对冲默认关闭: 键未设 = None/1,行为逐字等于 1.3.6。守卫四路覆盖
|
||||
(env/直接构造/dataclasses.replace/client 直传),单一定义点是
|
||||
`config.check_hedge_assembly`。
|
||||
"""
|
||||
|
||||
def _two_source_env(self, **overrides):
|
||||
"""双源 env(同 provider 避免注册表依赖): 隔离单源 warning 的干扰。"""
|
||||
return _env(
|
||||
**{
|
||||
"LLM__QWEN__2__BASE_URL": "https://gw-b.example/v1",
|
||||
"LLM__QWEN__2__API_KEY": "sk-b",
|
||||
"LLM__QWEN__2__MODEL": "qwen-plus",
|
||||
"LLM__QWEN__2__TIMEOUT_S": "90",
|
||||
**overrides,
|
||||
}
|
||||
)
|
||||
|
||||
def test_hedge_keys_from_env_skip_source_loader(self):
|
||||
"""两键为 3 段键,天然不被 `_load_sources` 当源字段;`HEDGE` 进保留段防 4 段撞名。"""
|
||||
env = self._two_source_env(**{"LLM__HEDGE__AFTER_S": "8", "LLM__HEDGE__MAX_EXTRA": "2"})
|
||||
with _captured_warnings(): # max_extra>1 的 v1 单路 warning, 本用例不断言它
|
||||
s = GatewaySettings.from_env("LLM", env=env)
|
||||
assert s.hedge_after_s == 8.0
|
||||
assert s.hedge_max_extra == 2
|
||||
assert {src.name for src in s.sources} == {"qwen_1", "qwen_2"} # HEDGE 键未造源
|
||||
# `HEDGE` 在保留段: `LLM__HEDGE__1__*` 四段键不得造出一个名为 hedge_1 的源
|
||||
env_collision = _env(
|
||||
**{
|
||||
"LLM__HEDGE__1__BASE_URL": "https://gw-c.example/v1",
|
||||
"LLM__HEDGE__1__API_KEY": "sk-c",
|
||||
"LLM__HEDGE__1__MODEL": "m-c",
|
||||
"LLM__HEDGE__1__TIMEOUT_S": "60",
|
||||
}
|
||||
)
|
||||
s2 = GatewaySettings.from_env("LLM", env=env_collision)
|
||||
assert [src.name for src in s2.sources] == ["qwen_1"]
|
||||
|
||||
def test_hedge_keys_unset_mean_disabled(self):
|
||||
"""默认关闭: 两键未设 = None/1,且不产生任何 warning。"""
|
||||
with _captured_warnings() as warnings:
|
||||
s = GatewaySettings.from_env("LLM", env=_env())
|
||||
assert s.hedge_after_s is None and s.hedge_max_extra == 1
|
||||
assert not warnings
|
||||
|
||||
def test_hedge_after_s_domain_four_paths(self):
|
||||
"""非法值四条装配路全部当场 ValueError(消息须定位得到是哪个键/参数)。"""
|
||||
from tests.unit.test_client import _client
|
||||
|
||||
# 路 1: env(origin 是实际命中的键名)
|
||||
with pytest.raises(ValueError, match="LLM__HEDGE__AFTER_S"):
|
||||
GatewaySettings.from_env("LLM", env=_env(**{"LLM__HEDGE__AFTER_S": "0"}))
|
||||
with pytest.raises(ValueError, match="LLM__HEDGE__AFTER_S"):
|
||||
GatewaySettings.from_env("LLM", env=_env(**{"LLM__HEDGE__AFTER_S": "abc"}))
|
||||
base = GatewaySettings.from_env("LLM", env=_env())
|
||||
# 路 2: 直接构造
|
||||
fields = {f.name: getattr(base, f.name) for f in dataclasses.fields(base)}
|
||||
with pytest.raises(ValueError, match="hedge_after_s"):
|
||||
GatewaySettings(**{**fields, "hedge_after_s": float("nan")})
|
||||
# 路 3: dataclasses.replace
|
||||
with pytest.raises(ValueError, match="hedge_after_s"):
|
||||
dataclasses.replace(base, hedge_after_s=-1)
|
||||
# 路 4: client 直传(不经 settings 那道守卫)
|
||||
with pytest.raises(ValueError, match=r"GatewayClient\(hedge_after_s"):
|
||||
_client(hedge_after_s=0)
|
||||
|
||||
def test_hedge_guard_empty_sources_raises_with_hedge_location(self):
|
||||
"""直传路空 sources + 设阈值: ValueError 且消息定位到 hedge(不是裸 min() 报错)。"""
|
||||
from polygateway.config import check_hedge_assembly
|
||||
|
||||
with pytest.raises(ValueError, match="sources") as ei:
|
||||
check_hedge_assembly(
|
||||
hedge_after_s=8,
|
||||
hedge_max_extra=1,
|
||||
sources=[],
|
||||
call_deadline_s=None,
|
||||
origin="GatewayClient(hedge_after_s=8)",
|
||||
)
|
||||
assert "hedge_after_s" in str(ei.value) # 定位得到是哪个键
|
||||
# 未启用对冲(None)时空 sources 直接放行: 交叉守卫没有可校验的对象
|
||||
assert (
|
||||
check_hedge_assembly(
|
||||
hedge_after_s=None,
|
||||
hedge_max_extra=1,
|
||||
sources=[],
|
||||
call_deadline_s=None,
|
||||
origin="test",
|
||||
)
|
||||
is None
|
||||
)
|
||||
|
||||
def test_hedge_guard_below_min_timeout_raises(self):
|
||||
"""阈值 ≥ 最小源 timeout_s = 对冲永不可能触发,装配期炸掉(ValueError)。"""
|
||||
env = self._two_source_env(**{"LLM__HEDGE__AFTER_S": "90"}) # min(timeout)=90
|
||||
with pytest.raises(ValueError, match="timeout_s"):
|
||||
GatewaySettings.from_env("LLM", env=env)
|
||||
# 边界内侧合法(89 < 90)
|
||||
s = GatewaySettings.from_env(
|
||||
"LLM", env=self._two_source_env(**{"LLM__HEDGE__AFTER_S": "89"})
|
||||
)
|
||||
assert s.hedge_after_s == 89.0
|
||||
|
||||
def test_hedge_guard_ttft_warns(self):
|
||||
"""阈值 ≥ 最小已设 ttft_timeout_s: 流式被看门狗先切,装配期 warning 而非 ValueError。"""
|
||||
env = self._two_source_env(
|
||||
**{
|
||||
"LLM__QWEN__1__TTFT_TIMEOUT_S": "30",
|
||||
"LLM__QWEN__1__INTER_TOKEN_TIMEOUT_S": "15",
|
||||
"LLM__HEDGE__AFTER_S": "35", # ≥ ttft 30, < timeout 90
|
||||
}
|
||||
)
|
||||
with _captured_warnings() as warnings:
|
||||
s = GatewaySettings.from_env("LLM", env=env)
|
||||
assert s.hedge_after_s == 35.0 # warning 不是拒绝: 非流式仍有效
|
||||
assert any("ttft_timeout_s" in m for m in warnings)
|
||||
# 阈值低于看门狗时不告警
|
||||
with _captured_warnings() as warnings2:
|
||||
GatewaySettings.from_env(
|
||||
"LLM",
|
||||
env=self._two_source_env(
|
||||
**{
|
||||
"LLM__QWEN__1__TTFT_TIMEOUT_S": "30",
|
||||
"LLM__QWEN__1__INTER_TOKEN_TIMEOUT_S": "15",
|
||||
"LLM__HEDGE__AFTER_S": "25",
|
||||
}
|
||||
),
|
||||
)
|
||||
assert not warnings2
|
||||
|
||||
def test_hedge_guard_single_source_warns(self):
|
||||
"""单源 scope 设阈值: 装配期 warning 放行,运行期拿不到候选自然静默。"""
|
||||
with _captured_warnings() as warnings:
|
||||
s = GatewaySettings.from_env("LLM", env=_env(**{"LLM__HEDGE__AFTER_S": "8"}))
|
||||
assert s.hedge_after_s == 8.0
|
||||
assert any("单源" in m for m in warnings)
|
||||
|
||||
def test_hedge_guard_deadline_conflict_raises(self):
|
||||
"""阈值 ≥ call_deadline_s: 期限先于对冲触发,对冲形同虚设 → ValueError(§6)。"""
|
||||
env = self._two_source_env(**{"LLM__HEDGE__AFTER_S": "35", "LLM__CALL_DEADLINE_S": "30"})
|
||||
with pytest.raises(ValueError, match="call_deadline_s"):
|
||||
GatewaySettings.from_env("LLM", env=env)
|
||||
# 边界值(恰好相等)同样拒绝
|
||||
with pytest.raises(ValueError, match="call_deadline_s"):
|
||||
GatewaySettings.from_env(
|
||||
"LLM",
|
||||
env=self._two_source_env(
|
||||
**{"LLM__HEDGE__AFTER_S": "30", "LLM__CALL_DEADLINE_S": "30"}
|
||||
),
|
||||
)
|
||||
# 阈值 < 期限是合法组合
|
||||
s = GatewaySettings.from_env(
|
||||
"LLM",
|
||||
env=self._two_source_env(**{"LLM__HEDGE__AFTER_S": "29", "LLM__CALL_DEADLINE_S": "30"}),
|
||||
)
|
||||
assert s.hedge_after_s == 29.0 and s.call_deadline_s == 30.0
|
||||
|
||||
def test_hedge_max_extra_v1_cap(self):
|
||||
"""max_extra 值域 [1,3] 的四路校验;>1 已接受但 warning 声明 v1 仅单路生效(H5)。"""
|
||||
# 域外值无条件拒绝(即使对冲未启用: 非法值没有"惰性"豁免)
|
||||
with pytest.raises(ValueError, match="hedge_max_extra"):
|
||||
GatewaySettings.from_env("LLM", env=_env(**{"LLM__HEDGE__MAX_EXTRA": "0"}))
|
||||
with pytest.raises(ValueError, match="LLM__HEDGE__MAX_EXTRA"):
|
||||
GatewaySettings.from_env("LLM", env=_env(**{"LLM__HEDGE__MAX_EXTRA": "x"}))
|
||||
with pytest.raises(ValueError, match="hedge_max_extra"):
|
||||
GatewaySettings.from_env("LLM", env=_env(**{"LLM__HEDGE__MAX_EXTRA": "4"}))
|
||||
base = GatewaySettings.from_env("LLM", env=_env())
|
||||
with pytest.raises(ValueError, match="hedge_max_extra"):
|
||||
dataclasses.replace(base, hedge_max_extra=0)
|
||||
# 2/3 接受 + warning: v1 运行期恒单路(对冲任务不再携带首 token 观测,
|
||||
# 行为面由 test_hedge.py 的 ft_events 断言钉住),梯次追加为 H5 预留
|
||||
env = self._two_source_env(**{"LLM__HEDGE__AFTER_S": "8", "LLM__HEDGE__MAX_EXTRA": "2"})
|
||||
with _captured_warnings() as warnings:
|
||||
s = GatewaySettings.from_env("LLM", env=env)
|
||||
assert s.hedge_max_extra == 2
|
||||
assert any("单路" in m for m in warnings)
|
||||
|
||||
def test_from_settings_propagates_hedge_to_client(self):
|
||||
"""from_settings 透传: RetryMW 拿到归一化阈值;max_extra 不下传(v1 无消费者)。"""
|
||||
env = self._two_source_env(**{"LLM__HEDGE__AFTER_S": "8"})
|
||||
s = GatewaySettings.from_env("LLM", env=env)
|
||||
client = GatewayClient.from_settings(s)
|
||||
assert client._hedge_after_s == 8.0
|
||||
assert client._terminal._hedge_after_s == 8.0
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
"""`deadline.py` 值域校验与五种形态区分测试(计划 §5 批次 A/B)。
|
||||
|
||||
用真实事件循环时钟(期限 0.05s、体 0.3s,4-10 倍余量),不标 slow:
|
||||
被测对象是"哪一种 TimeoutError"的身份判据,注入钟无法覆盖 `asyncio.timeout`。
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from polygateway.deadline import ensure_call_deadline, with_call_deadline
|
||||
from polygateway.errors import CallDeadlineExceeded
|
||||
|
||||
# —— 批次 A: 值域 ——
|
||||
|
||||
|
||||
def test_ensure_call_deadline_accepts_none_and_positive():
|
||||
assert ensure_call_deadline(None, "origin") is None
|
||||
assert ensure_call_deadline(3, "origin") == 3.0
|
||||
assert ensure_call_deadline(0.5, "origin") == 0.5
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad",
|
||||
[0, 0.0, -1, -0.5, float("nan"), float("inf"), float("-inf"), "1", True, False, object(), []],
|
||||
)
|
||||
def test_ensure_call_deadline_rejects_out_of_range(bad):
|
||||
with pytest.raises(ValueError) as exc:
|
||||
ensure_call_deadline(bad, "GatewayClient(call_deadline_s=...)")
|
||||
assert "GatewayClient(call_deadline_s=...)" in str(exc.value)
|
||||
|
||||
|
||||
def test_ensure_call_deadline_rejects_huge_int_without_leaking_overflow():
|
||||
"""超出 float 值域的巨大 int 也统一 ValueError,不泄漏 OverflowError。"""
|
||||
with pytest.raises(ValueError) as exc:
|
||||
ensure_call_deadline(10**400, "origin")
|
||||
assert "origin" in str(exc.value)
|
||||
|
||||
|
||||
# —— 批次 B: 五种形态 ——
|
||||
|
||||
|
||||
async def test_deadline_expiry_raises_call_deadline_exceeded():
|
||||
async def body():
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
with pytest.raises(CallDeadlineExceeded) as exc:
|
||||
await with_call_deadline(body(), deadline_s=0.05, scope="llm")
|
||||
assert exc.value.scope == "llm"
|
||||
assert exc.value.deadline_s == 0.05
|
||||
|
||||
|
||||
async def test_inner_timeout_before_expiry_propagates_as_is():
|
||||
async def body():
|
||||
async with asyncio.timeout(0.01):
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
with pytest.raises(TimeoutError) as exc:
|
||||
await with_call_deadline(body(), deadline_s=5.0, scope="llm")
|
||||
assert not isinstance(exc.value, CallDeadlineExceeded)
|
||||
|
||||
|
||||
async def test_cleanup_timeout_after_expiry_is_not_relabelled():
|
||||
"""到期后清理路径自抛 TimeoutError → 原样上抛(钉住身份比较,不看 expired())。"""
|
||||
|
||||
async def body():
|
||||
try:
|
||||
await asyncio.sleep(0.3)
|
||||
except asyncio.CancelledError:
|
||||
raise TimeoutError("cleanup") from None
|
||||
|
||||
with pytest.raises(TimeoutError) as exc:
|
||||
await with_call_deadline(body(), deadline_s=0.05, scope="llm")
|
||||
assert not isinstance(exc.value, CallDeadlineExceeded)
|
||||
assert str(exc.value) == "cleanup"
|
||||
|
||||
|
||||
async def test_external_cancel_before_expiry_propagates_cancelled():
|
||||
entered = asyncio.Event()
|
||||
|
||||
async def body():
|
||||
entered.set()
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
task = asyncio.create_task(with_call_deadline(body(), deadline_s=5.0, scope="llm"))
|
||||
await entered.wait()
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
|
||||
|
||||
async def test_external_cancel_after_expiry_propagates_cancelled():
|
||||
"""到期已在途、外部又取消 → 仍是 CancelledError(取消优先,不被改标)。"""
|
||||
|
||||
started = asyncio.Event()
|
||||
|
||||
async def body():
|
||||
started.set()
|
||||
try:
|
||||
await asyncio.sleep(0.3)
|
||||
except asyncio.CancelledError:
|
||||
await asyncio.sleep(0.2) # 清理期,期间遭外部取消
|
||||
raise
|
||||
|
||||
task = asyncio.create_task(with_call_deadline(body(), deadline_s=0.05, scope="llm"))
|
||||
await started.wait()
|
||||
await asyncio.sleep(0.1) # 让期限先到期,进入清理
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
|
||||
|
||||
async def test_narrow_success_returns_value_without_pending_cancellation():
|
||||
async def body():
|
||||
await asyncio.sleep(0.01)
|
||||
return "ok"
|
||||
|
||||
async def runner():
|
||||
return await with_call_deadline(body(), deadline_s=0.2, scope="llm")
|
||||
|
||||
task = asyncio.create_task(runner())
|
||||
assert await task == "ok"
|
||||
assert task.cancelling() == 0
|
||||
|
||||
|
||||
async def test_domain_error_inside_window_propagates():
|
||||
"""计时器已触发但取消尚未投递的窗口内,体内先抛领域异常 → 原样上抛,期限静默让位。
|
||||
|
||||
忙等超过期限: 计时器回调已在 loop 上触发,但任务不挂起取消就投递不进来,
|
||||
此刻体内同步抛出的领域异常必须原样逃逸(设计 §5.2 形态四)。
|
||||
"""
|
||||
|
||||
class BoomError(RuntimeError):
|
||||
pass
|
||||
|
||||
async def body():
|
||||
end = time.monotonic() + 0.1
|
||||
while time.monotonic() < end:
|
||||
pass
|
||||
raise BoomError("boom")
|
||||
|
||||
with pytest.raises(BoomError):
|
||||
await with_call_deadline(body(), deadline_s=0.02, scope="llm")
|
||||
|
||||
|
||||
async def test_none_deadline_takes_the_legacy_path():
|
||||
async def body():
|
||||
await asyncio.sleep(0.05)
|
||||
return "ok"
|
||||
|
||||
assert await with_call_deadline(body(), deadline_s=None, scope="llm") == "ok"
|
||||
@@ -13,6 +13,7 @@ import pytest
|
||||
from loguru import logger
|
||||
|
||||
from polygateway.errors import (
|
||||
CallDeadlineExceeded,
|
||||
RequestRejectedError,
|
||||
ResultInvalidError,
|
||||
SourceDeadError,
|
||||
@@ -197,6 +198,8 @@ class ScriptedEmbedTransport:
|
||||
def __init__(self, script):
|
||||
self.script = list(script)
|
||||
self.calls = []
|
||||
# 取消用例的确定性窗口: 进入 hang 分支即置位, 不用 sleep 撞窗口
|
||||
self.entered = asyncio.Event()
|
||||
|
||||
async def embed(self, *, texts, source, call_id):
|
||||
self.calls.append((source.name, list(texts), call_id))
|
||||
@@ -204,6 +207,7 @@ class ScriptedEmbedTransport:
|
||||
if isinstance(action, Exception):
|
||||
raise action
|
||||
if action == "hang":
|
||||
self.entered.set()
|
||||
await asyncio.Event().wait()
|
||||
if action == "ok":
|
||||
return _vec_for(texts)
|
||||
@@ -315,6 +319,20 @@ class TestEmbedBatching:
|
||||
assert resp.cost is None
|
||||
|
||||
|
||||
class TestEmbedGenerationMs:
|
||||
"""裸生成时间按批累加(1.3.7 H8): generation_ms = 各批 transport 耗时之和。"""
|
||||
|
||||
async def test_generation_ms_sums_batch_transports(self):
|
||||
# 0.25s 为二进制可精确表示值: int 截断下非精确值会因浮点误差少 1ms
|
||||
clock = FakeClock()
|
||||
transport = _ClockAdvancingEmbedTransport([(0.25, "ok"), (0.25, "ok")], clock)
|
||||
client, _ = _embed_client([_src()], [], transport=transport, now=clock)
|
||||
resp = await client.embed(["a", "bb", "ccc", "dddd"])
|
||||
assert resp.call_stats is not None
|
||||
assert resp.call_stats.attempts == 2
|
||||
assert resp.call_stats.generation_ms == 500
|
||||
|
||||
|
||||
class TestEmbedPostProcess:
|
||||
async def test_normalize_l2(self):
|
||||
raw = EmbeddingTransportResult(
|
||||
@@ -365,6 +383,20 @@ class TestEmbedGovernance:
|
||||
await task
|
||||
assert (await limiter.source_stats("e1")).inflight == 0
|
||||
|
||||
async def test_cancel_in_flight_keeps_the_reservation(self):
|
||||
"""S7(与 chat 同口径): transport 在途被取消 → 用量未知 → 保留预扣而非退成 0。"""
|
||||
transport = ScriptedEmbedTransport(["hang"])
|
||||
client, limiter = _embed_client(
|
||||
[_src(max_concurrency=1, tpm=1000, est_tokens=400)], [], transport=transport
|
||||
)
|
||||
task = asyncio.create_task(client.embed(["a"]))
|
||||
await transport.entered.wait()
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
stats = await limiter.source_stats("e1")
|
||||
assert stats.tpm_used == 400 and stats.inflight == 0
|
||||
|
||||
async def test_single_timeout_does_not_exhaust_stall_budget(self):
|
||||
"""issue #8: 一次耗满 timeout 的尝试不得吃掉 stall 预算而使重试失效。
|
||||
|
||||
@@ -580,7 +612,12 @@ class TestReasonlessTelemetryContract:
|
||||
client, _ = _embed_client([_src(enable_thinking=True)], script, telemetry=recorder)
|
||||
with pytest.raises(AllSourcesExhausted if exhausted else RequestRejectedError):
|
||||
await client.embed(["text"])
|
||||
assert len(recorder.rows) == len(script)
|
||||
# 1.3.5: 逐次 attempt 行之外,本次逻辑调用另有**一条**终态行
|
||||
attempts = [r for r in recorder.rows if r["event_kind"] == "attempt"]
|
||||
terminals = [r for r in recorder.rows if r["event_kind"] == "terminal_failure"]
|
||||
assert len(attempts) == len(script)
|
||||
assert len(terminals) == 1
|
||||
# 推理档在三类行上都必须是 NULL: embed payload 从不带推理参数
|
||||
assert all(r["error"] and r["reasoning_effort"] is None for r in recorder.rows)
|
||||
|
||||
async def test_embedding_wire_ignores_reasoning_configuration(self):
|
||||
@@ -600,3 +637,110 @@ class TestReasonlessTelemetryContract:
|
||||
assert seen == [{"model": "embed-1", "input": ["text"]}]
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
class TestEmbedLogicalCallStats:
|
||||
"""分批共享同一逻辑调用(1.3.5 设计 §3/§3.5)。"""
|
||||
|
||||
async def test_three_batches_count_three_attempts(self):
|
||||
"""分批是库的实现细节,但每批都真打了一次网关,故计 3 次尝试。"""
|
||||
client, _ = _embed_client([_src()], ["ok", "ok", "ok"], batch_size=2)
|
||||
resp = await client.embed(["a", "bb", "ccc", "dddd", "eeeee"])
|
||||
assert resp.call_stats is not None
|
||||
assert resp.call_stats.attempts == 3
|
||||
|
||||
async def test_separate_calls_get_distinct_logical_ids(self):
|
||||
"""一次公开调用一个 ID: 两次 `embed` 不得共用同一个。
|
||||
|
||||
共用就意味着上下文被提升成了 client 实例属性(库铁律禁止的形态)。
|
||||
"""
|
||||
client, _ = _embed_client([_src()], ["ok"] * 5, batch_size=2)
|
||||
first = await client.embed(["a", "bb", "ccc", "dddd", "eeeee"]) # 3 批
|
||||
second = await client.embed(["x", "y"]) # 1 批
|
||||
assert first.call_stats.attempts == 3 and second.call_stats.attempts == 1
|
||||
assert first.call_stats.logical_call_id != second.call_stats.logical_call_id
|
||||
|
||||
async def test_retry_within_a_batch_is_counted(self):
|
||||
client, _ = _embed_client([_src(), _src(name="e2")], [TransientError("t1"), "ok"])
|
||||
resp = await client.embed(["a"])
|
||||
assert resp.call_stats.attempts == 2
|
||||
|
||||
async def test_empty_input_is_zero_attempts_and_writes_no_telemetry_row(self):
|
||||
"""合法零尝试: 返回真实统计,且**不写任何遥测行**(设计 §3 M2)。
|
||||
|
||||
与 cache_hit 不同——不要按"遥测必录"推断空输入也有台账行。
|
||||
"""
|
||||
rec = _MemoryRecorder()
|
||||
client, _ = _embed_client([_src()], [], telemetry=rec)
|
||||
resp = await client.embed([])
|
||||
assert resp.call_stats is not None
|
||||
assert resp.call_stats.attempts == 0
|
||||
assert resp.call_stats.logical_call_id # 真实 ID,不是空串
|
||||
assert rec.rows == [] # 零遥测行
|
||||
|
||||
|
||||
class _SlowEmbedTransport:
|
||||
"""假 embedding transport: 每批真实耗时 `delay` 秒。
|
||||
|
||||
"N 批共享一份期限"只能用真实等待来证——`asyncio.timeout` 认的是事件循环
|
||||
时钟,注入钟推不动它(计划 §5 批次 D)。
|
||||
"""
|
||||
|
||||
def __init__(self, *, delay):
|
||||
self._delay = delay
|
||||
self.calls = []
|
||||
|
||||
async def embed(self, *, texts, source, call_id):
|
||||
self.calls.append(list(texts))
|
||||
await asyncio.sleep(self._delay)
|
||||
return _vec_for(texts)
|
||||
|
||||
|
||||
class TestEmbedCallDeadline:
|
||||
"""embedding 的期限语义: 整次调用一份,空输入豁免(计划 §5 批次 D/E)。"""
|
||||
|
||||
async def test_one_deadline_is_shared_across_all_batches(self):
|
||||
"""按批各起一份会让期限被批数放大 N 倍: 单批 0.05s 远小于期限 0.5s 时将永不到期。
|
||||
|
||||
余量刷到 10 倍(单批 0.05s vs 期限 0.5s): 要报假结论得单批慢 10 倍,
|
||||
而不是机器抳一下就变色。
|
||||
"""
|
||||
transport = _SlowEmbedTransport(delay=0.05)
|
||||
client, _ = _embed_client([_src()], [], batch_size=1, transport=transport)
|
||||
loop = asyncio.get_running_loop()
|
||||
started = loop.time()
|
||||
with pytest.raises(CallDeadlineExceeded) as exc:
|
||||
await client.embed([str(i) for i in range(20)], call_deadline_s=0.5)
|
||||
elapsed = loop.time() - started
|
||||
assert exc.value.scope == "embed"
|
||||
# 按批计的话 20 批全都能跑完(根本不会抛),共享一份则跑不到头
|
||||
assert 1 <= len(transport.calls) < 20
|
||||
assert elapsed < 20 * 0.05, f"总时长疑似随批数放大: {elapsed}s"
|
||||
|
||||
async def test_empty_input_is_exempt_from_the_deadline(self):
|
||||
"""`texts == []` 早返回在 try 之外(零尝试、无等待可治),再小的期限也不该拦它。"""
|
||||
transport = ScriptedEmbedTransport([])
|
||||
client, _ = _embed_client([_src()], [], transport=transport)
|
||||
resp = await client.embed([], call_deadline_s=1e-6)
|
||||
assert resp.vectors == []
|
||||
assert resp.call_stats is not None and resp.call_stats.attempts == 0
|
||||
assert transport.calls == []
|
||||
|
||||
async def test_the_same_tiny_deadline_does_fire_on_a_non_empty_input(self):
|
||||
"""对照组: 上一条用的 1e-6 秒确实是会到期的值,豁免不是因为期限没生效。"""
|
||||
transport = _SlowEmbedTransport(delay=0.05)
|
||||
client, _ = _embed_client([_src()], [], transport=transport)
|
||||
with pytest.raises(CallDeadlineExceeded):
|
||||
await client.embed(["a"], call_deadline_s=1e-6)
|
||||
|
||||
async def test_illegal_per_call_value_is_rejected_at_the_entry(self):
|
||||
"""per-call 非法值当场 ValueError,且消息指向 `embed(...)` 而非某个 env 键。"""
|
||||
transport = ScriptedEmbedTransport([])
|
||||
client, _ = _embed_client([_src()], [], transport=transport)
|
||||
with pytest.raises(ValueError, match=r"embed\(call_deadline_s"):
|
||||
await client.embed(["a"], call_deadline_s=0)
|
||||
assert transport.calls == []
|
||||
|
||||
def test_illegal_constructor_value_is_rejected_at_assembly(self):
|
||||
with pytest.raises(ValueError, match=r"EmbeddingClient\(call_deadline_s"):
|
||||
_embed_client([_src()], [], call_deadline_s=-1)
|
||||
|
||||
@@ -0,0 +1,594 @@
|
||||
"""对冲编排测试(issue #24 设计 §4.6 / 计划批次 G)。
|
||||
|
||||
设施纪律(计划 §5): 两源 scope、事件驱动假 transport(每源一对 entered/release
|
||||
Event + 可脚本化"先置首 token 再挂起")、**真实 loop 钟**(不注入 FakeClock)、
|
||||
`hedge_after_s=0.05`、断言容差 4–10×;取消窗口用 entered 双 Event 栅栏钉死,
|
||||
禁 sleep 撞窗口;计时只断言下界与相对比较,不断言精确值。
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from polygateway import CallDeadlineExceeded
|
||||
from polygateway.backends.memory.breaker import InMemoryGate
|
||||
from polygateway.backends.memory.limiter import InMemoryLimiter
|
||||
from polygateway.errors import AllSourcesExhausted, TransientError
|
||||
from polygateway.middleware.retry import RetryMW
|
||||
from polygateway.sources import SourceCooldownMemo
|
||||
from polygateway.types import (
|
||||
BackpressurePolicy,
|
||||
BreakerConfig,
|
||||
ChatRequest,
|
||||
GlobalLimits,
|
||||
RetryPolicy,
|
||||
_CallContext,
|
||||
)
|
||||
from tests.contracts.conftest import FakeClock
|
||||
from tests.unit.test_retry import RecordingSelector, StaticSelector, _ok, _src
|
||||
|
||||
_BREAKER = BreakerConfig(fail_threshold=3, cooldown_s=60.0, probe_ttl_s=120.0)
|
||||
_NO_GLOBAL = GlobalLimits(max_concurrency=0, rpm=0, tpm=0)
|
||||
_HEDGE_AFTER_S = 0.05 # 真实 loop 钟阈值;断言上界取 10×(0.5s)
|
||||
|
||||
|
||||
class HedgeTransport:
|
||||
"""事件驱动假 transport: 按源剧本精确控制首 token 置位与完成时刻。
|
||||
|
||||
剧本动作(每源一条队列,耗尽后重复最后一项——429 风暴用例需无限供应):
|
||||
("hang",) — 置位该源 entered,挂起直到该源 release(或被取消)
|
||||
("token_then_hang",) — 先置 first_token_event 再 hang(首 token 已至)
|
||||
("succeed", content) — 立即成功
|
||||
("succeed_after", delay, content)— 真实 loop 钟睡 delay 后成功
|
||||
("fail_after", delay, factory) — 睡 delay 后抛 `factory()` 新造的异常
|
||||
"""
|
||||
|
||||
def __init__(self, scripts: dict[str, list[tuple]]):
|
||||
self._scripts = {name: list(actions) for name, actions in scripts.items()}
|
||||
self.calls: list[str] = []
|
||||
# 逐次记录收到的 first_token_event 身份(H5: 对冲路恒为 None,不再梯次)
|
||||
self.ft_events: list[object] = []
|
||||
self.entered = {name: asyncio.Event() for name in scripts}
|
||||
self.release = {name: asyncio.Event() for name in scripts}
|
||||
|
||||
async def complete(
|
||||
self, *, messages, source, stream, overlay, call_id, reasoning_effort, first_token_event
|
||||
):
|
||||
self.calls.append(source.name)
|
||||
self.ft_events.append(first_token_event)
|
||||
actions = self._scripts[source.name]
|
||||
action = actions.pop(0) if len(actions) > 1 else actions[0]
|
||||
kind = action[0]
|
||||
if kind == "hang":
|
||||
self.entered[source.name].set()
|
||||
await self.release[source.name].wait()
|
||||
return _ok(f"ok-{source.name}")
|
||||
if kind == "token_then_hang":
|
||||
if first_token_event is not None:
|
||||
first_token_event.set()
|
||||
self.entered[source.name].set()
|
||||
await self.release[source.name].wait()
|
||||
return _ok(f"ok-{source.name}")
|
||||
if kind == "succeed":
|
||||
return _ok(action[1])
|
||||
if kind == "succeed_after":
|
||||
await asyncio.sleep(action[1])
|
||||
return _ok(action[2])
|
||||
if kind == "fail_after":
|
||||
await asyncio.sleep(action[1])
|
||||
raise action[2]()
|
||||
raise AssertionError(f"未知剧本动作: {action!r}")
|
||||
|
||||
|
||||
class BlockingLimiter:
|
||||
"""限流包装: 挂起指定源的 try_acquire 直到测试放行(确定性复现"对冲准入挂起")。
|
||||
|
||||
`acquiring` 置位 = 对冲准入已停在该源闸内;`allow` 置位后才继续。只拦对冲
|
||||
会走到的源,原路准入不受影响;其余方法逐字委托内层 InMemoryLimiter。
|
||||
"""
|
||||
|
||||
def __init__(self, inner: InMemoryLimiter, block_source: str):
|
||||
self._inner = inner
|
||||
self._block_source = block_source
|
||||
self.acquiring = asyncio.Event()
|
||||
self.allow = asyncio.Event()
|
||||
|
||||
async def try_acquire(self, source_key, est_tokens):
|
||||
if source_key == self._block_source:
|
||||
self.acquiring.set()
|
||||
await self.allow.wait()
|
||||
return await self._inner.try_acquire(source_key, est_tokens)
|
||||
|
||||
async def acquire(self, source_key, est_tokens):
|
||||
return await self._inner.acquire(source_key, est_tokens)
|
||||
|
||||
async def source_stats(self, source_key):
|
||||
return await self._inner.source_stats(source_key)
|
||||
|
||||
async def mark_progress(self):
|
||||
return await self._inner.mark_progress()
|
||||
|
||||
async def progress_age_s(self):
|
||||
return await self._inner.progress_age_s()
|
||||
|
||||
|
||||
class RecordingEmitter:
|
||||
"""逐次遥测假 emitter: 记录每行的源/错误标签/逻辑调用 ID/attempt call_id。"""
|
||||
|
||||
def __init__(self):
|
||||
self.rows: list[dict] = []
|
||||
|
||||
async def emit_attempt(
|
||||
self,
|
||||
*,
|
||||
request,
|
||||
source,
|
||||
call_id,
|
||||
latency_ms,
|
||||
response,
|
||||
error,
|
||||
reasoning_applies,
|
||||
operation,
|
||||
):
|
||||
self.rows.append(
|
||||
{
|
||||
"source": source.name,
|
||||
"call_id": call_id,
|
||||
"logical_call_id": request.call_context.logical_call_id
|
||||
if request.call_context is not None
|
||||
else None,
|
||||
"error": error,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _harness(
|
||||
sources,
|
||||
transport,
|
||||
*,
|
||||
hedge_after_s=_HEDGE_AFTER_S,
|
||||
max_attempts=3,
|
||||
emitter=None,
|
||||
selector=None,
|
||||
limiter=None,
|
||||
gate=None,
|
||||
stall_window_s=300.0,
|
||||
now=None,
|
||||
):
|
||||
"""真实 loop 钟装配(对冲计时纪律: 只用 loop 相对时长,不注入 FakeClock)。
|
||||
|
||||
`now` 仅供"注入钟与对冲触发正交"用例注入 FakeClock——触发路径结构性不读
|
||||
它,注入只是为了证明这一点。
|
||||
"""
|
||||
limiter = limiter or InMemoryLimiter(
|
||||
scope="llm",
|
||||
sources={s.name: s for s in sources},
|
||||
global_limits=_NO_GLOBAL,
|
||||
lease_ttl_s=100.0,
|
||||
)
|
||||
gate = gate or InMemoryGate(config=_BREAKER)
|
||||
mw = RetryMW(
|
||||
scope="llm",
|
||||
sources=sources,
|
||||
# 固定配置序: 原路恒为 s1、对冲路恒为 s2,断言不依赖选源器内部状态
|
||||
selector=selector if selector is not None else StaticSelector(),
|
||||
limiter=limiter,
|
||||
gate=gate,
|
||||
transport=transport,
|
||||
retry=RetryPolicy(max_attempts=max_attempts, backoff_base_s=0.01, backoff_max_s=0.05),
|
||||
backpressure=BackpressurePolicy(stall_window_s=stall_window_s, poll_interval_s=0.01),
|
||||
quota_full="wait",
|
||||
cooldown_memo=SourceCooldownMemo(),
|
||||
emitter=emitter,
|
||||
hedge_after_s=hedge_after_s,
|
||||
**({"now": now} if now is not None else {}),
|
||||
)
|
||||
return mw, limiter, gate
|
||||
|
||||
|
||||
def _req(*, stream=False, ctx=None):
|
||||
return ChatRequest(
|
||||
messages=[{"role": "user", "content": "hi"}], stream=stream, call_context=ctx
|
||||
)
|
||||
|
||||
|
||||
def _ctx():
|
||||
return _CallContext(now=time.monotonic)
|
||||
|
||||
|
||||
class TestHedgeTrigger:
|
||||
"""触发两形态(验收矩阵 ①): 非流式纯时间阈值;流式以首 token 未至为判据。"""
|
||||
|
||||
async def test_non_stream_triggers_hedge_and_fast_leg_wins(self):
|
||||
"""s1 挂起、s2 即时成功: 对冲截断长尾,赢家为对冲路(⑩: 计时不含触发前等待)。"""
|
||||
s1, s2 = _src("s1"), _src("s2")
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed_after", 0.02, "fast")]})
|
||||
mw, _, _ = _harness([s1, s2], transport)
|
||||
ctx = _ctx()
|
||||
started = time.monotonic()
|
||||
# wait_for 是防挂安全带(红相位无实现时 5s 判负),不是计时断言
|
||||
resp = await asyncio.wait_for(mw(_req(stream=False, ctx=ctx)), timeout=5)
|
||||
elapsed = time.monotonic() - started
|
||||
assert resp.content == "fast" and resp.source_name == "s2"
|
||||
assert elapsed < 10 * _HEDGE_AFTER_S # 挂起路被对冲截断,而非等到释放
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 1 and stats.hedge_won is True
|
||||
# 赢家裸生成时间: 下界 = s2 实际 transport 耗时(0.02s 留截断余量),
|
||||
# 且严格小于总时长(不含 0.05s 触发窗等待)
|
||||
assert stats.generation_ms >= 15
|
||||
assert stats.generation_ms < stats.total_latency_ms
|
||||
|
||||
async def test_stream_triggers_only_when_first_token_absent(self):
|
||||
"""两例(①): 首 token 未至 → 触发;先置首 token 再挂起 → 不触发。"""
|
||||
# 例一: 流式但首 token 未至,阈值到 → 对冲触发
|
||||
s1, s2 = _src("s1"), _src("s2")
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "fast")]})
|
||||
mw, _, _ = _harness([s1, s2], transport)
|
||||
ctx = _ctx()
|
||||
resp = await asyncio.wait_for(mw(_req(stream=True, ctx=ctx)), timeout=5)
|
||||
assert resp.source_name == "s2"
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 1 and stats.hedge_won is True and stats.attempts == 2
|
||||
|
||||
# 例二: 首 token 已至(假 transport 先 set 再挂起)→ 不触发,误杀慢生成即此处
|
||||
transport2 = HedgeTransport({"s1": [("token_then_hang",)], "s2": [("succeed", "other")]})
|
||||
mw2, _, _ = _harness([_src("s1"), _src("s2")], transport2)
|
||||
ctx2 = _ctx()
|
||||
|
||||
async def release_later():
|
||||
await asyncio.sleep(4 * _HEDGE_AFTER_S) # 4× 余量确认窗口已过
|
||||
transport2.release["s1"].set()
|
||||
|
||||
releaser = asyncio.create_task(release_later())
|
||||
resp2 = await asyncio.wait_for(mw2(_req(stream=True, ctx=ctx2)), timeout=5)
|
||||
await releaser
|
||||
assert resp2.source_name == "s1"
|
||||
assert transport2.calls == ["s1"] # 对冲从未发出
|
||||
stats2 = ctx2.snapshot()
|
||||
assert stats2.hedges == 0 and stats2.hedge_won is False and stats2.attempts == 1
|
||||
|
||||
async def test_injected_clock_jump_does_not_trigger_hedge(self):
|
||||
"""对冲触发只认真实 loop 钟: 注入钟跳 10^6 秒不得触发对冲(设计 §8 验收矩阵)。
|
||||
|
||||
s1 挂起剧本 + 触发窗内注入钟拨快 10^6 秒: 若触发路径误读注入钟,对冲会
|
||||
**立即**发出;断言对冲实际发出时刻不早于真实 loop 阈值(下界断言,不断
|
||||
精确值),形态同 test_client.py:1974 deadline 的注入钟对应用例。
|
||||
"""
|
||||
clock = FakeClock()
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "hedged")]})
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport, now=clock)
|
||||
ctx = _ctx()
|
||||
task = asyncio.ensure_future(mw(_req(ctx=ctx)))
|
||||
await transport.entered["s1"].wait() # 原路在途,触发窗计时中
|
||||
clock.advance(1_000_000.0) # 跳变落在窗内: 误读注入钟即立刻触发
|
||||
started = time.monotonic()
|
||||
resp = await asyncio.wait_for(task, timeout=5)
|
||||
elapsed = time.monotonic() - started
|
||||
assert resp.source_name == "s2" # 对冲确由真实 loop 阈值触发并截断长尾
|
||||
assert transport.calls == ["s1", "s2"]
|
||||
# 下界留 20% 调度余量;误读注入钟的触发是毫秒级,与此差一个数量级以上
|
||||
assert elapsed >= _HEDGE_AFTER_S * 0.8
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 1 and stats.hedge_won is True
|
||||
|
||||
|
||||
class TestHedgeRouting:
|
||||
"""异源排除与静默放弃(验收矩阵 ②③)。"""
|
||||
|
||||
async def test_hedge_goes_to_other_source(self):
|
||||
"""对冲请求落在另一源;两 attempt 行共享同一 logical_call_id(②⑤)。"""
|
||||
emitter = RecordingEmitter()
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "hedged")]})
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport, emitter=emitter)
|
||||
ctx = _ctx()
|
||||
resp = await asyncio.wait_for(mw(_req(ctx=ctx)), timeout=5)
|
||||
assert resp.source_name == "s2"
|
||||
assert transport.calls == ["s1", "s2"] # 第二请求落在异源
|
||||
# H5 接缝: 原路携带首 token 观测,对冲路恒 None(v1 单路,不再梯次)
|
||||
assert [e is not None for e in transport.ft_events] == [True, False]
|
||||
assert len(emitter.rows) == 2
|
||||
assert {r["logical_call_id"] for r in emitter.rows} == {ctx.logical_call_id}
|
||||
assert emitter.rows[0]["call_id"] != emitter.rows[1]["call_id"] # 各 attempt 独立 ID
|
||||
|
||||
async def test_hedge_silent_when_no_candidate(self):
|
||||
"""异源配额被占满 → 准入失败静默放弃: 不对冲、不抛错、原请求照等(③)。"""
|
||||
s1 = _src("s1")
|
||||
s2 = _src("s2", max_concurrency=1)
|
||||
limiter = InMemoryLimiter(
|
||||
scope="llm", sources={"s1": s1, "s2": s2}, global_limits=_NO_GLOBAL, lease_ttl_s=100.0
|
||||
)
|
||||
held = await limiter.try_acquire("s2", 0) # 外部预占满 s2 并发
|
||||
assert held is not None
|
||||
try:
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "x")]})
|
||||
mw, _, _ = _harness([s1, s2], transport, limiter=limiter)
|
||||
ctx = _ctx()
|
||||
task = asyncio.ensure_future(mw(_req(ctx=ctx)))
|
||||
await transport.entered["s1"].wait()
|
||||
# 4× 余量: 给对冲窗与那次注定失败的准入留足发生时间
|
||||
await asyncio.sleep(4 * _HEDGE_AFTER_S)
|
||||
assert transport.calls == ["s1"] # 对冲静默未发出
|
||||
transport.release["s1"].set()
|
||||
resp = await asyncio.wait_for(task, timeout=5)
|
||||
assert resp.source_name == "s1"
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 0 and stats.attempts == 1 and stats.hedge_won is False
|
||||
finally:
|
||||
await held.release()
|
||||
|
||||
async def test_pick_exclude_all_is_not_a_rejection(self):
|
||||
"""exclude 覆盖全源 → 返回 None 且 gate_rejections==0、reasons 不写(排除 ≠ 拒绝)。
|
||||
|
||||
admission 级直接钉(admission.py:192 `continue` 语义): 若未来重构把排除计入
|
||||
gate_rejections,`on_no_runnable` 的"全源熔断类拒绝"判据会被污染,此钉当场报警。
|
||||
"""
|
||||
transport = HedgeTransport({"s1": [("succeed", "x")], "s2": [("succeed", "y")]})
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport)
|
||||
reasons = {"prior": "rate_limited"} # 既有原因须原样保留
|
||||
picked, gate_rejections = await mw._admission.pick(
|
||||
reasons, {}, exclude=frozenset({"s1", "s2"})
|
||||
)
|
||||
assert picked is None
|
||||
assert gate_rejections == 0
|
||||
assert reasons == {"prior": "rate_limited"}
|
||||
|
||||
async def test_hedge_silent_when_candidate_circuit_open(self):
|
||||
"""对冲候选被熔断开路 → 静默放弃: 不对冲、不抛错、原请求照等(②③的另一形态)。
|
||||
|
||||
现有限流闸用例只钉了"配额占满"一条静默路径;开路/pacer 拒绝走 pick 的另一
|
||||
分支(gate_rejections 计数、reasons 写 circuit_open、settle_and_release 后
|
||||
返回 None),同样不得发出对冲请求。
|
||||
"""
|
||||
gate = InMemoryGate(config=_BREAKER)
|
||||
entry = await gate.try_enter("s2", "test-owner")
|
||||
assert entry.allowed
|
||||
await gate.record_failure(entry, "source_dead", True) # SourceDead 一击即熔
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "x")]})
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport, gate=gate)
|
||||
ctx = _ctx()
|
||||
task = asyncio.ensure_future(mw(_req(ctx=ctx)))
|
||||
await transport.entered["s1"].wait()
|
||||
# 4× 余量: 给对冲窗与那次注定被开路拒绝的准入留足发生时间
|
||||
await asyncio.sleep(4 * _HEDGE_AFTER_S)
|
||||
assert transport.calls == ["s1"] # 对冲静默未发出
|
||||
transport.release["s1"].set()
|
||||
resp = await asyncio.wait_for(task, timeout=5)
|
||||
assert resp.source_name == "s1"
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 0 and stats.attempts == 1 and stats.hedge_won is False
|
||||
|
||||
async def test_primary_done_during_hedge_admission_sends_no_hedge(self):
|
||||
"""原路在对冲准入 await 期间已完成: 释放对冲准入直接裁定,一个对冲请求都不发。
|
||||
|
||||
剧本钉死窗口(禁 sleep 猜): s2 的 try_acquire 挂起(对冲准入停在闸内) →
|
||||
放行 s1 → 轮询 s1 inflight 归零(_attempt finally 结算完,primary 必 done)
|
||||
→ 此刻才放行对冲准入。pick 返回时原路已了结,编排必须不落 create_task。
|
||||
"""
|
||||
s1 = _src("s1", tpm=1000, est_tokens=400)
|
||||
s2 = _src("s2", tpm=1000, est_tokens=400)
|
||||
inner = InMemoryLimiter(
|
||||
scope="llm",
|
||||
sources={"s1": s1, "s2": s2},
|
||||
global_limits=_NO_GLOBAL,
|
||||
lease_ttl_s=100.0,
|
||||
)
|
||||
limiter = BlockingLimiter(inner, "s2")
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "hedge")]})
|
||||
mw, _, _ = _harness([s1, s2], transport, limiter=limiter)
|
||||
ctx = _ctx()
|
||||
task = asyncio.ensure_future(mw(_req(ctx=ctx)))
|
||||
await transport.entered["s1"].wait() # 原路在途
|
||||
await limiter.acquiring.wait() # 对冲准入停在 s2 闸内(触发窗已过)
|
||||
transport.release["s1"].set() # 原路放行完成
|
||||
while (await inner.source_stats("s1")).inflight != 0:
|
||||
await asyncio.sleep(0.001) # 结算完 = primary 已 done(同一任务步内返回)
|
||||
limiter.allow.set() # pick 此刻才返回: primary.done() 已成立
|
||||
resp = await asyncio.wait_for(task, timeout=5)
|
||||
assert resp.content == "ok-s1" and resp.source_name == "s1"
|
||||
assert transport.calls == ["s1"] # 对冲 HTTP 从未发出(未修前这里会看到 s2)
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 0 and stats.attempts == 1 and stats.hedge_won is False
|
||||
s2_stats = await inner.source_stats("s2")
|
||||
assert s2_stats.inflight == 0 and s2_stats.tpm_used == 0 # 对冲准入按 0 结算释放
|
||||
|
||||
async def test_hedge_silent_when_single_source(self):
|
||||
"""单源 scope: 运行期拿不到异源候选自然静默,行为与不配阈值逐字相同(②)。"""
|
||||
transport = HedgeTransport({"s1": [("hang",)]})
|
||||
mw, _, _ = _harness([_src("s1")], transport)
|
||||
ctx = _ctx()
|
||||
task = asyncio.ensure_future(mw(_req(ctx=ctx)))
|
||||
await transport.entered["s1"].wait()
|
||||
await asyncio.sleep(4 * _HEDGE_AFTER_S) # 窗口已过,仍无候选
|
||||
assert transport.calls == ["s1"]
|
||||
transport.release["s1"].set()
|
||||
resp = await asyncio.wait_for(task, timeout=5)
|
||||
assert resp.content == "ok-s1"
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 0 and stats.attempts == 1 and stats.hedge_won is False
|
||||
|
||||
|
||||
class TestHedgeSettlementAndSignals:
|
||||
"""赢输记账与熔断/健康信号(验收矩阵 ④⑤;设计 §3 关键判断: 挂起 ≠ 源死亡)。"""
|
||||
|
||||
async def test_winner_settles_actual_loser_keeps_est(self):
|
||||
"""赢家按真实 usage 结算;输家取消落 1.3.6 S3 格: est 预扣保留(④)。"""
|
||||
s1 = _src("s1", tpm=1000, est_tokens=400)
|
||||
s2 = _src("s2", tpm=1000, est_tokens=400)
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "win")]})
|
||||
mw, limiter, _ = _harness([s1, s2], transport)
|
||||
resp = await asyncio.wait_for(mw(_req()), timeout=5)
|
||||
assert resp.source_name == "s2"
|
||||
winner_stats = await limiter.source_stats("s2")
|
||||
loser_stats = await limiter.source_stats("s1")
|
||||
assert winner_stats.tpm_used == 15 # 预扣 400,实测 10+5 → settle 后只记 15
|
||||
assert loser_stats.tpm_used == 400 # 输家 est 保留(可能被上游计费,保守下限)
|
||||
assert winner_stats.inflight == 0 and loser_stats.inflight == 0
|
||||
|
||||
async def test_loser_row_labelled_hedge_cancelled(self):
|
||||
"""输家 attempt 行 error=='hedge_cancelled',赢家行无 error,同行逻辑调用(④⑤)。
|
||||
|
||||
终态行是 client 级语义且成功调用本就不写终态行(emit_terminal_once 只在
|
||||
异常/取消路径触发),MW 级可观测面即这两条 attempt 行。
|
||||
"""
|
||||
emitter = RecordingEmitter()
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "win")]})
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport, emitter=emitter)
|
||||
ctx = _ctx()
|
||||
await asyncio.wait_for(mw(_req(ctx=ctx)), timeout=5)
|
||||
assert len(emitter.rows) == 2
|
||||
loser = next(r for r in emitter.rows if r["source"] == "s1")
|
||||
winner = next(r for r in emitter.rows if r["source"] == "s2")
|
||||
assert loser["error"] == "hedge_cancelled"
|
||||
assert winner["error"] is None
|
||||
assert loser["logical_call_id"] == winner["logical_call_id"] == ctx.logical_call_id
|
||||
|
||||
async def test_loser_does_not_feed_breaker(self):
|
||||
"""输家取消不喂熔断失败计数、不喂健康分;赢家照常 record_success(④)。"""
|
||||
selector = RecordingSelector()
|
||||
gate = InMemoryGate(config=_BREAKER)
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "win")]})
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport, selector=selector, gate=gate)
|
||||
await asyncio.wait_for(mw(_req()), timeout=5)
|
||||
gate_s1 = gate._gates["s1"]
|
||||
assert gate_s1.a0 + gate_s1.a1 == 0 # 熔断失败率窗口无样本
|
||||
assert selector.outcomes == [("s2", True)] # 健康喂数只有赢家的成功
|
||||
assert (await gate.try_enter("s1", "w")).allowed # 挂起源未被标记
|
||||
|
||||
async def test_attempts_two_and_no_task_leak(self):
|
||||
"""快照 attempts==2(含输家);返回后无本调用残留任务(⑤)。"""
|
||||
before = asyncio.all_tasks()
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("succeed", "win")]})
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport)
|
||||
ctx = _ctx()
|
||||
await asyncio.wait_for(mw(_req(ctx=ctx)), timeout=5)
|
||||
assert ctx.snapshot().attempts == 2
|
||||
assert asyncio.all_tasks() == before
|
||||
|
||||
|
||||
class TestHedgeCancellation:
|
||||
"""取消穿透(⑦)与期限组合(⑧): 两任务同消、不 shield、不留后台任务。"""
|
||||
|
||||
async def test_external_cancel_cancels_both_legs(self):
|
||||
"""两路均在途时外部取消: CancelledError 上抛,两 permit 释放。
|
||||
|
||||
输家标记只在赢家产生后才置位——外部取消下没有赢家,两行都是普通
|
||||
"cancelled"(⑦;竞速误贴属设计 §4.5 已批准残留)。
|
||||
"""
|
||||
emitter = RecordingEmitter()
|
||||
s1, s2 = _src("s1"), _src("s2")
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("hang",)]})
|
||||
mw, limiter, _ = _harness([s1, s2], transport, emitter=emitter)
|
||||
task = asyncio.ensure_future(mw(_req()))
|
||||
# 双 Event 栅栏: 确认对冲已触发、两路均在途,再取消(禁 sleep 猜窗口)
|
||||
await transport.entered["s1"].wait()
|
||||
await transport.entered["s2"].wait()
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
assert {r["source"]: r["error"] for r in emitter.rows} == {
|
||||
"s1": "cancelled",
|
||||
"s2": "cancelled",
|
||||
}
|
||||
assert (await limiter.source_stats("s1")).inflight == 0
|
||||
assert (await limiter.source_stats("s2")).inflight == 0
|
||||
|
||||
async def test_deadline_cuts_hedged_tree(self):
|
||||
"""client 级 call_deadline_s=0.2 + 两路挂起 → CallDeadlineExceeded,permit 全释放(⑧)。"""
|
||||
from tests.unit.test_client import _client
|
||||
|
||||
s1, s2 = _src("s1"), _src("s2")
|
||||
limiter = InMemoryLimiter(
|
||||
scope="llm", sources={"s1": s1, "s2": s2}, global_limits=_NO_GLOBAL
|
||||
)
|
||||
transport = HedgeTransport({"s1": [("hang",)], "s2": [("hang",)]})
|
||||
client = _client(
|
||||
sources=[s1, s2],
|
||||
transport=transport,
|
||||
limiter=limiter,
|
||||
call_deadline_s=0.2,
|
||||
hedge_after_s=_HEDGE_AFTER_S,
|
||||
)
|
||||
async with client:
|
||||
with pytest.raises(CallDeadlineExceeded):
|
||||
await client.chat([{"role": "user", "content": "hi"}], stream=False)
|
||||
assert transport.calls == ["s1", "s2"] # 期限截止前对冲确已触发
|
||||
assert (await limiter.source_stats("s1")).inflight == 0
|
||||
assert (await limiter.source_stats("s2")).inflight == 0
|
||||
|
||||
|
||||
class TestHedgeWinnerAdjudication:
|
||||
"""赢家裁定(⑩): 取快者,含原路后发先至的对称面。"""
|
||||
|
||||
async def test_primary_late_success_wins_back(self):
|
||||
"""s1 挂 0.3s(6× 阈值)后成功、s2 对冲路在途: 原路先完成 → 原路赢。"""
|
||||
emitter = RecordingEmitter()
|
||||
transport = HedgeTransport({"s1": [("succeed_after", 0.3, "late")], "s2": [("hang",)]})
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport, emitter=emitter)
|
||||
ctx = _ctx()
|
||||
resp = await asyncio.wait_for(mw(_req(ctx=ctx)), timeout=5)
|
||||
assert resp.content == "late" and resp.source_name == "s1"
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 1 and stats.hedge_won is False
|
||||
# 裸生成时间为原路那次 transport 时长(≈300ms,只断言下界与相对关系)
|
||||
assert 250 <= stats.generation_ms <= stats.total_latency_ms
|
||||
loser = next(r for r in emitter.rows if r["source"] == "s2")
|
||||
assert loser["error"] == "hedge_cancelled" # 在途对冲路被裁为输家
|
||||
|
||||
|
||||
class TestHedgeFailureCombination:
|
||||
"""两败汇合(H6): 只计一次重试预算;429 分账逐字沿用 attempt 级机制。"""
|
||||
|
||||
async def test_both_fail_counts_budget_once(self):
|
||||
"""两路 Transient: max_attempts=2 时恰进第二轮(两败只计一次),第二轮两败后才耗尽。"""
|
||||
transport = HedgeTransport(
|
||||
{
|
||||
"s1": [("fail_after", 0.1, lambda: TransientError("p", source_name="s1"))],
|
||||
"s2": [("fail_after", 0.12, lambda: TransientError("h", source_name="s2"))],
|
||||
}
|
||||
)
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport, max_attempts=2)
|
||||
ctx = _ctx()
|
||||
with pytest.raises(AllSourcesExhausted) as ei:
|
||||
await mw(_req(ctx=ctx))
|
||||
assert ei.value.reason == "retry_exhausted"
|
||||
# 若两败计两次预算,第一轮即耗尽,这些调用根本不会发生
|
||||
assert transport.calls == ["s1", "s2", "s1", "s2"]
|
||||
# 两败轮次同样登记对冲路数(设计 §4.5: 实际并发发出即计)
|
||||
assert ctx.snapshot().hedges == 2
|
||||
|
||||
async def test_both_429_refund_no_budget(self):
|
||||
"""两路皆 429: 免预算且耗时退 stall 账——小 stall 窗下终局 stalled 而非耗尽。"""
|
||||
|
||||
def _429():
|
||||
return TransientError("throttled", status_code=429, retry_after_s=0.01)
|
||||
|
||||
transport = HedgeTransport(
|
||||
{"s1": [("fail_after", 0.1, _429)], "s2": [("fail_after", 0.12, _429)]}
|
||||
)
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport, max_attempts=1, stall_window_s=0.3)
|
||||
with pytest.raises(AllSourcesExhausted) as ei:
|
||||
await mw(_req())
|
||||
# max_attempts=1: 任一路计预算都会当场 retry_exhausted;
|
||||
# 两 429 免预算 → 循环到 stall 窗口判死
|
||||
assert ei.value.reason == "stalled"
|
||||
|
||||
async def test_mixed_429_and_failure_counts_budget(self):
|
||||
"""一路 429 一路 Transient → 计一次预算、不退还 stall 账(_combine_failures)。"""
|
||||
transport = HedgeTransport(
|
||||
{
|
||||
"s1": [
|
||||
(
|
||||
"fail_after",
|
||||
0.1,
|
||||
lambda: TransientError("rl", status_code=429, retry_after_s=0.01),
|
||||
)
|
||||
],
|
||||
"s2": [("fail_after", 0.12, lambda: TransientError("boom", source_name="s2"))],
|
||||
}
|
||||
)
|
||||
mw, _, _ = _harness([_src("s1"), _src("s2")], transport, max_attempts=1, stall_window_s=0.3)
|
||||
with pytest.raises(AllSourcesExhausted) as ei:
|
||||
await mw(_req())
|
||||
assert ei.value.reason == "retry_exhausted"
|
||||
assert transport.calls == ["s1", "s2"] # 恰一轮两路: 计一次预算即耗尽
|
||||
@@ -277,6 +277,7 @@ async def _complete(observed, *, call_id="a", stream=False):
|
||||
overlay={},
|
||||
call_id=call_id,
|
||||
reasoning_effort=None,
|
||||
first_token_event=None,
|
||||
)
|
||||
|
||||
|
||||
@@ -1266,6 +1267,7 @@ async def test_structured_first_attempt_requires_exact_initial_messages():
|
||||
overlay={},
|
||||
call_id="first",
|
||||
reasoning_effort=None,
|
||||
first_token_event=None,
|
||||
)
|
||||
event = capture.attempts(session_id="first", parent_call_id="parent")[0].http[0]
|
||||
assert not request_is_valid(event)
|
||||
|
||||
@@ -13,6 +13,7 @@ from polygateway.backends.memory.breaker import InMemoryGate
|
||||
from polygateway.backends.memory.limiter import InMemoryLimiter
|
||||
from polygateway.errors import (
|
||||
AllSourcesExhausted,
|
||||
CallDeadlineExceeded,
|
||||
CircuitOpenError,
|
||||
RequestRejectedError,
|
||||
ResultInvalidError,
|
||||
@@ -60,6 +61,8 @@ class ScriptedOcrTransport:
|
||||
def __init__(self, script):
|
||||
self.script = list(script)
|
||||
self.calls = []
|
||||
# 取消用例的确定性窗口: 进入 hang 分支即置位, 不用 sleep 撞窗口
|
||||
self.entered = asyncio.Event()
|
||||
|
||||
async def _next(self, method, source, call_id):
|
||||
self.calls.append((method, source.name, call_id))
|
||||
@@ -67,6 +70,7 @@ class ScriptedOcrTransport:
|
||||
if isinstance(action, Exception):
|
||||
raise action
|
||||
if action == "hang":
|
||||
self.entered.set()
|
||||
await asyncio.Event().wait()
|
||||
return _TEXT_OK if action == "text" else _LAYOUT_OK
|
||||
|
||||
@@ -192,6 +196,21 @@ class TestSuccessPaths:
|
||||
await client.parse_layout(b"")
|
||||
|
||||
|
||||
class TestOcrGenerationMs:
|
||||
"""OCR 裸生成时间单次覆盖(1.3.7 H8): 计时只包 transport 调用本身。"""
|
||||
|
||||
async def test_generation_ms_single_transport_call(self):
|
||||
# 0.5s 为二进制可精确表示值: int 截断下非精确值会因浮点误差少 1ms
|
||||
clock = FakeClock()
|
||||
transport = ClockAdvancingOcrTransport([(0.5, "text")], clock)
|
||||
client, _, _ = _client([_src()], [], now=clock, transport=transport)
|
||||
r = await client.recognize_text(b"jpg")
|
||||
assert r.text == "LINE-1"
|
||||
assert r.call_stats is not None
|
||||
assert r.call_stats.attempts == 1
|
||||
assert r.call_stats.generation_ms == 500
|
||||
|
||||
|
||||
class TestFailover:
|
||||
async def test_transient_retries_with_backoff(self):
|
||||
sleeps = []
|
||||
@@ -395,6 +414,20 @@ class TestCancellation:
|
||||
stats = await limiter.source_stats("m1")
|
||||
assert stats.inflight == 0 # permit 在 finally 释放
|
||||
|
||||
async def test_cancel_in_flight_still_settles_zero(self):
|
||||
"""S6: OCR 的 0 token 是**事实**而非"未知", 取消也不得改成按 est 结算。"""
|
||||
transport = ScriptedOcrTransport(["hang"])
|
||||
client, limiter, _ = _client(
|
||||
[_src(max_concurrency=1, tpm=1000, est_tokens=400)], [], transport=transport
|
||||
)
|
||||
task = asyncio.create_task(client.recognize_text(b"jpg"))
|
||||
await transport.entered.wait()
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
stats = await limiter.source_stats("m1")
|
||||
assert stats.tpm_used == 0 and stats.inflight == 0
|
||||
|
||||
|
||||
class TestCheckHealth:
|
||||
class _HealthTransport(ScriptedOcrTransport):
|
||||
@@ -613,5 +646,68 @@ class TestReasonlessTelemetryContract:
|
||||
client, _, _ = _client([_src(enable_thinking=True)], script, telemetry=recorder)
|
||||
with pytest.raises(AllSourcesExhausted if exhausted else RequestRejectedError):
|
||||
await getattr(client, method)(b"image")
|
||||
assert len(recorder.rows) == len(script)
|
||||
# 1.3.5: 逐次 attempt 行之外,本次逻辑调用另有**一条**终态行
|
||||
attempts = [r for r in recorder.rows if r["event_kind"] == "attempt"]
|
||||
terminals = [r for r in recorder.rows if r["event_kind"] == "terminal_failure"]
|
||||
assert len(attempts) == len(script)
|
||||
assert len(terminals) == 1
|
||||
# 终态行的 operation 是**公开方法**名,与尝试行一致
|
||||
assert terminals[0]["operation"] == method
|
||||
assert all(r["error"] and r["reasoning_effort"] is None for r in recorder.rows)
|
||||
|
||||
|
||||
class TestOcrLogicalCallStats:
|
||||
"""OCR 两个公开方法各自拥有一次逻辑调用(1.3.5 设计 §3/§3.5)。"""
|
||||
|
||||
async def test_text_success_counts_one_attempt(self):
|
||||
client, _, _ = _client([_src()], ["text"])
|
||||
r = await client.recognize_text(b"jpg")
|
||||
assert r.call_stats is not None and r.call_stats.attempts == 1
|
||||
|
||||
async def test_layout_two_http_calls_count_as_one_attempt(self):
|
||||
"""POST + ZIP GET 在同一次 transport 调用内,计 **1** 次尝试而非 2。
|
||||
|
||||
`attempts` 的语义是"调用 transport 端口的次数",不是 HTTP 请求条数。
|
||||
"""
|
||||
client, _, _ = _client([_src()], ["layout"])
|
||||
r = await client.parse_layout(b"jpg")
|
||||
assert r.call_stats is not None and r.call_stats.attempts == 1
|
||||
|
||||
async def test_retry_counts_every_attempt(self):
|
||||
client, _, _ = _client([_src(), _src(name="m2")], [TransientError("t1"), "text"])
|
||||
r = await client.recognize_text(b"jpg")
|
||||
assert r.call_stats.attempts == 2
|
||||
|
||||
async def test_input_validation_stays_outside_the_stats_boundary(self):
|
||||
"""`image` 类型/空校验先于上下文创建(M1 例外),保持原异常行为。"""
|
||||
client, _, _ = _client([_src()], [])
|
||||
with pytest.raises(TypeError):
|
||||
await client.recognize_text("not-bytes")
|
||||
with pytest.raises(ValueError):
|
||||
await client.recognize_text(b"")
|
||||
|
||||
|
||||
class TestOcrCallDeadline:
|
||||
"""OCR 两个公开入口的期限与 per-call 校验(计划 §3.4/§5 批次 D/E)。"""
|
||||
|
||||
async def test_expiry_on_a_hanging_transport(self):
|
||||
client, limiter, _ = _client([_src()], ["hang"])
|
||||
with pytest.raises(CallDeadlineExceeded) as exc:
|
||||
await client.recognize_text(b"jpg", call_deadline_s=0.05)
|
||||
assert exc.value.scope == "ocr" and exc.value.deadline_s == 0.05
|
||||
# 清理照常在 finally 完成: 在途计数必须归零(OCR 无 token,结算恒 0)
|
||||
stats = await limiter.source_stats("m1")
|
||||
assert stats.inflight == 0 and stats.tpm_used == 0
|
||||
|
||||
@pytest.mark.parametrize("method", ["recognize_text", "parse_layout"])
|
||||
async def test_illegal_per_call_value_names_the_entry_it_came_from(self, method):
|
||||
"""两个入口各自报自己的名字: 多入口部署里才定位得到是哪次调用传错了。"""
|
||||
transport = ScriptedOcrTransport([])
|
||||
client, _, _ = _client([_src()], [], transport=transport)
|
||||
with pytest.raises(ValueError, match=rf"{method}\(call_deadline_s"):
|
||||
await getattr(client, method)(b"jpg", call_deadline_s=float("inf"))
|
||||
assert transport.calls == []
|
||||
|
||||
def test_illegal_constructor_value_is_rejected_at_assembly(self):
|
||||
with pytest.raises(ValueError, match=r"OcrClient\(call_deadline_s"):
|
||||
_client([_src()], [], call_deadline_s=0)
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
SSE 帧样本按三项目真实网关响应形态二次构造(OpenAI 兼容 chunk 结构)。
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
|
||||
import httpx
|
||||
@@ -14,6 +15,7 @@ from polygateway.errors import (
|
||||
SourceDeadError,
|
||||
TransientError,
|
||||
)
|
||||
from polygateway.middleware.retry import backoff_delay
|
||||
from polygateway.middleware.telemetry import TelemetryEmitter
|
||||
from polygateway.pricing import ModelPrice, PricingTable
|
||||
from polygateway.providers import ProviderProfile, ThinkingWire, register_provider
|
||||
@@ -21,9 +23,18 @@ from polygateway.transports._http_errors import summarize_body
|
||||
from polygateway.transports.openai_compat import (
|
||||
OpenAICompatTransport,
|
||||
_iter_sse_deltas,
|
||||
_parse_retry_after,
|
||||
_sse_data_payload,
|
||||
_translate_429,
|
||||
)
|
||||
from polygateway.types import (
|
||||
ChatRequest,
|
||||
Effort,
|
||||
LLMResponse,
|
||||
RetryPolicy,
|
||||
SourceConfig,
|
||||
ThinkingObservation,
|
||||
)
|
||||
from polygateway.types import ChatRequest, Effort, LLMResponse, SourceConfig, ThinkingObservation
|
||||
|
||||
|
||||
def _source(**overrides):
|
||||
@@ -69,7 +80,9 @@ def _transport_for(handler, *, registry=None):
|
||||
)
|
||||
|
||||
|
||||
async def _complete(transport, source, *, stream=True, overlay=None, reasoning_effort=None):
|
||||
async def _complete(
|
||||
transport, source, *, stream=True, overlay=None, reasoning_effort=None, first_token_event=None
|
||||
):
|
||||
return await transport.complete(
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
source=source,
|
||||
@@ -77,6 +90,7 @@ async def _complete(transport, source, *, stream=True, overlay=None, reasoning_e
|
||||
overlay=overlay or {},
|
||||
call_id="cid-1",
|
||||
reasoning_effort=reasoning_effort,
|
||||
first_token_event=first_token_event,
|
||||
)
|
||||
|
||||
|
||||
@@ -116,7 +130,7 @@ async def _recorded_cost(result, source):
|
||||
source_name=source.name,
|
||||
usage_source=result.usage_source,
|
||||
)
|
||||
await TelemetryEmitter(recorder, pricing=_PRICING, text_cap=None).emit_attempt(
|
||||
await TelemetryEmitter(recorder, pricing=_PRICING, text_cap=None, scope="LLM").emit_attempt(
|
||||
request=ChatRequest(messages=[{"role": "user", "content": "hi"}]),
|
||||
source=source,
|
||||
call_id="cid-1",
|
||||
@@ -124,6 +138,7 @@ async def _recorded_cost(result, source):
|
||||
response=response,
|
||||
error=None,
|
||||
reasoning_applies=True,
|
||||
operation="chat",
|
||||
)
|
||||
return recorder.rows[0]["cost"]
|
||||
|
||||
@@ -203,6 +218,65 @@ class TestStreamHappyPath:
|
||||
assert await _recorded_cost(result, source) is None
|
||||
|
||||
|
||||
class TestFirstTokenEvent:
|
||||
"""首 token 处置位(1.3.7 对冲 H2): 流式置位、非流式永不置位、None 不观测。"""
|
||||
|
||||
async def test_stream_sets_first_token_event(self):
|
||||
"""流式首 token(内容或思考增量)到达即置位——对冲触发窗的取消信号。"""
|
||||
|
||||
def handler(request):
|
||||
return _sse_stream(
|
||||
_chunk(reasoning="ponder"), _chunk(content="hi"), _chunk(usage=_USAGE)
|
||||
)
|
||||
|
||||
event = asyncio.Event()
|
||||
result = await _complete(_transport_for(handler), _source(), first_token_event=event)
|
||||
assert result.content == "hi"
|
||||
assert event.is_set()
|
||||
|
||||
async def test_non_stream_never_sets_first_token_event(self):
|
||||
"""非流式物理上无中途信号: 即使调用方给了事件,本路径也永不置位。"""
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
200, json={"choices": [{"message": {"content": "42"}}], "usage": _USAGE}
|
||||
)
|
||||
|
||||
event = asyncio.Event()
|
||||
result = await _complete(
|
||||
_transport_for(handler), _source(), stream=False, first_token_event=event
|
||||
)
|
||||
assert result.content == "42"
|
||||
assert not event.is_set()
|
||||
|
||||
async def test_none_first_token_event_keeps_behavior(self):
|
||||
"""`None` = 调用方不观测首 token(未启用对冲): 行为与旧版逐字相同。"""
|
||||
|
||||
def handler(request):
|
||||
return _sse_stream(_chunk(content="ok"), _chunk(usage=_USAGE))
|
||||
|
||||
result = await _complete(_transport_for(handler), _source(), first_token_event=None)
|
||||
assert result.content == "ok"
|
||||
assert result.ttft_ms is not None
|
||||
|
||||
async def test_first_token_event_is_required_keyword(self):
|
||||
"""端口必填约定: 漏传必须 TypeError——默认值会把"漏传"伪装成"不观测"。"""
|
||||
|
||||
def handler(request):
|
||||
return _sse_stream(_chunk(content="ok"), _chunk(usage=_USAGE))
|
||||
|
||||
transport = _transport_for(handler)
|
||||
with pytest.raises(TypeError):
|
||||
await transport.complete(
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
source=_source(),
|
||||
stream=True,
|
||||
overlay={},
|
||||
call_id="cid-1",
|
||||
reasoning_effort=None,
|
||||
)
|
||||
|
||||
|
||||
class TestMissingDoneSemantics:
|
||||
def _no_done_handler(self, request):
|
||||
return _sse_stream(_chunk(content="partial"), _chunk(usage=_USAGE), done=False)
|
||||
@@ -1143,3 +1217,67 @@ async def test_custom_profile_raw_roots_cannot_override_managed_intent(key):
|
||||
assert sent == []
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
class TestRetryAfterNonFinite:
|
||||
"""F1: `Retry-After` 非有限值必须当作"无提示"(计划 §3.6 / §5 批次 G)。
|
||||
|
||||
`float("inf")` 能被 `float()` 成功解析,又能通过既有的 `seconds > 0`——
|
||||
它会一路变成 `retry_after_s=inf`,而 `backoff_delay` 的 `max(delay, retry_after)`
|
||||
取大之后就是一次**永不醒来**的退避 sleep(库不夹 `backoff_max_s`)。
|
||||
"""
|
||||
|
||||
def _translate(self, raw, *, name="qwen_1"):
|
||||
return _translate_429(_source(name=name), "{}", {"retry-after": raw}, {"body_text": "{}"})
|
||||
|
||||
@pytest.mark.parametrize("raw", ["inf", "-inf", "1e999", "Infinity"])
|
||||
def test_non_finite_becomes_no_hint_with_exactly_one_warning(self, raw):
|
||||
messages: list[str] = []
|
||||
sink_id = logger.add(messages.append, level="WARNING")
|
||||
try:
|
||||
exc = self._translate(raw)
|
||||
finally:
|
||||
logger.remove(sink_id)
|
||||
assert exc.retry_after_s is None
|
||||
hits = [m for m in messages if "retry_after_not_finite" in m]
|
||||
assert len(hits) == 1
|
||||
assert "qwen_1" in hits[0] # 告警要能定位到源
|
||||
assert raw not in hits[0] # 但不回显原始头字符串(不拼接、不截断)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"raw", ["nan", "", " ", "-1", "0", "Wed, 21 Oct 2026 07:28:00 GMT", "soon"]
|
||||
)
|
||||
def test_other_unusable_values_stay_silent(self, raw):
|
||||
"""429 风暴下逐次告警会淹掉真信号: 只有非有限值这一类新增告警。
|
||||
|
||||
`nan` 仍走既有的 `seconds > 0` 恒假值语义,本版**不为它新增分支**。
|
||||
"""
|
||||
messages: list[str] = []
|
||||
sink_id = logger.add(messages.append, level="WARNING")
|
||||
try:
|
||||
exc = self._translate(raw)
|
||||
finally:
|
||||
logger.remove(sink_id)
|
||||
assert exc.retry_after_s is None
|
||||
assert [m for m in messages if "retry_after_not_finite" in m] == []
|
||||
|
||||
def test_finite_positive_still_reaches_backoff(self):
|
||||
"""有限正数一字不改地保留,并照常参与 `max(delay, retry_after)` 取大。"""
|
||||
exc = self._translate("2.5")
|
||||
assert exc.retry_after_s == 2.5
|
||||
policy = RetryPolicy(max_attempts=3, backoff_base_s=0.001, backoff_max_s=0.01)
|
||||
assert backoff_delay(policy, 1, exc, lambda: 0.5) == 2.5
|
||||
# 而非有限值被吃掉之后,退避退回纯指数,不会变成永不醒来的 sleep
|
||||
assert backoff_delay(policy, 1, self._translate("inf"), lambda: 0.5) < 1.0
|
||||
|
||||
def test_source_name_is_a_required_keyword(self):
|
||||
"""私有函数的必填 kw: 漏传即 `TypeError`,不给默认值掩盖调用点漏改。"""
|
||||
with pytest.raises(TypeError):
|
||||
_parse_retry_after("2.5")
|
||||
assert _parse_retry_after("2.5", source_name="qwen_1") == 2.5
|
||||
|
||||
def test_insufficient_quota_is_still_source_dead(self):
|
||||
"""分类判据不受本次改动影响(告警只加在 429 限速那一支)。"""
|
||||
body = json.dumps({"error": {"type": "insufficient_quota"}})
|
||||
exc = _translate_429(_source(), body, {"retry-after": "inf"}, {"body_text": body})
|
||||
assert isinstance(exc, SourceDeadError)
|
||||
|
||||
@@ -72,7 +72,9 @@ class _DummyMw:
|
||||
|
||||
|
||||
class _DummyTransport:
|
||||
async def complete(self, *, messages, source, stream, overlay, call_id, reasoning_effort):
|
||||
async def complete(
|
||||
self, *, messages, source, stream, overlay, call_id, reasoning_effort, first_token_event
|
||||
):
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
@@ -124,6 +126,18 @@ class _DummyRecorder:
|
||||
reasoning_tokens,
|
||||
tenant_id,
|
||||
meta,
|
||||
thinking_observation,
|
||||
reasoning_effort,
|
||||
scope,
|
||||
operation,
|
||||
logical_call_id,
|
||||
event_kind,
|
||||
http_status_code,
|
||||
error_type,
|
||||
cause_type,
|
||||
error_body,
|
||||
attempts,
|
||||
total_latency_ms,
|
||||
) -> None: ...
|
||||
|
||||
|
||||
@@ -275,8 +289,47 @@ class TestTelemetryRecorderSignature:
|
||||
params = inspect.signature(TelemetryRecorder.record_llm_call).parameters
|
||||
assert {"tenant_id", "meta"} <= set(params)
|
||||
|
||||
def test_call_observability_fields_are_declared(self):
|
||||
"""1.3.5 十列进协议(issue #19/#23);字段总数以实测为准不凭记忆。
|
||||
|
||||
本签名同时是装配闸的事实源(`_assert_recorder_shape` 按它派生参数名),
|
||||
故它与实现一旦漂移,下游自定义 recorder 会在装配期就被拒。
|
||||
"""
|
||||
import inspect
|
||||
|
||||
params = inspect.signature(TelemetryRecorder.record_llm_call).parameters
|
||||
assert {
|
||||
"scope",
|
||||
"operation",
|
||||
"logical_call_id",
|
||||
"event_kind",
|
||||
"http_status_code",
|
||||
"error_type",
|
||||
"cause_type",
|
||||
"error_body",
|
||||
"attempts",
|
||||
"total_latency_ms",
|
||||
} <= set(params)
|
||||
assert len(params) - 1 == 36 # 减掉 self
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"name", ["tenant_id", "meta", "thinking_observation", "reasoning_effort"]
|
||||
"name",
|
||||
[
|
||||
"tenant_id",
|
||||
"meta",
|
||||
"thinking_observation",
|
||||
"reasoning_effort",
|
||||
"scope",
|
||||
"operation",
|
||||
"logical_call_id",
|
||||
"event_kind",
|
||||
"http_status_code",
|
||||
"error_type",
|
||||
"cause_type",
|
||||
"error_body",
|
||||
"attempts",
|
||||
"total_latency_ms",
|
||||
],
|
||||
)
|
||||
def test_caller_dimensions_have_no_default(self, name):
|
||||
import inspect
|
||||
|
||||
@@ -169,7 +169,7 @@ def _source(model="qwen-max"):
|
||||
class TestEmitterCost:
|
||||
async def test_success_row_costed(self):
|
||||
rec = _MemoryRecorder()
|
||||
emitter = TelemetryEmitter(rec, pricing=_TABLE, text_cap=None)
|
||||
emitter = TelemetryEmitter(rec, pricing=_TABLE, text_cap=None, scope="LLM")
|
||||
await emitter.emit_attempt(
|
||||
request=_REQ,
|
||||
source=_source(),
|
||||
@@ -178,18 +178,19 @@ class TestEmitterCost:
|
||||
response=_resp(),
|
||||
error=None,
|
||||
reasoning_applies=True,
|
||||
operation="chat",
|
||||
)
|
||||
assert rec.rows[0]["cost"] == pytest.approx(7.2)
|
||||
|
||||
async def test_cache_hit_row_costs_zero(self):
|
||||
rec = _MemoryRecorder()
|
||||
emitter = TelemetryEmitter(rec, pricing=_TABLE, text_cap=None)
|
||||
await emitter.emit_cache_hit(request=_REQ, response=_resp(cache_hit=True))
|
||||
emitter = TelemetryEmitter(rec, pricing=_TABLE, text_cap=None, scope="LLM")
|
||||
await emitter.emit_cache_hit(request=_REQ, response=_resp(cache_hit=True), operation="chat")
|
||||
assert rec.rows[0]["cost"] == 0.0
|
||||
|
||||
async def test_failure_row_cost_none(self):
|
||||
rec = _MemoryRecorder()
|
||||
emitter = TelemetryEmitter(rec, pricing=_TABLE, text_cap=None)
|
||||
emitter = TelemetryEmitter(rec, pricing=_TABLE, text_cap=None, scope="LLM")
|
||||
await emitter.emit_attempt(
|
||||
request=_REQ,
|
||||
source=_source(),
|
||||
@@ -198,12 +199,13 @@ class TestEmitterCost:
|
||||
response=None,
|
||||
error="TransientError: boom",
|
||||
reasoning_applies=True,
|
||||
operation="chat",
|
||||
)
|
||||
assert rec.rows[0]["cost"] is None
|
||||
|
||||
async def test_unknown_model_none_without_blocking(self):
|
||||
rec = _MemoryRecorder()
|
||||
emitter = TelemetryEmitter(rec, pricing=_TABLE, text_cap=None)
|
||||
emitter = TelemetryEmitter(rec, pricing=_TABLE, text_cap=None, scope="LLM")
|
||||
await emitter.emit_attempt(
|
||||
request=_REQ,
|
||||
source=_source(model="mystery"),
|
||||
@@ -212,13 +214,14 @@ class TestEmitterCost:
|
||||
response=_resp(model="mystery"),
|
||||
error=None,
|
||||
reasoning_applies=True,
|
||||
operation="chat",
|
||||
)
|
||||
assert rec.rows[0]["cost"] is None
|
||||
|
||||
async def test_no_pricing_keeps_none(self):
|
||||
"""未注入价格表 = M1 现状: cost 恒 None(回归)。"""
|
||||
rec = _MemoryRecorder()
|
||||
emitter = TelemetryEmitter(rec, text_cap=None)
|
||||
emitter = TelemetryEmitter(rec, text_cap=None, scope="LLM")
|
||||
await emitter.emit_attempt(
|
||||
request=_REQ,
|
||||
source=_source(),
|
||||
@@ -227,5 +230,6 @@ class TestEmitterCost:
|
||||
response=_resp(),
|
||||
error=None,
|
||||
reasoning_applies=True,
|
||||
operation="chat",
|
||||
)
|
||||
assert rec.rows[0]["cost"] is None
|
||||
|
||||
+282
-4
@@ -5,6 +5,7 @@
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import dataclasses
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -75,18 +76,82 @@ class FakeTransport:
|
||||
self.script = list(script)
|
||||
self.calls = []
|
||||
self.efforts = []
|
||||
# 取消用例的确定性窗口: 进入 hang 分支即置位, 用例据此取消而非 sleep 猜时长
|
||||
self.entered = asyncio.Event()
|
||||
|
||||
async def complete(self, *, messages, source, stream, overlay, call_id, reasoning_effort):
|
||||
async def complete(
|
||||
self, *, messages, source, stream, overlay, call_id, reasoning_effort, first_token_event
|
||||
):
|
||||
self.calls.append((source.name, call_id))
|
||||
self.efforts.append(reasoning_effort)
|
||||
action = self.script.pop(0)
|
||||
if isinstance(action, Exception):
|
||||
raise action
|
||||
if action == "hang":
|
||||
self.entered.set()
|
||||
await asyncio.Event().wait()
|
||||
return action
|
||||
|
||||
|
||||
class _GenClockTransport:
|
||||
"""委托 FakeTransport 的薄包装: 每次调用返回前按脚本推进注入钟(1.3.7 批次 C)。
|
||||
|
||||
generation_ms 的口径是"只计 transport 调用本身",故推进必须发生在被包
|
||||
transport 内部;退避耗时由用例自带的 sleep 闭包推进,与本包装无关。
|
||||
"""
|
||||
|
||||
def __init__(self, script, advances, clock):
|
||||
self._inner = FakeTransport(script)
|
||||
self._advances = list(advances)
|
||||
self._clock = clock
|
||||
|
||||
@property
|
||||
def calls(self):
|
||||
return self._inner.calls
|
||||
|
||||
async def complete(
|
||||
self, *, messages, source, stream, overlay, call_id, reasoning_effort, first_token_event
|
||||
):
|
||||
advance = self._advances.pop(0)
|
||||
result = await self._inner.complete(
|
||||
messages=messages,
|
||||
source=source,
|
||||
stream=stream,
|
||||
overlay=overlay,
|
||||
call_id=call_id,
|
||||
reasoning_effort=reasoning_effort,
|
||||
first_token_event=first_token_event,
|
||||
)
|
||||
self._clock.advance(advance)
|
||||
return result
|
||||
|
||||
|
||||
class HangingGate(InMemoryGate):
|
||||
"""在指定记账写回处永久挂起的门控: 把"取消落在某个 await 上"变成确定性事件。
|
||||
|
||||
只覆盖 `record_success` / `record_failure` 两个写回点, 其余行为沿用真实内存实现。
|
||||
"""
|
||||
|
||||
def __init__(self, *, hang_on, **kwargs):
|
||||
super().__init__(**kwargs)
|
||||
self._hang_on = hang_on
|
||||
self.entered = asyncio.Event()
|
||||
|
||||
async def _hang(self):
|
||||
self.entered.set()
|
||||
await asyncio.Event().wait()
|
||||
|
||||
async def record_success(self, entry, *, count_attempt=True):
|
||||
if self._hang_on == "success":
|
||||
await self._hang()
|
||||
return await super().record_success(entry, count_attempt=count_attempt)
|
||||
|
||||
async def record_failure(self, entry, reason, force_open):
|
||||
if self._hang_on == "failure":
|
||||
await self._hang()
|
||||
return await super().record_failure(entry, reason, force_open)
|
||||
|
||||
|
||||
class FakeSleep:
|
||||
"""记录退避时长,立即返回(不真等)。"""
|
||||
|
||||
@@ -108,6 +173,9 @@ def _harness(
|
||||
rng=lambda: 0.0,
|
||||
selector=None,
|
||||
pacer=None,
|
||||
gate=None,
|
||||
transport=None,
|
||||
sleep=None,
|
||||
):
|
||||
clock = clock or FakeClock()
|
||||
limiter = InMemoryLimiter(
|
||||
@@ -117,9 +185,9 @@ def _harness(
|
||||
lease_ttl_s=100.0,
|
||||
now=clock,
|
||||
)
|
||||
gate = InMemoryGate(config=_BREAKER, now=clock)
|
||||
transport = FakeTransport(script)
|
||||
sleep = FakeSleep()
|
||||
gate = gate if gate is not None else InMemoryGate(config=_BREAKER, now=clock)
|
||||
transport = transport if transport is not None else FakeTransport(script)
|
||||
sleep = sleep if sleep is not None else FakeSleep()
|
||||
mw = RetryMW(
|
||||
scope="llm",
|
||||
sources=sources,
|
||||
@@ -466,6 +534,111 @@ class TestScopeUnavailable:
|
||||
assert resp.content == "ok" and released["done"]
|
||||
|
||||
|
||||
class TestCancellationSettlement:
|
||||
"""1.3.6 §6.3 结算矩阵: 取消时 `settle()` 的取值只由"该刻库知道什么"决定。
|
||||
|
||||
取消窗口一律用真实 `asyncio.Event` 钉死(不用 sleep 撞窗口), 否则红绿都不可信。
|
||||
"""
|
||||
|
||||
async def test_cancel_in_flight_keeps_the_reservation(self):
|
||||
"""S3: transport 在途被取消 → 端口已开始、用量未知 → 保留预扣(不凭空退款)。"""
|
||||
mw, limiter, _, transport, *_ = _harness(
|
||||
[_src("a", max_concurrency=1, tpm=1000, est_tokens=400)], ["hang"]
|
||||
)
|
||||
task = asyncio.ensure_future(mw(_REQ))
|
||||
await transport.entered.wait()
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
stats = await limiter.source_stats("a")
|
||||
assert stats.tpm_used == 400 # est 保留, 而非退成 0
|
||||
assert stats.inflight == 0
|
||||
|
||||
async def test_cancel_after_usage_known_keeps_real_usage(self):
|
||||
"""S4: 真实 usage 已算出后被取消 → 结算仍是真实值, 不被 est 覆写。"""
|
||||
clock = FakeClock()
|
||||
gate = HangingGate(hang_on="success", config=_BREAKER, now=clock)
|
||||
mw, limiter, *_ = _harness(
|
||||
[_src("a", max_concurrency=1, tpm=1000, est_tokens=400)],
|
||||
[_ok()],
|
||||
clock=clock,
|
||||
gate=gate,
|
||||
)
|
||||
task = asyncio.ensure_future(mw(_REQ))
|
||||
await gate.entered.wait()
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
assert (await limiter.source_stats("a")).tpm_used == 15 # 10+5 实测
|
||||
|
||||
async def test_cancel_in_dead_failure_branch_keeps_full_refund(self):
|
||||
"""S5-dead: 源已判死时的既有 `0` 不得因取消退化成 est(不得继续占额度)。"""
|
||||
clock = FakeClock()
|
||||
gate = HangingGate(hang_on="failure", config=_BREAKER, now=clock)
|
||||
mw, limiter, *_ = _harness(
|
||||
[_src("a", max_concurrency=1, tpm=1000, est_tokens=400)],
|
||||
[SourceDeadError("401", source_name="a", status_code=401)],
|
||||
clock=clock,
|
||||
gate=gate,
|
||||
)
|
||||
task = asyncio.ensure_future(mw(_REQ))
|
||||
await gate.entered.wait()
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
assert (await limiter.source_stats("a")).tpm_used == 0
|
||||
|
||||
async def test_cancel_in_transient_failure_branch_keeps_the_reservation(self):
|
||||
"""S5-transient: 瞬时失败的结算决定在首个 await 之前定死, 取消拿到同一个 est。"""
|
||||
clock = FakeClock()
|
||||
gate = HangingGate(hang_on="failure", config=_BREAKER, now=clock)
|
||||
mw, limiter, *_ = _harness(
|
||||
[_src("a", max_concurrency=1, tpm=1000, est_tokens=400)],
|
||||
[TransientError("boom", source_name="a", status_code=500)],
|
||||
clock=clock,
|
||||
gate=gate,
|
||||
)
|
||||
task = asyncio.ensure_future(mw(_REQ))
|
||||
await gate.entered.wait()
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
assert (await limiter.source_stats("a")).tpm_used == 400
|
||||
|
||||
async def test_unclassified_exception_still_refunds_in_full(self):
|
||||
"""S8 防越界: 未分类异常(无 except 接住)仍逐字走 1.3.5 的全额退还。"""
|
||||
mw, limiter, *_ = _harness(
|
||||
[_src("a", max_concurrency=1, tpm=1000, est_tokens=400)], [RuntimeError("boom")]
|
||||
)
|
||||
with pytest.raises(RuntimeError):
|
||||
await mw(_REQ)
|
||||
stats = await limiter.source_stats("a")
|
||||
assert stats.tpm_used == 0 and stats.inflight == 0
|
||||
|
||||
async def test_real_zero_usage_success_settles_zero(self):
|
||||
"""防越界: 真实 usage 恰为 0 是**已知事实**, 不得被当成"未知"改按 est 结算。"""
|
||||
zero = dataclasses.replace(_ok(), prompt_tokens=0, completion_tokens=0)
|
||||
mw, limiter, *_ = _harness([_src("a", tpm=1000, est_tokens=400)], [zero])
|
||||
await mw(_REQ)
|
||||
assert (await limiter.source_stats("a")).tpm_used == 0
|
||||
|
||||
async def test_circuit_open_rejection_settles_zero_end_to_end(self):
|
||||
"""S1 端到端: 开路拒绝的 pick 预扣后按 0 结算, 不给 tpm_used 增加任何量。"""
|
||||
clock = FakeClock()
|
||||
script = [TransientError(str(i)) for i in range(9)]
|
||||
mw, limiter, *_ = _harness(
|
||||
[_src("a", max_concurrency=1, tpm=10000, est_tokens=400)],
|
||||
script,
|
||||
clock=clock,
|
||||
max_attempts=99,
|
||||
)
|
||||
# 3 次瞬时失败后 a 开路 → 第 4 次 pick 被拒绝
|
||||
with pytest.raises(CircuitOpenError):
|
||||
await mw(_REQ)
|
||||
# 三次瞬时失败各保留 est = 1200; 开路那次 pick 若漏了 settle(0) 会再 +400
|
||||
assert (await limiter.source_stats("a")).tpm_used == 1200
|
||||
|
||||
|
||||
class TestCancellation:
|
||||
async def test_cancel_mid_flight_releases_permit(self):
|
||||
mw, limiter, _, _, _, _ = _harness([_src("a", max_concurrency=1)], ["hang"])
|
||||
@@ -776,3 +949,108 @@ class TestRateLimitPushback:
|
||||
await mw(_REQ)
|
||||
assert ei.value.reason == "retry_exhausted"
|
||||
assert len(transport.calls) == 3
|
||||
|
||||
|
||||
class TestGenerationMs:
|
||||
"""裸生成时间(1.3.7 H8): 只计 transport 调用本身,不含退避/准入/遥测收尾。"""
|
||||
|
||||
def _ctx(self, clock):
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
return _CallContext(now=clock)
|
||||
|
||||
async def test_generation_ms_excludes_backoff_and_admission(self):
|
||||
"""[Transient, ok] 脚本: 退避推进 5s、成功次 transport 推进 0.25s。
|
||||
|
||||
generation_ms 恒等于成功次 transport 的 250ms;若口径混入了退避,
|
||||
它会涨到 5250ms 量级——与 total_latency_ms 的下界断言互为对偶。
|
||||
推进量取二进制可精确表示值(0.25/5.0): int 截断下 0.2 之类会因浮点
|
||||
误差落到 199,断言随之抖动(同 test_types 既有用例只用 1.5/2.0 的惯例)。
|
||||
"""
|
||||
clock = FakeClock()
|
||||
|
||||
async def advancing_sleep(seconds):
|
||||
clock.advance(seconds)
|
||||
|
||||
transport = _GenClockTransport(
|
||||
[TransientError("boom", source_name="a"), _ok()], [0.0, 0.25], clock
|
||||
)
|
||||
mw, *_ = _harness(
|
||||
[_src("a")],
|
||||
[],
|
||||
clock=clock,
|
||||
transport=transport,
|
||||
sleep=advancing_sleep,
|
||||
rng=lambda: 2.0, # backoff = 2.0 * (0.5 + 2.0) = 5.0s
|
||||
)
|
||||
ctx = self._ctx(clock)
|
||||
resp = await mw(dataclasses.replace(_REQ, call_context=ctx))
|
||||
assert resp.content == "ok" and len(transport.calls) == 2
|
||||
stats = ctx.snapshot()
|
||||
assert stats.generation_ms == 250
|
||||
assert stats.total_latency_ms >= 5250
|
||||
|
||||
async def test_generation_ms_zero_hedge_flags_without_hedging(self):
|
||||
"""无对冲时 hedges/hedge_won 恒 0/False(对冲登记是 T3 的事)。"""
|
||||
mw, *_ = _harness([_src("a")], [_ok()])
|
||||
ctx = self._ctx(FakeClock())
|
||||
await mw(dataclasses.replace(_REQ, call_context=ctx))
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 0 and stats.hedge_won is False
|
||||
|
||||
|
||||
class TestLogicalAttemptCounting:
|
||||
"""尝试登记在 transport 调用**之前**(1.3.5 设计 §4)。
|
||||
|
||||
登记点若挪到成功之后,失败与取消的尝试就会从计数里消失——而那正是
|
||||
诊断时最需要看见的几次。
|
||||
"""
|
||||
|
||||
def _ctx(self, clock):
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
return _CallContext(now=clock)
|
||||
|
||||
async def test_single_success_counts_one(self):
|
||||
mw, _, _, _, _, clock = _harness([_src("a")], [_ok()])
|
||||
ctx = self._ctx(clock)
|
||||
await mw(dataclasses.replace(_REQ, call_context=ctx))
|
||||
assert ctx.snapshot().attempts == 1
|
||||
|
||||
async def test_failed_retries_are_counted(self):
|
||||
"""两次可重试失败 + 一次成功 = 3 次尝试,不是 1 次。"""
|
||||
mw, _, _, transport, _, clock = _harness(
|
||||
[_src("a")], [TransientError("t1"), TransientError("t2"), _ok()]
|
||||
)
|
||||
ctx = self._ctx(clock)
|
||||
await mw(dataclasses.replace(_REQ, call_context=ctx))
|
||||
assert ctx.snapshot().attempts == 3 == len(transport.calls)
|
||||
|
||||
async def test_budget_free_429_still_counts_as_an_attempt(self):
|
||||
"""429 免的是重试预算,不是"没发生过"——它确实打到了网关。"""
|
||||
mw, _, _, transport, _, clock = _harness(
|
||||
[_src("a")],
|
||||
[
|
||||
TransientError("t1", status_code=429, retry_after_s=1.0),
|
||||
TransientError("t2", status_code=429, retry_after_s=1.0),
|
||||
_ok(),
|
||||
],
|
||||
)
|
||||
ctx = self._ctx(clock)
|
||||
await mw(dataclasses.replace(_REQ, call_context=ctx))
|
||||
assert ctx.snapshot().attempts == 3 == len(transport.calls)
|
||||
|
||||
async def test_retry_exhausted_counts_every_attempt(self):
|
||||
mw, _, _, transport, _, clock = _harness(
|
||||
[_src("a")], [TransientError(str(i)) for i in range(5)], max_attempts=3
|
||||
)
|
||||
ctx = self._ctx(clock)
|
||||
with pytest.raises(AllSourcesExhausted):
|
||||
await mw(dataclasses.replace(_REQ, call_context=ctx))
|
||||
assert ctx.snapshot().attempts == 3 == len(transport.calls)
|
||||
|
||||
async def test_absent_context_does_not_break_the_call(self):
|
||||
"""库内现场构造的 `ChatRequest` 没有上下文,不得因此报错(设计 §3.5)。"""
|
||||
mw, _, _, _, _, _ = _harness([_src("a")], [_ok()])
|
||||
resp = await mw(_REQ)
|
||||
assert resp.content == "ok" and _REQ.call_context is None
|
||||
|
||||
+609
-81
File diff suppressed because it is too large
Load Diff
@@ -610,3 +610,193 @@ class TestSourceConfigEffortNormalization:
|
||||
"""非字符串同样只能是 `ValueError`: 公共入口不许把类型错误漏成 `AttributeError`。"""
|
||||
with pytest.raises(ValueError, match="推理档位"):
|
||||
_make_source(reasoning_effort=3)
|
||||
|
||||
|
||||
class TestCallStatsAndContext:
|
||||
"""逻辑调用统计内核(1.3.5 设计 §3/§4)。"""
|
||||
|
||||
def test_call_stats_is_frozen_snapshot(self):
|
||||
from polygateway.types import CallStats
|
||||
|
||||
stats = CallStats(logical_call_id="lc-1", attempts=2, total_latency_ms=15)
|
||||
with pytest.raises(dataclasses.FrozenInstanceError):
|
||||
stats.attempts = 3
|
||||
|
||||
def test_callstats_hedge_fields_default(self):
|
||||
"""1.3.7 三字段全带默认值: 仅旧三参数构造不炸,无对冲恒 0/0/False。"""
|
||||
from polygateway.types import CallStats
|
||||
|
||||
stats = CallStats(logical_call_id="lc-1", attempts=2, total_latency_ms=15)
|
||||
assert stats.hedges == 0
|
||||
assert stats.generation_ms == 0
|
||||
assert stats.hedge_won is False
|
||||
explicit = CallStats(
|
||||
logical_call_id="lc-2",
|
||||
attempts=2,
|
||||
total_latency_ms=15,
|
||||
hedges=1,
|
||||
generation_ms=42,
|
||||
hedge_won=True,
|
||||
)
|
||||
assert (explicit.hedges, explicit.generation_ms, explicit.hedge_won) == (1, 42, True)
|
||||
|
||||
def test_callcontext_record_generation_overwrite_and_accumulate(self):
|
||||
"""chat/OCR 覆盖(结构化重问最后一轮为准);embedding 分批累加。"""
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
ctx = _CallContext(now=_FakeMonotonic())
|
||||
ctx.record_generation(100, accumulate=False)
|
||||
ctx.record_generation(30, accumulate=False)
|
||||
assert ctx.snapshot().generation_ms == 30
|
||||
ctx.record_generation(50, accumulate=True)
|
||||
assert ctx.snapshot().generation_ms == 80
|
||||
|
||||
def test_callcontext_register_hedge_counts(self):
|
||||
"""对冲路实际发出即计数;赢家裁定后一次性登记赢家身份。"""
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
ctx = _CallContext(now=_FakeMonotonic())
|
||||
ctx.register_hedge(hedge_won=False)
|
||||
ctx.register_hedge(hedge_won=True)
|
||||
stats = ctx.snapshot()
|
||||
assert stats.hedges == 2 and stats.hedge_won is True
|
||||
|
||||
def test_snapshot_includes_hedge_fields(self):
|
||||
"""快照把三字段带出: 裸生成时间与对冲计数不停留在内部状态里。"""
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
ctx = _CallContext(now=_FakeMonotonic())
|
||||
ctx.record_generation(200, accumulate=False)
|
||||
ctx.register_hedge(hedge_won=True)
|
||||
stats = ctx.snapshot()
|
||||
assert (stats.hedges, stats.generation_ms, stats.hedge_won) == (1, 200, True)
|
||||
|
||||
def test_context_counts_attempts_and_freezes_elapsed(self):
|
||||
"""快照是同步冻结的时间切片: 登记两次尝试后耗时按注入钟折算成毫秒。"""
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
clock = _FakeMonotonic()
|
||||
ctx = _CallContext(now=clock)
|
||||
clock.advance(1.5)
|
||||
ctx.register_attempt()
|
||||
ctx.register_attempt()
|
||||
stats = ctx.snapshot()
|
||||
assert stats.attempts == 2
|
||||
assert stats.total_latency_ms == 1500 # 秒→毫秒,不混用单位
|
||||
|
||||
def test_snapshot_is_repeatable_and_tracks_later_time(self):
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
clock = _FakeMonotonic()
|
||||
ctx = _CallContext(now=clock)
|
||||
first = ctx.snapshot()
|
||||
clock.advance(2.0)
|
||||
second = ctx.snapshot()
|
||||
assert first.total_latency_ms == 0 and second.total_latency_ms == 2000
|
||||
assert first.logical_call_id == second.logical_call_id
|
||||
|
||||
def test_each_context_gets_its_own_logical_id(self):
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
clock = _FakeMonotonic()
|
||||
assert _CallContext(now=clock).logical_call_id != _CallContext(now=clock).logical_call_id
|
||||
|
||||
def test_claim_terminal_is_true_once(self):
|
||||
"""终态去重位: 保证每逻辑调用至多写一条终态行(设计 §6 不变量 I3)。"""
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
ctx = _CallContext(now=_FakeMonotonic())
|
||||
assert ctx.claim_terminal() is True
|
||||
assert ctx.claim_terminal() is False
|
||||
|
||||
def test_chat_request_context_does_not_affect_equality_or_repr(self):
|
||||
"""上下文是库内部件: 进 `compare`/`repr` 会污染既有请求语义与日志。"""
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
ctx = _CallContext(now=_FakeMonotonic())
|
||||
bare = ChatRequest(messages=[{"role": "user", "content": "hi"}])
|
||||
with_ctx = dataclasses.replace(bare, call_context=ctx)
|
||||
assert with_ctx.call_context is ctx
|
||||
assert with_ctx == bare
|
||||
assert "call_context" not in repr(with_ctx)
|
||||
|
||||
def test_replace_preserves_the_same_context_reference(self):
|
||||
"""洋葱各层经 `replace` 派生请求,上下文必须是同一实例而非拷贝。"""
|
||||
from polygateway.types import _CallContext
|
||||
|
||||
ctx = _CallContext(now=_FakeMonotonic())
|
||||
req = ChatRequest(messages=[{"role": "user", "content": "hi"}], call_context=ctx)
|
||||
derived = dataclasses.replace(req, stream=False)
|
||||
assert derived.call_context is ctx
|
||||
|
||||
def test_four_responses_default_call_stats_to_none(self):
|
||||
"""第三方合成响应的 `None` 表示未知,不得伪造 0(设计 §3)。"""
|
||||
from polygateway.types import (
|
||||
EmbeddingResponse,
|
||||
OcrLayoutResult,
|
||||
OcrTextResult,
|
||||
)
|
||||
|
||||
llm = LLMResponse(
|
||||
content="c",
|
||||
thinking="",
|
||||
model="m",
|
||||
provider="p",
|
||||
prompt_tokens=1,
|
||||
completion_tokens=1,
|
||||
latency_ms=1,
|
||||
ttft_ms=None,
|
||||
max_inter_token_ms=None,
|
||||
cache_hit=False,
|
||||
call_id="c1",
|
||||
)
|
||||
emb = EmbeddingResponse(
|
||||
vectors=[],
|
||||
dim=0,
|
||||
model="m",
|
||||
provider="p",
|
||||
prompt_tokens=0,
|
||||
usage_source="measured",
|
||||
latency_ms=0,
|
||||
call_id="c2",
|
||||
source_name="s",
|
||||
)
|
||||
text = OcrTextResult(
|
||||
text="", source_name="s", usage=Usage(0, 0), latency_ms=0, call_id="c3", raw={}
|
||||
)
|
||||
layout = OcrLayoutResult(
|
||||
elements=[],
|
||||
page_sizes=[],
|
||||
source_name="s",
|
||||
usage=Usage(0, 0),
|
||||
latency_ms=0,
|
||||
call_id="c4",
|
||||
raw={},
|
||||
)
|
||||
assert (llm.call_stats, emb.call_stats, text.call_stats, layout.call_stats) == (
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
|
||||
def test_call_stats_is_exported_from_package_root(self):
|
||||
"""四份平铺字段会漂移,故统计以单一对象出现在公共 API(设计 §3)。"""
|
||||
import polygateway
|
||||
from polygateway.types import CallStats
|
||||
|
||||
assert polygateway.CallStats is CallStats
|
||||
assert "CallStats" in polygateway.__all__
|
||||
|
||||
|
||||
class _FakeMonotonic:
|
||||
"""确定性单调钟;不复用 contracts 的 FakeClock 以免 unit 反向依赖契约包。"""
|
||||
|
||||
def __init__(self, start: float = 1000.0) -> None:
|
||||
self.t = start
|
||||
|
||||
def __call__(self) -> float:
|
||||
return self.t
|
||||
|
||||
def advance(self, seconds: float) -> None:
|
||||
self.t += seconds
|
||||
|
||||
@@ -35,6 +35,7 @@ from polygateway.types import (
|
||||
USAGE_SOURCES,
|
||||
BackpressurePolicy,
|
||||
BreakerConfig,
|
||||
CallStats,
|
||||
ChatRequest,
|
||||
EmbeddingTransportResult,
|
||||
GlobalLimits,
|
||||
@@ -44,6 +45,9 @@ from polygateway.types import (
|
||||
SourceConfig,
|
||||
)
|
||||
|
||||
# 终态行的快照入参(1.3.5): `emit_terminal_failure` 不再收 `latency_ms`。
|
||||
_MIGRATED_STATS = CallStats(logical_call_id="lcid-mig", attempts=1, total_latency_ms=1)
|
||||
|
||||
_REQ = ChatRequest(messages=[{"role": "user", "content": "hi"}])
|
||||
_DOMAIN = sorted(USAGE_SOURCES)
|
||||
|
||||
@@ -135,6 +139,7 @@ async def test_salvage_override_stays_in_domain(usage):
|
||||
overlay={},
|
||||
call_id="cid",
|
||||
reasoning_effort=None,
|
||||
first_token_event=None,
|
||||
)
|
||||
assert result.usage_source in USAGE_SOURCES
|
||||
|
||||
@@ -255,7 +260,7 @@ def _resp(usage_source):
|
||||
@pytest.mark.parametrize("emitted", _DOMAIN)
|
||||
async def test_emit_attempt_success_stays_in_domain(emitted):
|
||||
recorder = _MemoryRecorder()
|
||||
await TelemetryEmitter(recorder, text_cap=None).emit_attempt(
|
||||
await TelemetryEmitter(recorder, text_cap=None, scope="LLM").emit_attempt(
|
||||
request=_REQ,
|
||||
source=_src(),
|
||||
call_id="cid",
|
||||
@@ -263,6 +268,7 @@ async def test_emit_attempt_success_stays_in_domain(emitted):
|
||||
response=_resp(emitted),
|
||||
error=None,
|
||||
reasoning_applies=True,
|
||||
operation="chat",
|
||||
)
|
||||
assert recorder.rows[0]["usage_source"] in USAGE_SOURCES
|
||||
|
||||
@@ -270,7 +276,7 @@ async def test_emit_attempt_success_stays_in_domain(emitted):
|
||||
async def test_emit_attempt_failed_attempt_stays_in_domain():
|
||||
"""失败尝试无 response,`usage_source` 取 emitter 自己的字面量。"""
|
||||
recorder = _MemoryRecorder()
|
||||
await TelemetryEmitter(recorder, text_cap=None).emit_attempt(
|
||||
await TelemetryEmitter(recorder, text_cap=None, scope="LLM").emit_attempt(
|
||||
request=_REQ,
|
||||
source=_src(),
|
||||
call_id="cid",
|
||||
@@ -278,6 +284,7 @@ async def test_emit_attempt_failed_attempt_stays_in_domain():
|
||||
response=None,
|
||||
error="boom",
|
||||
reasoning_applies=True,
|
||||
operation="chat",
|
||||
)
|
||||
assert recorder.rows[0]["usage_source"] in USAGE_SOURCES
|
||||
|
||||
@@ -285,8 +292,10 @@ async def test_emit_attempt_failed_attempt_stays_in_domain():
|
||||
@pytest.mark.parametrize("emitted", _DOMAIN)
|
||||
async def test_emit_cache_hit_stays_in_domain(emitted):
|
||||
recorder = _MemoryRecorder()
|
||||
await TelemetryEmitter(recorder, text_cap=None).emit_cache_hit(
|
||||
request=_REQ, response=_resp(emitted)
|
||||
await TelemetryEmitter(recorder, text_cap=None, scope="LLM").emit_cache_hit(
|
||||
request=_REQ,
|
||||
response=_resp(emitted),
|
||||
operation="chat",
|
||||
)
|
||||
assert recorder.rows[0]["usage_source"] in USAGE_SOURCES
|
||||
|
||||
@@ -294,7 +303,11 @@ async def test_emit_cache_hit_stays_in_domain(emitted):
|
||||
async def test_emit_terminal_failure_stays_in_domain():
|
||||
"""终态失败无具体源,`usage_source` 同样取 emitter 字面量。"""
|
||||
recorder = _MemoryRecorder()
|
||||
await TelemetryEmitter(recorder, text_cap=None).emit_terminal_failure(
|
||||
request=_REQ, call_id="cid", latency_ms=10, error="cancelled"
|
||||
await TelemetryEmitter(recorder, text_cap=None, scope="LLM").emit_terminal_failure(
|
||||
request=_REQ,
|
||||
call_id="cid",
|
||||
error="cancelled",
|
||||
operation="chat",
|
||||
stats=_MIGRATED_STATS,
|
||||
)
|
||||
assert recorder.rows[0]["usage_source"] in USAGE_SOURCES
|
||||
|
||||
Reference in New Issue
Block a user