Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 iomgaa
|
bd82f16921 | ||
|
iomgaa
|
2c9c5a3aa7 | ||
|
iomgaa
|
9185eab8c9 |
@@ -200,3 +200,7 @@ PGW_TELEMETRY_BACKEND=none # sqlite | postgres | none(必填)
|
||||
# SOAK_OCR__GLOBAL__MAX_CONCURRENCY=16
|
||||
# SOAK_OCR__GLOBAL__RPM=300
|
||||
# SOAK_OCR_FAULT_SOURCES=monkey_3,monkey_4
|
||||
|
||||
# OpenAI 保真模式是 per-call response_mode="openai"，无新增环境键。
|
||||
# raw-only reasoning_effort 放调用 overlay；对应源不得设置 ENABLE_THINKING 或
|
||||
# REASONING_EFFORT，请求 reasoning_effort=None 只继承源配置，不撤销源级意图。
|
||||
|
||||
@@ -1,5 +1,15 @@
|
||||
# Changelog
|
||||
|
||||
## 1.4.0(2026-10-01)
|
||||
|
||||
- 新增 `chat(response_mode="openai")`，返回 `LLMResponse.completion`：assistant 消息、工具调用、finish_reason、上游 response_id、usage 快照；支持流式聚合及非流式。
|
||||
- 修复 SSE 顶层 error 后接 DONE 被当成功、同帧 content/reasoning_content 丢失；工具实际进展纳入看门狗、首 token 与中段对冲。
|
||||
- 保真仅支持单 choice，拒绝 structured 组合；工具参数逐字拼接，不执行或修复工具；缺 DONE 不允许 salvage。
|
||||
- 保真缓存独立身份并完整回放；缺协议、畸形协议及无正文无工具的坏条目均按 miss 回源。**旧文本缓存可能含曾被吞错的半截结果，受影响集合须在升级首次读写前更换 namespace/salt（含 per-call 覆盖）**，库不会自动辨认旧污染。
|
||||
- 新模式不绕过 managed/raw 冲突校验；raw-only 仍须源级/请求级均无受管意图。`Transport.complete` 新增必填 `response_mode`，自定义实现需适配。
|
||||
- 分发采用 Apache-2.0；README 补充常驻事件循环线程桥接，不新增同步 API。
|
||||
|
||||
|
||||
## 1.3.8(2026-09-14)
|
||||
|
||||
流式中段沉默的计量与应对(issue #27 / #28):`generation_ms` 口径重定义为**净生成时间**(扣除所有超阈值中段沉默)并配 `stall_ms`/`stall_count` 归因字段与遥测两列;另加一条**默认关闭**的中段沉默对冲(主路首 token 之后相邻增量 chunk 间隔超阈值时,向异源并发再发一次)。两键均不设时行为逐字等于 1.3.7(默认关闭回归门: 既有 unit/contracts 全套件语义断言一行未改)。
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -19,6 +19,7 @@
|
||||
| 调用期限 | 一次逻辑调用可选一条**墙钟硬边界**(`{SCOPE}__CALL_DEADLINE_S` 或 `chat(call_deadline_s=...)`,缺省不启用):治理的是**等待**——重试退避、配额轮询、熔断冷却、结构化重问与 embedding 分批共享同一份期限。三条须知:①**返回时刻 = 期限 + 清理耗时**(遥测/结算/缓存收尾在 `finally` 里跑完,允许超期;实测构造达期限的 5–7 倍),库只承诺切断等待、不给返回上界;②**到期 ≠ 未产出、≠ 未计费**,上游可能已算完并计费;③**不配置即逐字保持 1.3.5 语义**(纯 429 序列仍可能长等、有限大 `Retry-After` 仍照睡)。到期抛 `CallDeadlineExceeded`,**不属四分类、不属 `GatewayUnavailableError` 族** |
|
||||
| 长尾对冲 | **默认关闭**的可选加速(`{SCOPE}__HEDGE__AFTER_S`,仅 chat):一次尝试挂起超阈值时向**异源**并发再发一次,先回者赢、输家取消。流式以「首 token 未至」为触发判据(不误杀慢生成),非流式只有纯总时长阈值;对冲走完整限流/熔断准入,拿不到配额静默放弃。成本须知:开启即用配额换延迟(触发窗口内 in-flight 翻倍),输家取消按 est 保留预扣且**可能已被上游计费**(取消止不住上游);输家不喂熔断,遥测 attempt 行标 `error="hedge_cancelled"` 可按 `logical_call_id` join 对账;两路皆败只计一次重试预算;与 `call_deadline_s` 正交组合(装配守卫强制对冲阈值 < 期限)
|
||||
| 中段沉默对冲(可选,超业界现状能力) | **默认关闭**(`{SCOPE}__HEDGE__STALL_AFTER_S`,仅流式 chat):主路**首 token 之后**相邻增量 chunk 沉默超阈值时向异源并发再发一次,与首 token 对冲 `HEDGE__AFTER_S` **独立生效**、可独设可组合;仅设它且请求非流式 → 逐字走单路(中段沉默无非流式概念)。**诚实标注:中段沉默触发并发副本在主流网关/库中无公开先例**(业界收敛为看门狗掐流重试)。成本须知:下游实测沉默 >20s 占 19.0%,触发率远高于首 token 对冲,开启即用配额换延迟,输家可能已被上游计费;看门狗掐死**喂**熔断而对冲输家取消**不喂**(不对称声明),详见[1.3.8 生成时间口径与中段沉默对冲](#138-生成时间口径与中段沉默对冲) |
|
||||
| OpenAI 消息保真(1.4.0) | `chat(response_mode="openai")` 返回完整 assistant 消息/工具调用、finish_reason、上游 ID 与 usage；单 choice，复用治理，不执行工具；默认仍为文本模式 |
|
||||
| 响应缓存 | Redis/内存;key 含 model + messages 摘要 + namespace(缓存隔离单位)+ salt + 采样参数 + 请求级推理档位(同 messages 跑 low 与 max 不互相命中),多模态 content 先摘要再 hash(防毒化);可 per-call 绕过(科研重采样) |
|
||||
| 流式看门狗 | TTFT / inter-token / 总超时三层活性;thinking token 刷活性不计结果;截断流(缺 `[DONE]`)判瞬时不入缓存 |
|
||||
| 推理可观测性 | "这次到底推理没推理"由多信号裁定(推理正文压倒 usage 明细),三态落在 `LLMResponse.thinking_observation`:`observed` / `absent` / `unknown`——**`unknown` 是"本次判不出",不是"没推理"**;本次实发档位与实测观测矛盾时按 `(源, 模型, 生效档位)` 各告警一次(能力表过期、开启未生效、注入了却观测不到;同一模型的 low 与 max 是两个独立的矛盾,不共用节流键);裁定结果随遥测落库 |
|
||||
@@ -137,7 +138,7 @@ stats.total_latency_ms # 含缓存 IO、退避、准入等待、重
|
||||
| `{SCOPE}__STREAM__STALL_THRESHOLD_MS` | 正整数毫秒 | **1000** | 仅改计量口径,不改任何行为;三层调整接口同权(env 键 / `GatewaySettings.stall_threshold_ms` / `GatewayClient(stall_threshold_ms=...)`);无 per-call 覆盖,不做自适应(防与对冲阈值耦出反馈环) |
|
||||
| `{SCOPE}__HEDGE__STALL_AFTER_S` | 有限正数秒 | **未设 = 关闭** | 独立于 `HEDGE__AFTER_S` 生效;守卫: `≥ min(各源 inter_token_timeout_s 或 timeout_s)`(对冲窗永不开)或 `≥ call_deadline_s` → 装配期 **ValueError**,单源 scope → warning;三层调整接口同权(env 键 / `GatewaySettings.hedge_stall_after_s` / `GatewayClient(hedge_stall_after_s=...)`);Embedding/OCR scope 配任一对冲键 → warning 后静默退化 |
|
||||
|
||||
对冲语义(两阶段触发窗/复检防误触发/取消穿透)与成本披露见 [CHANGELOG 未发布段](CHANGELOG.md);运维「挂起率」统计须知不对称:看门狗掐死**喂**熔断失败计数,对冲输家取消**不喂**熔断。
|
||||
对冲语义(两阶段触发窗/复检防误触发/取消穿透)与成本披露见 [CHANGELOG 1.3.8 段](CHANGELOG.md);运维「挂起率」统计须知不对称:看门狗掐死**喂**熔断失败计数,对冲输家取消**不喂**熔断。
|
||||
|
||||
## 安装
|
||||
|
||||
@@ -145,7 +146,7 @@ stats.total_latency_ms # 含缓存 IO、退避、准入等待、重
|
||||
|
||||
```bash
|
||||
pip install --extra-index-url https://gitea.iomgaa.online/api/packages/iomgaa/pypi/simple/ \
|
||||
"polygateway[redis,postgres,structured]>=1.3.8,<2"
|
||||
"polygateway[redis,postgres,structured]>=1.4.0,<2"
|
||||
```
|
||||
|
||||
核心仅依赖 `httpx` + `pydantic`;按需选 extras:
|
||||
@@ -610,3 +611,72 @@ make ci # 只读全量验证
|
||||
## 兼容性承诺
|
||||
|
||||
`LLMResponse` 等被下游消费的公共类型,字段**只增不删不改名**且新增字段必带默认值;`{SCOPE}__{PROVIDER}__{N}__{FIELD}` 与平铺韧性键名(`LLM_TIMEOUT` 等)沿用三项目既有习惯,不做破坏性改名。实验室内部库,随实验室项目需求演进。
|
||||
|
||||
## 1.4.0 OpenAI 工具消息保真（单 choice）
|
||||
|
||||
`chat(..., response_mode="openai")` 仍返回 `LLMResponse`，完整协议结果在
|
||||
`response.completion`：`message`、`finish_reason`、`response_id`、`usage`。
|
||||
`content/thinking` 是旧文本投影（可剥离 think 标签）；回传代理或继续对话应使用
|
||||
completion.message，不能从旧投影重建工具响应。嵌套 JSON 是独立快照但仍可修改。
|
||||
|
||||
```python
|
||||
response = await client.chat(
|
||||
messages,
|
||||
response_mode="openai",
|
||||
overlay={"tools": tools, "tool_choice": "auto"},
|
||||
)
|
||||
completion = response.completion
|
||||
messages.append(completion.message)
|
||||
# 工具执行与 role=tool 结果由调用方添加；库不会执行工具。
|
||||
```
|
||||
|
||||
保真是 JSON 字段和值保真，不是 HTTP 字节透明代理。仅支持单 choice（`n=1`），
|
||||
可含多个并行 function tool_calls；arguments 逐字拼接，不解析、修复或执行。
|
||||
SSE 支持 role/content/reasoning_content/tool_calls，未知增量字段会明确失败；
|
||||
非流式保留整个 message 扩展字段，不返回顶层 logprobs/created 等其他字段。
|
||||
上游实际模型仍读 `model_reported`，配置别名是 `model`。
|
||||
|
||||
| 边界 | 行为 |
|
||||
|---|---|
|
||||
| 空文本且有完整工具 | 正常成功；不完整工具或无任何有效产物仍失败 |
|
||||
| error 帧 | 所有模式立即判失败并走已有重试；后续 DONE 不挽救 |
|
||||
| 缺 DONE | 保真模式不受 salvage 放宽，防截断工具被执行 |
|
||||
| finish_reason=length | 原样返回，不修补参数；调用方自行判断是否执行 |
|
||||
| structured | 不可与保真并用；自行传 overlay.response_format 不启用库的修复/重问阶梯 |
|
||||
| raw 推理 | 保真不是校验逃生口；raw-only 要求源 ENABLE_THINKING/REASONING_EFFORT 均未设置，请求档为 None，再通过 overlay.reasoning_effort 传原值 |
|
||||
|
||||
工具实际增量与正文/推理共用首 token、inter-token、stall 对冲及沉默计量；注释、
|
||||
usage、role-only、重复工具身份不算进展。非流式无中段事件，保持纯时长阈值对冲。
|
||||
`Transport.complete` 的自定义实现须接收必填 `response_mode`；不支持保真时明确
|
||||
RequestRejected，不能伪造 completion 或静默退回文本。
|
||||
|
||||
缓存按返回模式隔离；缺协议、畸形协议或无正文无工具的条目按 miss 回源。命中保留历史上游 response_id/usage，生成本次治理 call_id，
|
||||
本次 attempts=0、cost=0。**旧文本缓存可能含修复前 error+DONE 的截断产物**：
|
||||
受影响调用集合须在首次读写前更换从未承载旧语义的 namespace 或 salt，包含工厂、
|
||||
全量注入和 per-call 覆盖，保留租户前缀与 epoch。未迁移仍可能读旧值，库不自动清除。
|
||||
遥测继续单一出口，不新增工具全文列；既有 TEXT_CAP 不覆盖 tool_calls.arguments。
|
||||
|
||||
### 线程模型的异步桥接
|
||||
|
||||
库不新增同步客户端。socketserver 等工作线程应向**一个常驻事件循环线程**提交：
|
||||
client 必须在该 loop 内创建、使用并关闭，不能每请求 asyncio.run 或跨 loop 共用
|
||||
httpx/Redis 组件。下例的 loop 和 client 已由宿主完成初始化：
|
||||
|
||||
```python
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
client.chat(messages, response_mode="openai", overlay=overlay), loop
|
||||
)
|
||||
try:
|
||||
response = future.result(timeout=timeout_s)
|
||||
except TimeoutError:
|
||||
future.cancel() # 断连也应取消；取消不等于上游未计费。
|
||||
raise
|
||||
```
|
||||
|
||||
应用退出时先停止接单、取消/等待在途调用，在同一个 loop 中 await client.aclose()，
|
||||
再停 loop 并 join 线程；注入组件仍由所有者关闭。不要从 loop 本身阻塞调用 future.result。
|
||||
|
||||
## 许可证
|
||||
|
||||
PolyGateway 使用 [Apache License 2.0](LICENSE)。内部移植代码的原创 owner 已明确授权；
|
||||
分发范围不包含 `reference/` 外部参考仓。
|
||||
|
||||
+2
-1
@@ -4,11 +4,12 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "polygateway"
|
||||
version = "1.3.8"
|
||||
version = "1.4.0"
|
||||
description = "PolyGateway：实验室统一的大语言模型（LLM/VLM/OCR）调度与中转库——多源、限流、重试、熔断、缓存、遥测"
|
||||
# registry 包页面的正文只认这一项:缺了页面就是一片空白(1.1.2 的教训,twine 会警告
|
||||
# long_description missing 但不阻塞上传)。README 在打包时被固化进产物,发布后再改无效。
|
||||
readme = "README.md"
|
||||
license = {file = "LICENSE"}
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"httpx>=0.27",
|
||||
|
||||
@@ -789,3 +789,20 @@ src/polygateway/
|
||||
| Q6 | CHSAnalyzer 的 judge 迁移路径 | **已拍板(2026-07-22 用户)**: M4 实测 judge/core-eval 评估流水线**零调用方、从未接线**(全仓仅自测消费),且实验室网关无 claude 系模型——本轮**豁免不动**,judge.py 原样保留;待评估流水线真正启用时再收编走库(届时裁判模型从网关现有模型选) |
|
||||
| Q4 | conda 环境名 | `PolyGateway` |
|
||||
| Q5 | 本仓库工程脚手架(git init、`.claude/` skills、Makefile、pyproject、import-linter)何时落地 | 本文档终审通过后、M1 编码前一次落地 |
|
||||
|
||||
|
||||
## 14. OpenAI 单 choice 消息保真（2026-10-01，#29 已批准增量）
|
||||
|
||||
本节明确修订 §5/§6.2/§7.1/§7.2/§7.5/§7.6 中原仅文本契约，详细边界见 [设计](designs/2026-10-01-issue29-openai-fidelity-design.md)。不新增 Agent Loop 或第二套治理。
|
||||
|
||||
| 接缝 | 增量契约 |
|
||||
|---|---|
|
||||
| 公共返回 | chat 追加 response_mode=text/openai，默认 text；LLMResponse/TransportResult 末尾追加 completion，内含 assistant message、finish_reason、response_id、usage；旧字段继续文本兼容投影 |
|
||||
| 请求所有权 | messages 字段保真；overlay 仍受 model/messages/stream/stream_options 保护；raw-only 复用既有路径，模式不绕过 managed 校验 |
|
||||
| 协议 | openai 单 choice；工具按 index 排序，name/arguments 逐片拼接；SSE 支持 role/content/reasoning_content/tool_calls，未知增量拒绝；非流式保留消息扩展字段；成功必含 assistant role |
|
||||
| 失败 | 所有 chat 模式 error 帧立即 TransientError；openai 缺 DONE 一律重试、不 salvage；tool-only 成功，纯空/不完整工具失败；text salvage 原语义不变 |
|
||||
| 活性 | 同帧正文/推理/工具实际进展只记一次；role/usage/finish/空帧/重复身份不算；沿既有看门狗、首 token、stall 对冲与唯一取整点 |
|
||||
| 缓存 | 仅 openai key 新增 response_mode 语义维度，默认 text key 不变；完整快照显式重建，缺 completion/role 或无有效正文/工具按 miss（与 transport 共用内核纯值判据，不依赖 transport 实现）；历史上游 id 回放，本次治理 id 重建 |
|
||||
| 迁移 | 旧 text 污染无法自动判断，受影响集合必须迁移 namespace/salt；新模式天然隔离；无新遥测列/表，统一 helper 与取消结算不变 |
|
||||
|
||||
本增量不会执行工具；length 结束原因与参数原样交回，由下游判断可执行性。嵌套 JSON 是独立可变快照，不承诺深冻结。许可已由原创 owner 明确批准 Apache-2.0。
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
---
|
||||
type: design
|
||||
node_id: design:2026-10-01-issue29-openai-fidelity-design
|
||||
title: OpenAI消息保真设计
|
||||
date: 2026-10-01
|
||||
---
|
||||
|
||||
# #29 OpenAI 消息保真设计
|
||||
|
||||
> 状态：方向已获人类批准；计划独立审查 P1/P2 已修订并实施，首轮独立代码审查发现的缓存空产物及测试双向依赖已修复，等待父会话最终复核；1.4.0 发布准备，尚未发布。
|
||||
> 基线：`0e29deac1a72bfee348ed2f8061ea2c33be217e8` / 1.3.8；日期：2026-10-01。
|
||||
> 需求：[issue #29](https://gitea.iomgaa.online/iomgaa/PolyGateway/issues/29)。
|
||||
> 权威约束：ARCHITECTURE §2.3、§5、§6、§7.1–7.8；实现前按本文修订对应架构段落。
|
||||
|
||||
## 1. 批准记录与范围
|
||||
|
||||
用户已明确同意：现有 `chat()` 增加显式 OpenAI 保真模式，`LLMResponse` 增加完整协议结果，复用治理链，修复工具聚合、活性、error 帧和缓存隔离；首版单 choice、不执行工具、不增加 Agent Loop。用户随后确认内部移植代码均为其原创且本人 owner，并明确授权 PolyGateway 使用 Apache-2.0；dissect3 也是其项目。以上不再重复请示。
|
||||
|
||||
Codex 不可用，人类已授权原生 subagent 替代独立审查。具体 API 拼写与防御约束由本设计落实，经独立审查后实施；若审查要求改变已批准边界，才重新讨论。设计阶段已结束；实现验证记录见计划末尾，独立代码审查/发布前禁止宣称 issue 已解决。
|
||||
|
||||
“保真”是受支持 OpenAI Chat Completions 消息字段和值保真，不是 HTTP 字节级透明代理，不覆盖任意供应商增量方言。model、stream、stream_options 仍由库控制；没有逐块回调、工具执行、同步客户端、美元预算或失败用量取证新功能。
|
||||
|
||||
## 2. 源码核验与方案比较
|
||||
|
||||
| 已核实接缝 | 现状与修复目标 |
|
||||
|---|---|
|
||||
| `transports/openai_compat.py::_sse_delta` | 只读首 choice；content 早返回丢同帧 reasoning；忽略 tool_calls/error |
|
||||
| `_complete_stream` / `_complete_once` | 纯文本判空；raw 只存 usage；需新增协议快照与 tool-only 成功语义 |
|
||||
| `client.py::chat` / `_build_payload` | messages 已直传，overlay 已合并，无需白名单重写 |
|
||||
| `thinking.py::validate_thinking_raw` | raw-only 已支持；受管与 raw 冲突是既定保护，不得绕过 |
|
||||
| `middleware/retry.py::_build_response` | 唯一响应构造出口，新增字段在此搬运，不新增治理循环 |
|
||||
| `middleware/cache.py` | 已有完整消息/overlay 身份，但嵌套新类型需重建、模式需隔离 |
|
||||
|
||||
| 方案 | 优势 | 代价/结论 |
|
||||
|---|---|---|
|
||||
| A：chat 显式模式 + LLMResponse 内嵌协议结果 | 生命周期、取消、缓存、遥测和对冲单一实现；兼容旧投影 | 需定义组合约束和缓存身份；**选定** |
|
||||
| B：新增 chat_completion 入口，复用私有核心 | 接口像 OpenAI | 重复公共签名、输入校验和终态封装，收益不足，否决 |
|
||||
| C：独立 SDK 客户端或代理自聚合 | 局部改动直观 | 无法自动解决响应上浮/缓存/线路错误，复制治理，否决 |
|
||||
|
||||
## 3. 公共契约
|
||||
|
||||
### 3.1 模式与类型
|
||||
|
||||
在 `types.py` 定义并从包根导出 `ResponseMode` 和 `OpenAIChatCompletion`：
|
||||
|
||||
```python
|
||||
ResponseMode = Literal["text", "openai"]
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OpenAIChatCompletion:
|
||||
"""单 choice 的协议消息快照；嵌套 JSON 并非深度不可变。"""
|
||||
message: dict[str, Any]
|
||||
finish_reason: str | None
|
||||
response_id: str | None
|
||||
usage: dict[str, Any] | None
|
||||
```
|
||||
|
||||
`GatewayClient.chat` 末尾追加 keyword-only `response_mode: ResponseMode = "text"`；`ChatRequest` 末尾同字段同默认。`TransportResult`、`LLMResponse` 末尾追加 `completion: OpenAIChatCompletion | None = None`，已有字段不改名、不改位置。保真模式成功时 completion 必非 None；文本模式不承诺提供 completion。
|
||||
|
||||
`Transport.complete` 追加必填 keyword-only `response_mode: ResponseMode`，依照现有端口纪律不使用默认值掩盖漏传。默认 transport 实现和测试 transport 一并适配；不新增 SDK 适配器（当前文件树尚无该实现）。自定义 transport 须实现新参数与返回契约，不支持模式时抛 RequestRejectedError，不静默回退。
|
||||
|
||||
completion 是独立于上游容器的深拷贝 JSON 快照；frozen 只保证字段不能重绑，不伪称深度不可变。调用方可用 `copy.deepcopy(response.completion.message)` 构造下一轮；其修改不得污染别次调用、缓存条目或另一路对冲。不上 MappingProxyType，避免与既有 dataclasses.asdict/JSON 序列化冲突。
|
||||
|
||||
### 3.2 请求与组合校验
|
||||
|
||||
| 条件 | 规则 |
|
||||
|---|---|
|
||||
| 未知模式 | 公共入口 ValueError，尚未创建逻辑调用上下文 |
|
||||
| openai + structured 非 None | 公共入口 ValueError；不注入 schema、不自动重问；overlay 的 response_format 仍可直传 |
|
||||
| openai + n | 仅允许缺省或严格整数 1（不接受 bool）；调用 overlay 在入口校验，选中源 extra_body 及最终 payload 在 HTTP 前校验，不接受被 overlay 遮蔽的非法源级 n |
|
||||
| text + n | 保留旧行为，本次不扩大旧文本多 choice 语义；明确只投影第一 choice |
|
||||
| 消息 | 保留 assistant/tool/reasoning_content/扩展字段，JSON 值不改写；保真入口校验可序列化并深拷贝，避免并发/重试期间调用方改动 |
|
||||
| overlay | 复用既有保护键、源级→请求级优先级；保真调用对快照作深拷贝，不原地改调用方容器 |
|
||||
|
||||
源级 n 错误属选源后的请求拒绝，抛 RequestRejectedError，finally 结算规则不变。保真响应必须恰好一个 choice；SSE choice.index 必须为整数 0，usage-only choices=[] 合法；非流式一个 choice 可省略 index，存在时必须为 0。上游返回多 choice 是协议失败 TransientError，不能静默截取。
|
||||
|
||||
### 3.3 推理所有权不随返回模式变化
|
||||
|
||||
复用 `effective_effort`、`resolve_thinking`、`validate_thinking_raw`，不改能力表与档位映射。raw-only 示例前提：源级 enable_thinking=None、reasoning_effort=None，请求级 reasoning_effort=None，然后 overlay 传 `{"reasoning_effort": "max"}`。`chat(reasoning_effort=None)` 是继承源，不是撤销源意图；raw 与任何有效受管意图（含 NONE）并存仍拒绝。raw-only 的 applied_effort 仍 None，不编造受管档位。
|
||||
|
||||
### 3.4 兼容投影与上游身份
|
||||
|
||||
完整协议 `message` 不过 `_finalize_text`，保留 content 的缺省/null/空串区别、空白、think 标签；非流式保留整个 message 的 JSON 扩展字段。`LLMResponse.content/thinking` 仍是旧 `_finalize_text` 投影，content 为 null/缺省时投影空串；只用于旧消费面和现有遥测，代理回传以 completion 为准。
|
||||
|
||||
response_id 是上游 id，允许未上报为 None，绝不拿 call_id 补齐；finish_reason 原样保存，缺失为 None，不从工具存在与否猜测；usage 保存原对象快照（最后一个非 null usage），不因 token 归一或 salvage 修改原文。字段存在但类型不符是协议失败。`length` 原样返回，不修复工具参数；下游决定是否执行。
|
||||
|
||||
## 4. 协议聚合与完成规则
|
||||
|
||||
新增内部模块 `transports/_openai_completion.py`，只处理 chat 单 choice 协议校验、每尝试聚合与快照；不做 I/O、重试、计时、provider 推理或缓存。现有 openai_compat 保留 HTTP、看门狗、投影、usage 归一化与领域上下文。
|
||||
|
||||
### 4.1 SSE 聚合状态
|
||||
|
||||
每次 complete 独立创建聚合器，不放 transport 实例共享状态。逐 JSON 帧先校验对象并查 error，再读取 choices/delta、元信息和 usage；坏类型统一转 TransientError，不让 AttributeError/TypeError 逃出治理。
|
||||
|
||||
| 字段 | 合并规则 |
|
||||
|---|---|
|
||||
| content / reasoning_content | 非 null 必为 str，逐片拼接；同帧同时收集；不 trim；从未出现则不造键，仅 null 则保留 null |
|
||||
| role | openai 模式若出现必须 assistant；只出现 role 不算活性；最终成功必须含 assistant role；缺失失败，不伪造字段 |
|
||||
| tool_calls | 必为列表；每项 index 必为非负严格整数；按 index 分桶，末尾按 index 排序，索引不输出到最终工具对象 |
|
||||
| id / type | 每工具的身份字段，缺失片段不覆盖；相同重复幂等，冲突拒绝；不把重复完整 id 拼两次 |
|
||||
| function.name / function.arguments | 字符串增量按到达顺序逐字拼接，允许空片段；arguments 不 JSON 解析/修复/重排 |
|
||||
| response id / finish_reason | 首个非 null 固定，后续相同值允许、冲突拒绝；model_reported 沿用既有首有效值口径 |
|
||||
|
||||
最终有效工具需非空 id、type="function"、非空 function.name，以及至少出现一次字符串 arguments（允许空串）；重复工具 id、缺字段、非 function 工具类型都拒绝，不伪造或丢弃。index 可非连续，按排序输出，不按列表位置猜 index。JSON 参数内容及工具业务 schema 不由库验证。
|
||||
|
||||
SSE、非流式及缓存最终快照均要求 role="assistant"；缺失或其他 role 为协议失败/缓存 miss；合法 role-only 开头不算活性。
|
||||
|
||||
非流式工具是完整列表：保留条目及扩展字段，仅校验上述必要字段/类型/唯一 id，不要求 index；存在的扩展字段不删除。SSE 保真仅支持 role/content/reasoning_content/tool_calls；未知 delta 字段或未知工具增量字段无法推定合并语义，明确拒绝为协议失败，避免宣称任意扩展字段无损；元数据如 logprobs 不属于本协议结果字段，文档明确不返回。
|
||||
|
||||
### 4.2 成功、截断与错误
|
||||
|
||||
| 场景 | 行为 |
|
||||
|---|---|
|
||||
| openai：空正文 + 有完整工具 | 成功，不触发空补全重试 |
|
||||
| openai：非空投影正文、无工具 | 成功；存在任何不完整工具则仍拒绝，不用正文掩盖坏工具 |
|
||||
| openai：无正文且无有效工具 | 维持 empty_completion 瞬时失败；reasoning-only 不算成功 |
|
||||
| openai：EOF 无 DONE | 无论 missing_done=salvage、是否已有正文/工具/finish，都失败；不打捞潜在可执行结果 |
|
||||
| text：EOF 无 DONE | 保持已有 early_eof/retry/salvage 及 estimated/unavailable 行为 |
|
||||
| 任一 chat 模式出现顶层 error 键 | 立即 TransientError，即使值 null/空对象；不读取后续 usage/DONE 挽救，不缓存 |
|
||||
| 非流式 200 error / 非 JSON / 坏 envelope | TransientError；HTTP 非 200 分类沿用现有规则 |
|
||||
|
||||
text 仍不接收 tool-only 成功，不承诺工具结果；共用解析路径的 error 检查、同帧双文本采集和真实工具进展活性修复适用于两模式。纯文本消费不新增工具完整性要求，以免把未承诺的工具结果变成默认请求失败。
|
||||
|
||||
## 5. 活性、治理、取消和可观测
|
||||
|
||||
一帧有非空正文、非空推理、非空工具 name/arguments 增量或首次建立工具身份时，产生**一次**进展事件。重复相同身份、空 arguments、空 delta、role-only、usage-only、finish-only、注释不刷新活性。同帧多个字段和两个工具仍只记录一个时间戳，避免制造零间隔或虚增 stall_count。
|
||||
|
||||
`stream_with_liveness_timeouts` 继续包进展迭代器；openai_compat 用同一事件更新时间戳、TTFT、max_gap、first_token_event、content_event。保持 `_stall_from_chunk_times` 与 retry 唯一取整点不变；真正沉默照常触发中段对冲。非流式不置两事件，stall=None，只保留纯时长阈值对冲。
|
||||
|
||||
请求经原 Telemetry→Cache→Structured（None 直通）→Retry→准入→Transport 链。重试、异源对冲、失败预算、源健康、限流结算均原样复用；输家取消仍保守 est 结算、不喂熔断，赢家携自身完整快照。CancelledError 全栈穿透，连接与 permit finally 释放，无新的 shield 或后台任务。
|
||||
|
||||
遵循 structured-logging：不新增数据库列/表/Recorder 参数。`_build_response` 搬运 completion；错误由 transport 抛出，经过既有 RetryMW._emit→TelemetryEmitter 单出口记 attempt/terminal 行。SSE error 使用现有 summarize_body/compose_message 有界摘要，携 source_name/operation=chat，最多记录有界错误体，不在日志新增完整工具参数、密钥或成功消息。
|
||||
|
||||
现有 `llm_calls` schema（`research-wiki/schemas/llm-calls.md`）及覆盖指标复用。确定性验收指标：error→成功恰好 attempts=2；tool-only attempts=1；命中 attempts=0；失败不写缓存；无孤儿 permit/task。失败 usage 仍 unavailable，费用仍未知，不删除失败事件；不扩展遥测全文覆盖，tool_calls.arguments 不受现有 TEXT_CAP 截断的已知边界须继续声明。
|
||||
|
||||
## 6. 缓存语义
|
||||
|
||||
`build_cache_key` 追加 `response_mode: ResponseMode = "text"`；仅非 text 时将 response_mode 写入 key 对象。默认文本 key 逐字不变；这是请求语义维度，不引入包版本/revision。tools、tool_choice、raw effort 已在 sampling/source fingerprint；assistant/tool 消息全字段已在 digest_messages，直接复用。
|
||||
|
||||
CacheMW 显式序列化和重建 OpenAIChatCompletion，读时验证容器、单消息/工具必要字段与元信息类型。openai 条目 completion 缺失/null/畸形或无有效产物视 miss+warning，不能补默认伪造协议成功；text 旧条目允许缺新字段。transport 与缓存共用 types 内部纯值判据：完整工具或非空投影正文；无工具时还要求协议原正文非空，不能靠缓存旧投影伪装成功；不收紧快照构造用途。错误条目回源成功后覆写，失败不写。
|
||||
|
||||
缓存中的 message/usage 与调用结果独立；命中保留历史上游 id/finish/usage，生成本次 call_id、清除历史 call_stats，最终返回本次 attempts=0；cost=0 延续原口径。缓存读写和序列化失败仅 warning，不冒泡，取消例外穿透；将 `_serialize` 异常纳入缓存降级边界，避免新增嵌套字段扩大失败面。
|
||||
|
||||
旧文本缓存可能已被 error+DONE 毒化，不能自动判别。发布文档要求受影响调用集合首次读写前迁移 namespace/salt，保留租户前缀与 epoch；覆盖工厂和 per-call 身份。未迁移仍会回放旧值，不能宣称修复自动清除历史缓存；openai 新身份天然不命中旧文本。
|
||||
|
||||
## 7. 旧行为审计与非功能约束
|
||||
|
||||
| 既有行为 | 处置 |
|
||||
|---|---|
|
||||
| 旧字段/默认文本/structured 阶梯/受管推理 | 保留；默认文本同帧漏 reasoning 属修复 |
|
||||
| HTTP 分类、retry/backoff、两类对冲、TPM/熔断 | 保留，不改算法与预算 |
|
||||
| 空补全、missing_done salvage | 文本保留；openai tool-only 放行、缺 DONE 强制失败 |
|
||||
| Redis TTL、命名空间、遥测幂等与单 helper | 保留；新增模式隔离与嵌套协议结果回放 |
|
||||
| 并发与取消 | 每尝试状态隔离，重试/对冲不得复用聚合器，CancelledError 穿透 |
|
||||
| 降级方向 | 缓存/遥测 warning；准入后端 fail-closed；不新增后端 |
|
||||
| 持久化/原子性/恢复 | 仅成功后一次写完整缓存值；沿用后端原子 set；无新的任务持久化或断点状态 |
|
||||
| 幂等与重复 | 重试/对冲可能产生重复上游计费，库不执行工具，因此不产生本地工具副作用；不承诺上游生成幂等 |
|
||||
|
||||
## 8. 测试、许可证与交付门
|
||||
|
||||
单元测试复用 MockTransport/SSE/受控事件设施；新增两工具交错分片、同帧 content+reasoning、原样下一轮、raw=max、stream=False、缺 DONE、安全判错、缓存和治理组合。每个行为组须保留基线红、新实现绿日志；不以本阶段基线通过冒充新功能证据。
|
||||
|
||||
真实 Redis 只测完整协议缓存往返/隔离与取消关联契约，不 mock Lua。真实模型跑专用通用函数工具双轮冒烟（标 slow，输出 Markdown），复用 `.env` 配置与预算，不在源码硬编码凭据；网络/额度失败如实失败，不改成无覆盖成功。不默认重跑全模型推理矩阵；本次不改 thinking 能力算法。
|
||||
|
||||
许可证：采用 Apache-2.0 已授权；实施前核查实际分发文件及注明移植/参考的片段，区分参考协议事实与复制代码。reference/ 只读且不打包；AGPL 仓存在不自动污染本库，也不由授权覆盖第三方权利。发现实际复制的不兼容片段须上报，不擅自改许可证。加入标准 LICENSE 与 pyproject 元数据；只有确有通知义务才加 NOTICE，不捏造第三方来源。
|
||||
|
||||
源码 docstring、README、CHANGELOG、ARCHITECTURE 同步；无新 env 键，`.env.example` 仅补 raw-only 源配置提示（不改默认）。Gitea Wiki 当前下线，不重建过期页面。线程代理文档说明：client 在同一常驻事件循环线程内创建/使用/关闭；工作线程 run_coroutine_threadsafe，等待超时/断连取消 Future；不逐请求 asyncio.run，不跨 loop 共享 client。
|
||||
|
||||
发布由父会话在独立代码审查、全套件、相关 slow、构建包 LICENSE/元数据核验之后执行；沿 CLAUDE 发布九步，外部验证 registry/Release/包页和下游下载，不仅 bump。issue 只有在可安装产物及验收证据齐备后关闭；本阶段不触发这些动作。
|
||||
@@ -0,0 +1,18 @@
|
||||
# #29 分发来源前置核查
|
||||
|
||||
日期：2026-10-01；基线 0e29dea；在 T1 前执行。本记录不是法律认证。
|
||||
|
||||
| 范围 | 实际核查与结论 |
|
||||
|---|---|
|
||||
| 跟踪源码 | `git ls-files src` 列出 polygateway Python 模块与 py.typed，无 vendor 子树、外部 LICENSE/NOTICE、打包外部代码目录 |
|
||||
| 内部移植 | rg 检索移植/参考/Copyright/SPDX/copied/adapted：streaming、sources、Redis Lua、ports、openai_compat 的来源为三个内部项目；用户已明确均本人原创并授权 Apache-2.0 |
|
||||
| 外部参考声明 | 阅读 types.py Effort、providers.py ThinkingWire/注册表、thinking.py 能力证据段；使用的是协议词汇/参数形态/实测结论，未发现来自外部仓的成段实现或第三方版权头；不把相同协议字符串认定为复制实现 |
|
||||
| 打包边界 | pyproject 仅从 src 发现包，reference 不跟踪、不纳入分发；本次不复制外部参考实现 |
|
||||
| 许可来源 | 系统 `/usr/share/common-licenses/Apache-2.0` 提供标准许可正文；构建时核对包内文本与元数据 |
|
||||
|
||||
未发现需额外通知的实际复制片段，因此不编造 NOTICE。外部参考仓含 AGPL 不等于本库受其许可约束；用户对原创代码的授权也不能覆盖第三方。实施/审查若发现新的实际复制证据，停止并提交主管处理，不用本文替代核查。
|
||||
|
||||
## 审查意见处置
|
||||
|
||||
P1 采纳：协议快照缺 role 不可直接进入下一轮，SSE/非流式/缓存统一要求 assistant；不补造 role，默认 text 不变。
|
||||
P2 采纳：将本次来源核查前移到 T0，T6 仅复核及落实 LICENSE/打包/文档。两项均是已批准方向的正确性修订，无需新增用户授权。
|
||||
@@ -225,6 +225,21 @@
|
||||
"id": "finding:2026-09-09-134-thinking-contracts-validation",
|
||||
"label": "1.3.4 T0–T4 与 T7 确定性验证",
|
||||
"type": "finding"
|
||||
},
|
||||
{
|
||||
"id": "design:2026-10-01-issue29-openai-fidelity-design",
|
||||
"label": "OpenAI消息保真设计",
|
||||
"type": "design"
|
||||
},
|
||||
{
|
||||
"id": "plan:2026-10-01-issue29-openai-fidelity",
|
||||
"label": "OpenAI消息保真实现计划",
|
||||
"type": "plan"
|
||||
},
|
||||
{
|
||||
"id": "review:2026-10-01-issue29-cache-review",
|
||||
"label": "保真缓存审查修复与1.4.0准备",
|
||||
"type": "review"
|
||||
}
|
||||
],
|
||||
"links": [
|
||||
@@ -472,6 +487,27 @@
|
||||
"relation": "implements",
|
||||
"evidence": "1.3.5 逻辑调用统计与结构化失败诊断实施计划落地已批准设计",
|
||||
"added": "2026-09-09T13:42:00.298915+00:00"
|
||||
},
|
||||
{
|
||||
"source": "plan:2026-10-01-issue29-openai-fidelity",
|
||||
"target": "design:2026-10-01-issue29-openai-fidelity-design",
|
||||
"relation": "implements",
|
||||
"evidence": "已批准方向与独立审查P1/P2修订",
|
||||
"added": "2026-10-01T17:52:42.425248+00:00"
|
||||
},
|
||||
{
|
||||
"source": "schema:llm-calls",
|
||||
"target": "design:2026-10-01-issue29-openai-fidelity-design",
|
||||
"relation": "implements",
|
||||
"evidence": "复用既有单出口与schema，无新增列；工具error为attempt失败",
|
||||
"added": "2026-10-01T17:52:43.831331+00:00"
|
||||
},
|
||||
{
|
||||
"source": "review:2026-10-01-issue29-cache-review",
|
||||
"target": "design:2026-10-01-issue29-openai-fidelity-design",
|
||||
"relation": "informs",
|
||||
"evidence": "统一缓存与transport的有效产物判据，保持内核依赖边界",
|
||||
"added": "2026-10-01T18:27:16.434742+00:00"
|
||||
}
|
||||
]
|
||||
}
|
||||
+23
-11
@@ -1,9 +1,8 @@
|
||||
# Research Wiki 索引
|
||||
|
||||
> 自动生成，更新时间：2026-09-09 18:01 UTC
|
||||
|
||||
## design (43)
|
||||
> 自动生成，更新时间：2026-10-01 18:27 UTC
|
||||
|
||||
## design (47)
|
||||
- [1.3.4 推理意图与测试证据设计](designs/2026-09-09-134-thinking-contracts-design.md) `design:2026-09-09-134-thinking-contracts-design`
|
||||
- [2026-07-20-m1-core-design](designs/2026-07-20-m1-core-design.md) `design:2026-07-20-m1-core-design`
|
||||
- [2026-07-20-m2-distributed-design](designs/2026-07-20-m2-distributed-design.md) `design:2026-07-20-m2-distributed-design`
|
||||
@@ -25,6 +24,9 @@
|
||||
- [2026-08-24-issue15-telemetry-pool-lifecycle-design](designs/2026-08-24-issue15-telemetry-pool-lifecycle-design.md) `design:2026-08-24-issue15-telemetry-pool-lifecycle-design`
|
||||
- [2026-09-04-reasoning-effort-design](designs/2026-09-04-reasoning-effort-design.md) `design:2026-09-04-reasoning-effort-design`
|
||||
- [2026-09-09-135-call-observability-design](designs/2026-09-09-135-call-observability-design.md) `design:2026-09-09-135-call-observability-design`
|
||||
- [2026-09-09-136-call-budgets-design](designs/2026-09-09-136-call-budgets-design.md) `design:2026-09-09-136-call-budgets-design`
|
||||
- [2026-09-10-24-hedged-requests-design](designs/2026-09-10-24-hedged-requests-design.md) `design:2026-09-10-24-hedged-requests-design`
|
||||
- [2026-09-11-streaming-stall-design](designs/2026-09-11-streaming-stall-design.md) `design:2026-09-11-streaming-stall-design`
|
||||
- [est_tokens 解耦: 拆分限流预扣与遥测用量兜底(issue #2)](designs/est-tokens-decoupling.md) `design:est-tokens-decoupling`
|
||||
- [GatewaySettings 装配校验补齐(第二轮)](designs/settings-invariants-round-2.md) `design:settings-invariants-round-2`
|
||||
- [GatewaySettings 跨字段不变量守卫的生效范围](designs/settings-invariant-guards.md) `design:settings-invariant-guards`
|
||||
@@ -38,6 +40,7 @@
|
||||
- [M2.5 治理韧性: 半死源隔离与健康感知调度](designs/m25-resilience.md) `design:m25-resilience`
|
||||
- [M3 OCR 端口族设计](designs/m3-ocr.md) `design:m3-ocr`
|
||||
- [M4 迁移验证设计(GovDoc→CHS,发 v1.0)](designs/m4-migration.md) `design:m4-migration`
|
||||
- [OpenAI消息保真设计](designs/2026-10-01-issue29-openai-fidelity-design.md) `design:2026-10-01-issue29-openai-fidelity-design`
|
||||
- [stall 判定改为非生产性等待口径](designs/issue8-stall-budget.md) `design:issue8-stall-budget`
|
||||
- [响应可观测字段扩展(Issue #3)](designs/response-observability-fields.md) `design:response-observability-fields`
|
||||
- [建表前先探测,判死只认「确定写不进去」](designs/issue9-telemetry-ddl-probe.md) `design:issue9-telemetry-ddl-probe`
|
||||
@@ -48,18 +51,22 @@
|
||||
- [调用方自定义维度设计(issue #11)](designs/issue11-caller-dimensions.md) `design:issue11-caller-dimensions`
|
||||
- [采样参数透传设计(issue #4)](designs/sampling-params.md) `design:sampling-params`
|
||||
|
||||
## finding (17)
|
||||
|
||||
## finding (22)
|
||||
- [1.3.4 推理契约验证与发布准备](findings/2026-09-09-134-thinking-contracts-validation.md) `finding:2026-09-09-134-thinking-contracts-validation`
|
||||
- [1.3.5 T2/T3/T4 验收证据：36 列遥测、失败终态、PG 存储兼容与变异矩阵](findings/2026-09-09-135-call-observability-validation.md) `finding:2026-09-09-135-call-observability-validation`
|
||||
- [1.3.5 发布完成与外部验收：合并后门、registry 产物与页面亲查](findings/2026-09-09-135-release-completion.md) `finding:2026-09-09-135-release-completion`
|
||||
- [1.3.6 可选调用期限与取消结算验证记录](findings/2026-09-10-136-call-deadline-validation.md) `finding:2026-09-10-136-call-deadline-validation`
|
||||
- [1.3.8 流式中段沉默:净口径与中段对冲验证报告](findings/2026-09-11-streaming-stall-validation.md) `finding:2026-09-11-streaming-stall-validation`
|
||||
- [2026-07-20-m2-soak-workload](findings/2026-07-20-m2-soak-workload.md) `finding:2026-07-20-m2-soak-workload`
|
||||
- [2026-07-21-m25-acceptance](findings/2026-07-21-m25-acceptance.md) `finding:2026-07-21-m25-acceptance`
|
||||
- [2026-07-21-p6-soak-baseline](findings/2026-07-21-p6-soak-baseline.md) `finding:2026-07-21-p6-soak-baseline`
|
||||
- [2026-07-22-m4-acceptance](findings/2026-07-22-m4-acceptance.md) `finding:2026-07-22-m4-acceptance`
|
||||
- [2026-07-22-p7-ocr-soak](findings/2026-07-22-p7-ocr-soak.md) `finding:2026-07-22-p7-ocr-soak`
|
||||
- [2026-09-11-streaming-stall-hedging-research](findings/2026-09-11-streaming-stall-hedging-research.md) `finding:2026-09-11-streaming-stall-hedging-research`
|
||||
- [2026-10-01-issue29-license-audit](findings/2026-10-01-issue29-license-audit.md) `finding:2026-10-01-issue29-license-audit`
|
||||
- [issue #16/#17 实测: M3 推理正常,失效的是推理的可观测信号](findings/2026-08-25-thinking-observability-regression.md) `finding:2026-08-25-thinking-observability-regression`
|
||||
- [issue #18 实测: 偶发红的是安全网本身,不是被测脚本](findings/2026-08-26-issue18-shared-pg-test-isolation.md) `finding:2026-08-26-issue18-shared-pg-test-isolation`
|
||||
- [issue #24 长尾对冲请求与裸生成时间验证报告](findings/2026-09-10-24-hedged-requests-validation.md) `finding:2026-09-10-24-hedged-requests-validation`
|
||||
- [M2 verifier 三项 Important 补齐(不变量接线/网关保护/P3 验收)](findings/m2-verifier-fixes.md) `finding:m2-verifier-fixes`
|
||||
- [M2 真实数据压测: 场景矩阵与数据清单](findings/m2-soak-workload.md) `finding:m2-soak-workload`
|
||||
- [M2.5 验收: P6 同场景 58.1% → 98.96%](findings/m25-acceptance.md) `finding:m25-acceptance`
|
||||
@@ -68,10 +75,11 @@
|
||||
- [P7 OCR soak 验收: 99.73% 与 13 不变量全 PASS](findings/p7-ocr-soak.md) `finding:p7-ocr-soak`
|
||||
- [推理开关与 reasoning_tokens: 供应商实测与业界做法](findings/2026-08-02-thinking-switch-and-reasoning-tokens.md) `finding:2026-08-02-thinking-switch-and-reasoning-tokens`
|
||||
|
||||
## plan (38)
|
||||
|
||||
## plan (42)
|
||||
- [1.3.4 推理契约实施计划](plans/2026-09-09-134-thinking-contracts.md) `plan:2026-09-09-134-thinking-contracts`
|
||||
- [1.3.5 逻辑调用统计与结构化失败诊断实施计划](plans/2026-09-09-135-call-observability.md) `plan:2026-09-09-135-call-observability`
|
||||
- [1.3.6 可选调用期限与取消结算修复实施计划](plans/2026-09-10-136-call-deadline.md) `plan:2026-09-10-136-call-deadline`
|
||||
- [1.3.8 流式中段沉默:计量口径与对冲实施计划](plans/2026-09-11-streaming-stall.md) `plan:2026-09-11-streaming-stall`
|
||||
- [2026-07-20-m1-core-plan](plans/2026-07-20-m1-core-plan.md) `plan:2026-07-20-m1-core-plan`
|
||||
- [2026-07-20-m2-distributed-plan](plans/2026-07-20-m2-distributed-plan.md) `plan:2026-07-20-m2-distributed-plan`
|
||||
- [2026-07-21-m25-resilience-plan](plans/2026-07-21-m25-resilience-plan.md) `plan:2026-07-21-m25-resilience-plan`
|
||||
@@ -90,12 +98,14 @@
|
||||
- [2026-09-04-reasoning-effort](plans/2026-09-04-reasoning-effort.md) `plan:2026-09-04-reasoning-effort`
|
||||
- [est_tokens 解耦实施计划](plans/est-tokens-decoupling.md) `plan:est-tokens-decoupling`
|
||||
- [issue #18 实现计划: 权限边界替代行数快照 + --table 锁死目标](plans/2026-08-26-issue18-pg-test-isolation.md) `plan:2026-08-26-issue18-pg-test-isolation`
|
||||
- [issue #24 长尾对冲请求与裸生成时间实施计划](plans/2026-09-10-24-hedged-requests.md) `plan:2026-09-10-24-hedged-requests`
|
||||
- [issue #8 实施计划: stall 非生产性等待口径](plans/issue8-stall-budget-plan.md) `plan:issue8-stall-budget-plan`
|
||||
- [M1 核心里程碑实现计划](plans/m1-core-plan.md) `plan:m1-core-plan`
|
||||
- [M2 分布式实现计划](plans/m2-distributed.md) `plan:m2-distributed`
|
||||
- [M2.5 治理韧性实现计划](plans/m25-resilience.md) `plan:m25-resilience`
|
||||
- [M3 OCR 实现计划](plans/m3-ocr.md) `plan:m3-ocr`
|
||||
- [M4 迁移实现计划(T0-T14)](plans/m4-migration.md) `plan:m4-migration`
|
||||
- [OpenAI消息保真实现计划](plans/2026-10-01-issue29-openai-fidelity.md) `plan:2026-10-01-issue29-openai-fidelity`
|
||||
- [plan-issue14-admission-wait-policy](plans/plan-issue14-admission-wait-policy.md) `plan:plan-issue14-admission-wait-policy`
|
||||
- [响应可观测字段扩展实现计划](plans/response-observability-fields.md) `plan:response-observability-fields`
|
||||
- [实现计划: HTTP 错误响应体留存(Issue #10)](plans/issue10-error-body-retention-plan.md) `plan:issue10-error-body-retention-plan`
|
||||
@@ -109,15 +119,17 @@
|
||||
- [调用方自定义维度实现计划(issue #11)](plans/issue11-caller-dimensions.md) `plan:issue11-caller-dimensions`
|
||||
- [采样参数透传实现计划(issue #4)](plans/sampling-params-plan.md) `plan:sampling-params-plan`
|
||||
|
||||
## review (1)
|
||||
|
||||
## review (6)
|
||||
- [2026-09-11-138-branch-review](reviews/2026-09-11-138-branch-review.md) `review:2026-09-11-138-branch-review`
|
||||
- [2026-09-11-138-verifier](reviews/2026-09-11-138-verifier.md) `review:2026-09-11-138-verifier`
|
||||
- [2026-09-11-streaming-stall-design-review](reviews/2026-09-11-streaming-stall-design-review.md) `review:2026-09-11-streaming-stall-design-review`
|
||||
- [2026-09-11-streaming-stall-plan-review](reviews/2026-09-11-streaming-stall-plan-review.md) `review:2026-09-11-streaming-stall-plan-review`
|
||||
- [保真缓存审查修复与1.4.0准备](reviews/2026-10-01-issue29-cache-review.md) `review:2026-10-01-issue29-cache-review`
|
||||
- [整分支审查: issue #14 熔断等待档](reviews/issue14-branch-review.md) `review:issue14-branch-review`
|
||||
|
||||
## schema (1)
|
||||
|
||||
- [表结构: llm_calls(遥测 36 字段)](schemas/llm-calls.md) `schema:llm-calls`
|
||||
|
||||
## metric (2)
|
||||
|
||||
- [OCR 治理调用成功率与错误分类分布](metrics/ocr-call-success.md) `metric:ocr-call-success`
|
||||
- [每次调用必录覆盖率(含缓存命中/失败/取消)](metrics/call-telemetry-coverage.md) `metric:call-telemetry-coverage`
|
||||
|
||||
@@ -160,3 +160,11 @@
|
||||
- [2026-09-09 13:42 UTC] 重建索引: 100 篇页面
|
||||
- [2026-09-09 16:49 UTC] 重建索引: 101 篇页面
|
||||
- [2026-09-09 18:01 UTC] 重建索引: 102 篇页面
|
||||
- [2026-10-01 17:52 UTC] 新增 design: OpenAI消息保真设计 (design:2026-10-01-issue29-openai-fidelity-design)
|
||||
- [2026-10-01 17:52 UTC] 新增 plan: OpenAI消息保真实现计划 (plan:2026-10-01-issue29-openai-fidelity)
|
||||
- [2026-10-01 17:52 UTC] 新增边: plan:2026-10-01-issue29-openai-fidelity --implements--> design:2026-10-01-issue29-openai-fidelity-design
|
||||
- [2026-10-01 17:52 UTC] 新增边: schema:llm-calls --implements--> design:2026-10-01-issue29-openai-fidelity-design
|
||||
- [2026-10-01 17:52 UTC] 重建索引: 119 篇页面
|
||||
- [2026-10-01 18:24 UTC] 新增 review: 保真缓存审查修复与1.4.0准备 (review:2026-10-01-issue29-cache-review)
|
||||
- [2026-10-01 18:27 UTC] 新增边: review:2026-10-01-issue29-cache-review --informs--> design:2026-10-01-issue29-openai-fidelity-design
|
||||
- [2026-10-01 18:27 UTC] 重建索引: 120 篇页面
|
||||
|
||||
@@ -31,3 +31,9 @@ date: 2026-07-20
|
||||
| 存储兼容 | 真实 PG 沙箱: auto 追加十列、manual 裁剪写入、旧行新列恒 NULL、新旧进程混写共存 | 2026-09-09 实跑 30 passed(见下方证据索引) |
|
||||
|
||||
本版**新增十列**(见 schema:llm-calls),不新建表、不新增依赖。证据索引见 `findings/2026-09-09-135-call-observability-validation.md`;覆盖度真实基线仍待首次生产运行填入,不写伪百分比。
|
||||
|
||||
## #29 验收增量（2026-10-01）
|
||||
|
||||
复用指标及 llm_calls，不另建表：error→正常 attempts=2、tool-only attempts=1、
|
||||
cache_hit attempts=0；协议失败不写缓存，取消不得遗留 permit/task。
|
||||
确定性证据位于 `tests/unit/test_openai_fidelity.py`；完整协议不新增遥测正文列。
|
||||
|
||||
@@ -0,0 +1,328 @@
|
||||
---
|
||||
type: plan
|
||||
node_id: plan:2026-10-01-issue29-openai-fidelity
|
||||
title: OpenAI消息保真实现计划
|
||||
date: 2026-10-01
|
||||
---
|
||||
|
||||
# #29 OpenAI 消息保真实现计划
|
||||
|
||||
> 状态：计划及首轮代码独审已完成，P1/P2 均采纳；缓存空产物和测试双向依赖已修复，1.4.0 发布准备由本轮落实，最终独审与 T8 外部发布仍由父会话执行。
|
||||
> 设计：[批准方向的设计落实稿](../designs/2026-10-01-issue29-openai-fidelity-design.md)。
|
||||
> 目标：同一 chat 治理链返回可供下游继续工具对话的完整单 choice 协议结果，拒绝截断和 error 帧伪成功。
|
||||
> 方案：增加 response_mode 与嵌套 completion；协议解析与聚合内聚在 transport 内部模块，治理层仅传模式/搬结果；旧文本默认保持兼容。
|
||||
> 技术：Python 3.12+、frozen dataclass、httpx SSE、现有 asyncio 看门狗/对冲、pytest、真实 Redis；不加运行时依赖。
|
||||
|
||||
## 0. 工作区、审批与执行纪律
|
||||
|
||||
| 项目 | 已核验值 |
|
||||
|---|---|
|
||||
| 工作目录 | `/home/iomgaa/.config/superpowers/worktrees/PolyGateway/issue29-openai-fidelity` |
|
||||
| 分支 | `feat/issue29-openai-fidelity` |
|
||||
| 基线 | `0e29deac1a72bfee348ed2f8061ea2c33be217e8` / 1.3.8 |
|
||||
| 原工作区 | `/home/iomgaa/Projects/PolyGateway`，main 上已有未提交文档与 .pi，不得覆盖或夹带提交 |
|
||||
| 基线命令 | `conda run -n PolyGateway pytest tests/unit -q` |
|
||||
| 基线结果 | 1627 passed in 12.25s；仅单元基线，不代表集成/slow/新增功能通过 |
|
||||
|
||||
所有 Python 命令用 PolyGateway conda 环境；始终从上述隔离目录执行，确认 pytest 配置 pythonpath=src，避免误测主工作区 editable 安装。不要复制主树 `.env` 到受跟踪路径；需要真实验证时显式读取已有配置，不输出秘密。reference/ 不存在于新 worktree 的跟踪树，不为规划复制/修改参考仓。
|
||||
|
||||
技术方向、内部原创权属和 Apache-2.0 已获用户明确批准；细化设计与计划由独立 subagent 审查，Codex 不再调用。只在新产品/权利边界未获授权时暂停。父会话掌握审查、后续编码、合并/发布权限；当前任务不执行这些步骤。
|
||||
|
||||
本计划不涉及参考实现迁移，保真校验不适用；涉及既有移植流式看门狗的消费面，必须对照本库 `streaming.py`、1.3.8 stall 设计逐段保持其超时/取消/换算语义，不能借工具支持改算法。
|
||||
|
||||
## 1. 文件结构与职责锁定
|
||||
|
||||
| 路径（相对工作目录） | 动作与职责 |
|
||||
|---|---|
|
||||
| `src/polygateway/types.py` | 新 ResponseMode/OpenAIChatCompletion；末尾追加 ChatRequest.response_mode、两个响应 completion |
|
||||
| `src/polygateway/__init__.py` | 导出新公共类型；最终发布时同步版本 |
|
||||
| `src/polygateway/ports.py` | complete 增必填模式参数；说明工具进展与单 choice 契约 |
|
||||
| `src/polygateway/client.py` | chat 新模式校验、消息/overlay 快照、组合拒绝；不重写治理入口 |
|
||||
| `src/polygateway/transports/_openai_completion.py`（新） | 协议结构校验、每尝试聚合、非流式快照；无 I/O/计时 |
|
||||
| `src/polygateway/transports/openai_compat.py` | error 检查、解析器接线、投影/usage/空补全/DONE、统一进展事件 |
|
||||
| `src/polygateway/middleware/retry.py` | 模式传入、completion 搬运；无重试/对冲算法改写 |
|
||||
| `src/polygateway/middleware/cache.py` | 模式 key、嵌套序列化/重建、缺协议 miss、序列化降级 |
|
||||
| `tests/unit/test_openai_completion.py`（新） | 纯协议聚合与非流式校验的源命名测试 |
|
||||
| `tests/unit/test_openai_compat.py` | 真实 transport 的 SSE/非流式/error/raw/DONE/活性测试 |
|
||||
| `tests/unit/test_types.py`, `test_client.py`, `test_ports.py`, `test_cache.py` | 公共类型、校验、端口、缓存往返与兼容 |
|
||||
| `tests/unit/test_retry.py`, `test_hedge.py`, `test_stall_hedge.py`, `test_deadline.py`, `test_backpressure.py` | 模式传递、错误重试、工具活性/对冲、取消与结算 |
|
||||
| `tests/integration/test_governance_stack.py`, `test_redis_cache.py`, `test_redis_cross_connection.py` | 完整链路、真实缓存往返、必要 fake 签名适配 |
|
||||
| `tests/e2e/conftest.py` | 现有取证 transport 包装显式转传模式，不改变 thinking 矩阵判据 |
|
||||
| `tests/e2e/test_openai_tools_live.py`（新） | 受控少量真实工具双轮冒烟，独立 slow 用例 |
|
||||
| `tests/unit/test_package.py` | LICENSE 文件/包元数据回归 |
|
||||
| `LICENSE`, `pyproject.toml` | 标准 Apache-2.0 与打包元数据；不捆绑 reference/ |
|
||||
| `README.md`, `CHANGELOG.md`, `.env.example`, `research-wiki/ARCHITECTURE.md` | 新模式、限制、raw 配置、缓存迁移、同步桥接说明 |
|
||||
| `research-wiki/findings/2026-10-01-issue29-license-audit.md`（新） | 分发源码第三方来源核查证据与范围 |
|
||||
| `research-wiki/graph/edges.json`, `index.md`, `log.md`, `schemas/llm-calls.md`, `metrics/call-telemetry-coverage.md` | 审查通过后的设计/计划/复用遥测关系注册，不新增表或指标实体 |
|
||||
|
||||
其余测试若显式调用 complete 必须补参数，仅机械适配；不重构不相关 fixture。`thinking.py`/能力表/streaming.py 算法不在预期改动内。不要为“兼容 SDK”创建不存在的 openai_sdk.py。
|
||||
|
||||
## 2. 跨任务接口
|
||||
|
||||
T1 定义公共字段，T2–T5 均消费同一类型名，禁止再起 tool_response/raw_response 别名。
|
||||
|
||||
```python
|
||||
ResponseMode = Literal["text", "openai"]
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OpenAIChatCompletion:
|
||||
"""单 choice 协议结果，message/usage 是独立 JSON 快照。"""
|
||||
message: dict[str, Any]
|
||||
finish_reason: str | None
|
||||
response_id: str | None
|
||||
usage: dict[str, Any] | None
|
||||
```
|
||||
|
||||
公共与端口变化均是追加参数/字段：
|
||||
|
||||
```python
|
||||
# GatewayClient.chat 的末尾 keyword-only 参数；其余参数保持原样。
|
||||
response_mode: ResponseMode = "text"
|
||||
# ChatRequest 末尾字段。
|
||||
response_mode: ResponseMode = "text"
|
||||
# Transport.complete 末尾必填 keyword-only 参数。
|
||||
response_mode: ResponseMode
|
||||
# TransportResult / LLMResponse 末尾字段。
|
||||
completion: OpenAIChatCompletion | None = None
|
||||
```
|
||||
|
||||
`build_cache_key` 在已有 keyword-only 参数之后增加 `response_mode: ResponseMode = "text"`，仅非 text 时参与摘要。缓存重建只依赖内核类型/本层校验，不 import transports；若复用协议类型重建的最低形态检查，把纯 JSON 快照检查放类型自身的具名构造方法，不把 transport 的 SSE 状态机下沉到内核。
|
||||
|
||||
T2 内部模块只由同目录 transport 消费，不成为公共导出。采用以下窄接口（内部类型在该模块定义）：
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class ChatDelta:
|
||||
"""一帧提取结果；progress 表示真实生成进展。"""
|
||||
content: str
|
||||
reasoning_content: str
|
||||
progress: bool
|
||||
```
|
||||
|
||||
```python
|
||||
class ChatCompletionAccumulator:
|
||||
"""每次 transport 尝试独享的单 choice 聚合状态。"""
|
||||
def __init__(self, *, response_mode: ResponseMode) -> None: ...
|
||||
def add_chunk(self, chunk: dict[str, Any]) -> ChatDelta: ...
|
||||
def finish(self) -> OpenAIChatCompletion: ...
|
||||
|
||||
|
||||
def parse_chat_completion(body: dict[str, Any]) -> OpenAIChatCompletion: ...
|
||||
```
|
||||
|
||||
上述 `...` 是签名约定而非待交付占位实现。聚合器抛含判据的 TransientError（operation=chat）；openai_compat 负责补选中源上下文、复用有界摘要。聚合器不判断 DONE、不读时钟；transport 掌握线路完整性。text 模式可共用帧提取与进展识别，但不调用严格 openai finish，保持旧文本工具容忍边界。
|
||||
|
||||
## 3. 分任务与独立验收
|
||||
|
||||
### T0 — 审查与来源前置核查
|
||||
|
||||
- [x] T1 前已核查分发来源，见 findings/2026-10-01-issue29-license-audit.md；发现不兼容复制即停止。
|
||||
- [ ] 父会话安排全新上下文原生 subagent 审查设计/计划，重点：单 choice、未知增量字段、身份重复/冲突、raw/managed、缓存错误降级及取消。
|
||||
- [ ] 核对每条意见，修订文档并记录接受/不接受的理由；如改变用户已批边界，走主管决定，不静默改契约。
|
||||
- [ ] 在隔离树注册文档关系，复用 llm-calls schema/call-telemetry-coverage 指标；不在主树运行生成器。
|
||||
- [ ] 文档审查通过后才调用 commit skill 提交设计/计划；当前阶段禁止提交。
|
||||
|
||||
验证命令：`wc -l research-wiki/designs/2026-10-01-issue29-openai-fidelity-design.md research-wiki/plans/2026-10-01-issue29-openai-fidelity.md` → 分别 ≤400、≤1000；`git diff --check` → 无错误。Wiki 命令经 `conda run -n PolyGateway python .claude/tools/research_wiki.py --help` 核对实际参数后执行 add_entity/add_edge/rebuild_index；检查生成 diff 不改原有条目正文。
|
||||
|
||||
### T1 — 公共契约贯穿完整治理链
|
||||
|
||||
文件：types.py、__init__.py、ports.py、client.py、middleware/retry.py；单元 types/client/ports/retry；所有显式 fake complete 签名。
|
||||
|
||||
先补测试：默认 text 不变、新类型可导入、末尾带默认字段不破旧构造；非法 response_mode、openai+structured、n=2/0/True 在适用边界拒绝；保真消息与嵌套 overlay 脱离调用方容器，返回上一轮 assistant 和 role=tool 请求 JSON 逐字段相同。
|
||||
|
||||
实现：chat 校验放在 context/handler 前；选源后非法 source.extra_body.n 在 transport HTTP 前 RequestRejected；模式作为必填端口参数穿透重试/对冲，_build_response 原样搬 completion。不把模式塞进发给上游的 payload。
|
||||
|
||||
验收：调用级拒绝不发 HTTP；源级拒绝结算/释放正常；自定义 transport fake 看见准确模式；既有旧构造回归全绿。命令：
|
||||
|
||||
```bash
|
||||
conda run -n PolyGateway pytest tests/unit/test_types.py tests/unit/test_client.py tests/unit/test_ports.py tests/unit/test_retry.py -q
|
||||
```
|
||||
|
||||
先红测试须为行为断言或缺失 API 的明确证据，记录基线 SHA 与测试名；绿后提交点：`feat: add explicit OpenAI response contract`。
|
||||
|
||||
### T2 — 内聚协议解析器与 error 帧修复
|
||||
|
||||
文件：新增 _openai_completion.py、openai_compat.py；新增 test_openai_completion.py，扩展 test_openai_compat.py。
|
||||
|
||||
补测 SSE/非流式/缓存缺 role、非 assistant role；合法 role-only 起始帧后工具成功。先红测试至少含下表各组；复用现有 SSE/MockTransport，不引入第二套 HTTP fake 框架。
|
||||
|
||||
| 组 | 输入与精确预期 |
|
||||
|---|---|
|
||||
| 并行工具 | 2 工具交错 index；同帧两工具；name/arguments 分片；空白/Unicode/引号逐字相等；最终按 index 排序 |
|
||||
| 身份 | id/type 同值重复只保留一次，冲突失败；缺 id/type/name/arguments、重复 id、bool/负数 index、非 function 类型失败 |
|
||||
| 文本 | 同帧正文+推理均保留；缺省/null/空串区别；原消息 think 标签不剥，旧投影仍剥 |
|
||||
| envelope | 顶层数组/null、choices/delta/tool_calls 非容器、坏字段类型、多 choice/非零 choice index 均受控失败 |
|
||||
| 非流式 |完整消息、扩展字段、usage/finish/id 全保留；工具参数不做业务 JSON 修复；原响应对象改动不影响快照 |
|
||||
| error | 首帧 error、正文后 error+usage+DONE、error=null、error={}、salvage 配置下 error 都立即失败；text/openai 两模式参数化 |
|
||||
|
||||
复用现有 summarize_body/compose_message，断言异常与遥测诊断有界，不打印工具参数正文。默认文本模式不因无用工具字段的严格 finish 校验而新增失败，但不吞顶层 error。无新 HTTP 状态映射。
|
||||
|
||||
命令：`conda run -n PolyGateway pytest tests/unit/test_openai_completion.py tests/unit/test_openai_compat.py tests/unit/test_http_error_body.py -q` → PASS；提交点：`fix: preserve chat deltas and reject upstream error frames`。
|
||||
|
||||
### T3 — 完成语义、活性与非流式接线
|
||||
|
||||
文件：openai_compat.py、ports.py docstring；test_openai_compat.py、test_streaming.py。
|
||||
|
||||
先红：tool-only + DONE 成功 attempts=1；空文本且空工具失败；正文+坏工具不成功；openai 缺 DONE 即使 missing_done=salvage 仍失败；text salvage 保持；finish_reason=length 原样交回，不解析 arguments。非流式工具同样成功，两事件永不置位、ttft/stall=None。
|
||||
|
||||
按设计接线：一帧一进展，同帧多个字段只记一个 chunk_time；真正非空工具增量/首次身份触发事件，重复身份和 role/usage/finish/空片段不触发。看门狗仍包进展迭代器，stall 派生/净额取整不动。
|
||||
|
||||
测试以受控 Event/FakeClock 驱动，避免用几十毫秒竞争制造脆弱测试：持续工具片段撑过 TTFT/inter-token；只有注释/usage 必超时；工具首进展设置 first_token_event；真实停顿计入 max_inter_token/stall，两字段与正文相同口径。
|
||||
|
||||
命令：`conda run -n PolyGateway pytest tests/unit/test_openai_completion.py tests/unit/test_openai_compat.py tests/unit/test_streaming.py -q` → PASS；提交点：`feat: complete tool responses with unified stream liveness`。
|
||||
|
||||
### T4 — 完整协议缓存往返与隔离
|
||||
|
||||
文件：middleware/cache.py；test_cache.py、tests/integration/test_redis_cache.py。
|
||||
|
||||
先红测试：同消息 text/openai key 不同、旧默认 text key 字节不变；tools、tool_choice、raw effort、tool result、assistant arguments 改动均 miss；相同请求完整 completion round-trip，非流式扩展字段不丢，message 的 null/缺省可区别；usage 缺省/null 均为 None；openai 缺 completion/null/坏 message/坏工具条目 miss 后回源，不冒 dict 假装 dataclass。
|
||||
|
||||
检查命中历史 response_id 不变、本次 call_id 重建、attempts=0、cache_hit=true、cost=0；变更一次命中消息不污染下一次命中；旧 text 缓存继续可读。失败路径无 set；序列化/backend set 抛错 warning 后仍返回结果；CancelledError 原样穿透。
|
||||
|
||||
实现保真重建校验不能 import transport（import-linter 必守）；内核值类型可提供纯值构造校验，协议拼接算法只保留在 transport；避免两份 SSE/tool 聚合逻辑。
|
||||
|
||||
命令：
|
||||
|
||||
```bash
|
||||
conda run -n PolyGateway pytest tests/unit/test_cache.py -q
|
||||
conda run -n PolyGateway pytest tests/integration/test_redis_cache.py -q
|
||||
conda run -n PolyGateway lint-imports
|
||||
```
|
||||
|
||||
真实 Redis 专用测试 namespace，沿既有 fixture 清理自有 key，不 flush 共享 DB；未配置 skip 必须记录为未覆盖，发布前补实测。提交点：`fix: isolate and round-trip OpenAI completion caches`。
|
||||
|
||||
### T5 — 完整链路、对冲、取消与遥测不变量
|
||||
|
||||
文件：test_retry.py、test_hedge.py、test_stall_hedge.py、test_deadline.py、test_backpressure.py、tests/integration/test_governance_stack.py、test_redis_cross_connection.py、tests/e2e/conftest.py；生产 retry 仅修必要搬运漏项，不改算法。
|
||||
|
||||
先红→绿证据须覆盖真实 OpenAICompatTransport 接入 GatewayClient，而不是只用返回预制 completion 的 fake 证明解析成功：
|
||||
|
||||
| 场景 | 断言 |
|
||||
|---|---|
|
||||
| 第一次部分正文/工具→error→DONE，第二次正常 | 只返回第二次 completion，attempts=2；第一次 error 的字段不会污染结果或缓存 |
|
||||
| 工具持续进展 | 首 token 对冲及时止窗，中段 event 复位，未错误触发 |
|
||||
| 真沉默触发对冲 | 副路完整工具结果胜出，返回自己的 id/usage，输家不泄露半成品 |
|
||||
| primary 超时但副路尚在运行 | 不提前判总失败；两败仍只烧一次超级尝试预算 |
|
||||
| 流中/准入中/退避中取消与 deadline | CancelledError/CallDeadlineExceeded 类型保留，permit/inflight/task 收口，TPM 保守结算 |
|
||||
| 观测与降级 | 正常/失败/cache_hit 遥测沿单 helper；attempt 行/terminal 行口径不变；后端写失败不影响结果 |
|
||||
|
||||
命令：
|
||||
|
||||
```bash
|
||||
conda run -n PolyGateway pytest tests/unit/test_retry.py tests/unit/test_hedge.py tests/unit/test_stall_hedge.py tests/unit/test_deadline.py tests/unit/test_backpressure.py -q
|
||||
conda run -n PolyGateway pytest tests/integration/test_governance_stack.py tests/integration/test_redis_cross_connection.py -q
|
||||
```
|
||||
|
||||
不删除“不需要失败计费”对应的现有失败遥测/attempts/结算；本任务不新增失败 token 取证。提交点：`test: cover governed tool retries hedges and cancellation`。
|
||||
|
||||
### T6 — 来源审计、Apache-2.0 与用户文档
|
||||
|
||||
文件：LICENSE、pyproject.toml、test_package.py、新 license-audit finding、README、CHANGELOG、.env.example、ARCHITECTURE；设计/计划 Wiki 注册关系。
|
||||
|
||||
| 工作 | 验收 |
|
||||
|---|---|
|
||||
| 来源审计复核（初审前置 T0） | 查 tracked 分发文件的移植/参考/版权注释及具体片段；列内部原创授权记录，外部只读 reference 的路径/许可/实际使用范围；未发现不等于法律认证 |
|
||||
| 第三方义务 | 对实际复制片段核实许可、保留通知；发现 AGPL 不兼容复制须上报，不借用户授权覆盖第三方 |
|
||||
| LICENSE | 使用标准 Apache-2.0 全文，不自行重写；版权归属沿项目元数据，不猜用户真实姓名；NOTICE 仅在实际需要时新增 |
|
||||
| 包元数据 | 当前 setuptools>=68，采用 `license = {file = "LICENSE"}` 兼容现有后端；验证 wheel/sdist 包含 license，不为 PEP 639 单独提高构建依赖 |
|
||||
| 文档 | 新 chat 示例 ≤15 行；response_mode 与 raw 所有权分开；单 choice、未知 SSE 方言失败、非流式、length、无 DONE 不打捞、缓存旧值迁移、取消/线程桥接限制写清 |
|
||||
|
||||
`.env.example` 只解释 raw-only 必须去掉源级受管意图，不加模式 env 键。README 应说明“完整协议结果”仅四字段及已定义 SSE 字段，不能暗示返回所有顶层 logprobs/created 等；上游模型使用 LLMResponse.model_reported，不把 model 配置别名当真实版本。
|
||||
|
||||
ARCHITECTURE 更新 §5 类型/签名、§6.2 空补全与 error、§7.1 DONE 与工具协议、§7.2/7.6 工具活性、§7.5 缓存维度，并明确这些只是批准方向的增量，不引入 Agent Loop。站点下线，无具体 Wiki 页面可同步。
|
||||
|
||||
验证：`conda run -n PolyGateway pytest tests/unit/test_package.py -q` 先红后绿；`git diff --check`；构建验收见 T8。提交点：`docs: document faithful chat and Apache-2.0 distribution`。
|
||||
|
||||
### T7 — 少量真实工具冒烟与独立验证
|
||||
|
||||
新增 tests/e2e/test_openai_tools_live.py，复用 tests/e2e/conftest.py 的请求取证/输出机制，不为本次重跑整个 thinking 矩阵。fixture 的 complete 显式转传 response_mode 是机械接线，不改取证判分算法。
|
||||
|
||||
测试使用通用无业务领域的函数，如 echo_value；强制工具选择但工具在测试内执行纯函数，库本体不执行。流式、非流式各一次工具请求，随后带原 assistant+tool 消息做第二轮；断言 ID、finish、arguments 可用和请求消息相等。并行两工具完整逐字拼接的硬门由确定性 fixture 保证，不把模型偶尔不发两个工具解释为库聚合正确。
|
||||
|
||||
真实测试标 slow/requires_llm，预算与超时不紧于现有 `.env` 生产配置。raw=max 使用已配支持模型的 raw-only 源；不发明新模型/API。Markdown 安全报告记录模式、源/model_reported、attempts、finish、工具数、结果/失败原因；不输出 key、敏感消息或完整参数。费用/网络异常如实呈现，不宽泛 skip。
|
||||
|
||||
命令：
|
||||
|
||||
```bash
|
||||
conda run -n PolyGateway pytest -m slow tests/e2e/test_openai_tools_live.py tests/e2e/test_smoke_gateway.py -q
|
||||
make ci
|
||||
```
|
||||
|
||||
全套件由当前前台执行；若预计长跑需 tmux，禁用缓冲、保存原退出码，不在 pytest 末尾接管道。生成唯一日志路径 `tests/outputs/issue29/<任务>-<red|green>-<时间>.log`，不提交产物。无新增功能代码的当前阶段仅跑过单元基线，不预填完成框。
|
||||
|
||||
父会话安排全新上下文 verifier 独立查实际 diff 与测试输出。若修改了准入/取消/退避生产逻辑，补 `tests/integration/test_redis_governance_time.py` 和 `test_redis_cross_connection.py` 对应 slow 子集；仅机械传参/协议解析时记录无 Lua/退避算法交集的理由。慢测试 skip/失败不得当通过。
|
||||
|
||||
- [ ] 审查员检查依赖铁律、未知字段策略、旧文本回归与大对象保留成本。
|
||||
- [ ] 每项行为能链接红绿日志、命令、SHA、退出码；基线失败必须归因，不能“重跑绿了”掩盖。
|
||||
- [ ] 合并前测试/独立审查通过；提交均调用 commit skill，不夹带主树既有文档差异。
|
||||
|
||||
### T8 — 发布与 issue 外部验收（父会话持有，不由本设计任务执行）
|
||||
|
||||
发布版本由父会话按新增公共能力确定，不能在规划中冒称已发布。遵守 CLAUDE.md §4.4.1 全步骤：README/CHANGELOG/两处版本先完成 → 合并 main/push → 合并后 lint/全套件/相交 slow → tag push → 构建/twine check → registry 上传 → 实际下载核验 → Release/挂库/包页核验。
|
||||
|
||||
| 发布门 | 可接受证据 |
|
||||
|---|---|
|
||||
| 构建 | `conda run -n PolyGateway python -m build` 和 twine check 成功；解包核 LICENSE、许可元数据、completion 类型/参数/解析器均在产物中 |
|
||||
| 安装 | 在独立临时环境从私有索引下载/安装已发布版本；必要时 --no-build-isolation 避免索引无 setuptools；核来源/hash，不误用本地 editable |
|
||||
| 外部页面 | registry 包正文与仓库链接、Gitea Release/tag 实际可见；不以 POST 返回码代替最终状态 |
|
||||
| 下游 | 以发布包运行不含业务逻辑的 tool 双轮接入验收；给 dissect3 说明同 loop 线程桥接和 raw-only 配置，不在本计划越权修改其业务仓 |
|
||||
| 关闭 issue | 再查询 open issue，#29 评论附版本/验收/限制与缓存迁移说明后关闭；若出现新 issue 单独定范围，不假称本计划涵盖未来问题 |
|
||||
|
||||
## 4. 自审清单与残余边界
|
||||
|
||||
| 需求 | 覆盖任务 |
|
||||
|---|---|
|
||||
| 工具双并行、消息往返、raw=max、非流式 | T1–T3、T7 |
|
||||
| error+DONE 所有模式失败、attempts=2 | T2、T5 |
|
||||
| 缺 DONE 不 salvage 工具、length 不修复 | T3 |
|
||||
| 活性、对冲、取消、准入/结算 | T3、T5 |
|
||||
| 完整缓存、模式隔离、旧默认 key 与污染迁移 | T4、T6 |
|
||||
| Apache 授权与第三方来源、分发验证 | T6、T8 |
|
||||
| 测试先红后绿、独立审查、外部发布完成 | 各任务、T7、T8 |
|
||||
|
||||
当前残余：首轮代码独审已完成，修复后待父会话最终复核；严格 SSE 方言会拒绝未定义增量字段，不承诺任意 OpenAI 兼容网关；旧文本毒化缓存只能由使用方迁移身份；公共 dataclass 嵌套 JSON 可变，靠快照隔离而非深冻结；模型真实冒烟依赖外部可用性。以上均需在文档和验收中如实保留，不当成全部自动解决。
|
||||
|
||||
## 5. 本轮实施记录（2026-10-01，未发布）
|
||||
|
||||
P1 最终 assistant role、P2 来源核查前置均已采纳，证据见来源核查 finding。
|
||||
T1–T6 代码/文档/许可已落实；T7 工具真实冒烟、Redis、质量门已执行，独立代码审查由父会话接续；T8 禁止本 worker 执行。
|
||||
|
||||
| 验证 | 实际结果/证据 |
|
||||
|---|---|
|
||||
| 基线红 | tests/outputs/issue29/baseline-red.log：error+DONE 未抛、双文本丢 reasoning、模式 API 缺失，3 failed |
|
||||
| 扩展红绿 | expanded-red.log 19 failed（git archive 0e29dea 的 src，不改主树）；expanded-green.log 同19 passed |
|
||||
| 防御红绿 | defense-red.log 3 failed：usage 裸错误/空 tool_calls 丢失；defense-green.log 同3 passed |
|
||||
| 全量 | ci-final.log + ci-final.exit=0：1812 passed、23 skipped、110 deselected，覆盖率95%；之后仅增测试/文档，无生产变更 |
|
||||
| 最后定向 | targeted-final.log：264 passed，包含后增的非流式事件/取消/managed冲突/名称分片回归 |
|
||||
| 真实缓存 | test_redis_cache.py 6 passed，专用随机 key，无共享数据清理 |
|
||||
| 真实工具 | live-final.log + live-final.exit=0：流式/非流式双轮2 passed，101.34s；安全Markdown在 tests/outputs/issue29/live |
|
||||
| 构建 | /tmp/pgw-issue29-dist-final 的 wheel/sdist、twine check、LICENSE/源码字节核验通过；仍1.3.8本地测试产物，绝非发布 |
|
||||
|
||||
23 skips 为17个已转专用时间语义的 Redis 契约变体、6个缺私有图像语料的 OCR 测试；未把这些当本次通过。110 deselected 为默认 slow；未重跑全模型 thinking 矩阵，因为只机械转传端口参数、未改其判据或能力表。
|
||||
|
||||
未提交代码，供父会话独立验收完整工作区。conda 启动有基础环境 RequestsDependencyWarning（pytest本体无该告警），pi-lens 使用错误环境时的缺包/枚举提示已用 conda 实证排除，不据此改变依赖。
|
||||
Wiki add_entity 会覆写无frontmatter同名文件，现场检查发现后已从本会话原始 tool-write 精确恢复两文档，并加 frontmatter 后保留 P1/P2 修订；没有丢失设计内容。
|
||||
|
||||
补充最终证据：unit-final.log 1678 passed（23.84s），smoke.log + smoke.exit=0 为既有真实 chat 冒烟4 passed（97.18s）；未跑昂贵全模型矩阵。源代码最后变更早于 ci-final，后续只增测试/文档；最后单元覆盖全部新增测试。
|
||||
|
||||
|
||||
## 6. 审查修复与1.4.0准备（2026-10-01，未发布）
|
||||
|
||||
已读取首轮独立报告并核验采纳 P2 空产物缓存、Minor 测试双向依赖，详见 [审查处置](../reviews/2026-10-01-issue29-cache-review.md)。本轮只在隔离 feature 树写入，主树既有脏文档未改；不合并、不 push/tag/上传、不关issue。
|
||||
|
||||
| 本轮命令/证据（日志均在 tests/outputs/issue29，不提交） | 实际结果 |
|
||||
|---|---|
|
||||
| 新缓存回归修复前：review-cache-red.log | 10 failed、1 passed：五种无效产物×两种缓存投影误命中；合法tool-only通过 |
|
||||
| 四模块定向：review-cache-green.log | 268 passed，17.71s；包含11项新增缓存回归 |
|
||||
| 真实Redis+包测试：review-redis-package.log | 13 passed，2.66s（6 Redis、7包） |
|
||||
| 首次 lint / make ci | 测试import顺序失败；已修正，不作为通过证据 |
|
||||
| tmux make ci：review-ci-final.log / .exit=0 | 1827 passed、23 skipped、110 deselected，317.28s；95%覆盖；格式102文件、ruff通过、依赖契约1 kept/0 broken |
|
||||
| 远端 tags 与 registry simple | 最大均1.3.8，没有1.4.0；版本两处改1.4.0，README约束>=1.4.0,<2，CHANGELOG定版 |
|
||||
| build / twine / 解包：review-build-final.log、review-twine.log、review-artifacts.log | 1.4.0 wheel/sdist 均通过；所有Python源码、README、标准LICENSE逐字相同，Version/License-File正确且无reference |
|
||||
| README数字断言 | inspect.signature 实测 TelemetryRecorder.record_llm_call 为38字段，未凭记忆更新 |
|
||||
|
||||
wheel SHA256 `3186d01d7710c3b5ef727b59f95e8c2db0821277a79491ffb913dd2ce34a888d`；sdist SHA256 `b254dedf65d25673cf57986a613a1f4e9895da8b5be1e6ee241f00065a95ad90`。这些是隔离树本地构建，不是registry产物。解包探针首次使用email默认ASCII解码比较中文README产生误报，改为METADATA正文原始字节比较后通过，未改包来迎合错误探针。
|
||||
|
||||
跳过项仍为17个独立slow Redis时间变体及6个缺私有图像语料OCR；110 slow未选中。本轮不重复付费网关：复用前轮流式/非流式工具双轮2通过和chat冒烟4通过，当前生产改动只统一空产物判据，工具/文本正常行为已有定向与全套件证明；合并后slow发布门由父执行。构建有setuptools许可表格弃用提醒（兼容setuptools>=68设计），并有基础conda RequestsDependencyWarning；均不等于当前构建或测试失败。
|
||||
|
||||
按commit skill提交feature全部实现、修复、许可与发布准备，原main脏文件及运行产物不纳入。提交后交父最终独立复核；T8外部发布尚未完成，不能提前关闭#29。
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
type: review
|
||||
node_id: review:2026-10-01-issue29-cache-review
|
||||
title: 保真缓存审查修复与1.4.0准备
|
||||
date: 2026-10-01
|
||||
---
|
||||
|
||||
# 保真缓存审查修复与1.4.0准备
|
||||
|
||||
|
||||
## 审查采纳
|
||||
|
||||
| 级别/问题 | 核验与处置 |
|
||||
|---|---|
|
||||
| Important/P2：空协议缓存绕过成功判据 | 已实证：新测试在修复前 10 failed、1 passed；role-only、null/空/空白正文+空工具、reasoning-only，分别带空投影/伪造旧投影均误命中。采纳并修复。 |
|
||||
| Minor：协议/治理测试互相导入 | 已核验双向依赖；将帧构造放基础协议测试中，上层 fidelity 单向复用，没有新增 helpers 抽象。 |
|
||||
|
||||
内核 `types._has_completion_output` 供 transport 和缓存共用：先经协议类型结构校验，有完整工具即有效；无工具须原协议正文和兼容投影均非空。快照类型仍允许描述空消息，不将成功判据强塞进构造器。缓存使用已有 warning/miss，text 旧缓存行为不变。没有 middleware→transport 依赖。
|
||||
|
||||
## 验证证据
|
||||
|
||||
定向 `test_cache/test_openai_completion/test_openai_fidelity/test_openai_compat` 为 268 passed；日志 `tests/outputs/issue29/review-cache-{red,green}.log`（运行产物不提交）。完整质量门1827 passed/23 skipped/110 deselected、95%覆盖、退出0；真实Redis+包13通过；1.4.0 wheel/sdist源码/README/LICENSE逐字校验及twine通过，详见计划最新记录；首轮 lint 发现一个测试 import 顺序，修正后重新运行全门，未掩盖首次失败。
|
||||
|
||||
## 发布准备边界
|
||||
|
||||
1.4.0 对应新增公共协议支持及 transport 端口参数；2026-10-01 远端 tags 和匿名 registry simple 最大版本均1.3.8，无1.4.0冲突。README安装约束、能力表、CHANGELOG定版、两处版本同步。新增 `.env.example` 仅四行说明，无凭据。源码/许可来源核查复用已独审 finding；新产物仍须父会话最终复核后发布。本轮不合并、不 push/tag/上传、不关issue；原main脏文件未改。
|
||||
@@ -177,3 +177,10 @@ ORDER BY model, calls DESC;
|
||||
## 1.3.4 测试侧证据（不新增 schema）
|
||||
|
||||
`tests/live_evidence.py`／e2e conftest 只在内存保存完整错误体与独立非流式身份，逐轮 Markdown 白名单输出到 `tests/outputs/134/live/`；凭据、Authorization、提示词、原始异常／响应均不落报告。生产数据仍经 TelemetryEmitter。评估复用 `metric:call-telemetry-coverage`，实际 live 覆盖基线待首次执行。
|
||||
|
||||
## #29 复用声明（2026-10-01）
|
||||
|
||||
OpenAI 保真没有新增遥测表/列；仍经 RetryMW._emit → TelemetryEmitter。
|
||||
error 帧是失败 attempt，正文与思考仍为兼容投影，不新增工具全文列。
|
||||
本页原标题/表格停留 36 字段；实际 `telemetry/schema.py` 已含 stall_ms/stall_count
|
||||
共 38 个 INSERT 字段（§7.8 已登记）。本次不改 DDL，真实定义以源码为准。
|
||||
|
||||
@@ -47,12 +47,14 @@ from polygateway.types import (
|
||||
OcrLayoutElement,
|
||||
OcrLayoutResult,
|
||||
OcrTextResult,
|
||||
OpenAIChatCompletion,
|
||||
ResponseMode,
|
||||
SourceConfig,
|
||||
TelemetryStatus,
|
||||
ThinkingObservation,
|
||||
)
|
||||
|
||||
__version__ = "1.3.8"
|
||||
__version__ = "1.4.0"
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_PROFILES",
|
||||
@@ -70,6 +72,8 @@ __all__ = [
|
||||
"GatewayUnavailableError",
|
||||
"GovernanceBackendError",
|
||||
"LLMResponse",
|
||||
"ResponseMode",
|
||||
"OpenAIChatCompletion",
|
||||
"ModelPrice",
|
||||
"OcrClient",
|
||||
"OcrLayoutElement",
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import copy
|
||||
import dataclasses
|
||||
import hashlib
|
||||
import json
|
||||
@@ -48,6 +49,7 @@ from polygateway.types import (
|
||||
ChatRequest,
|
||||
Effort,
|
||||
LLMResponse,
|
||||
ResponseMode,
|
||||
TelemetryStatus,
|
||||
_CallContext,
|
||||
coerce_effort,
|
||||
@@ -372,9 +374,15 @@ class GatewayClient:
|
||||
tenant_id: str | None = None,
|
||||
meta: Mapping[str, Any] | None = None,
|
||||
call_deadline_s: float | None = None,
|
||||
response_mode: ResponseMode = "text",
|
||||
) -> LLMResponse:
|
||||
"""一次治理调用(签名冻结,ARCH §5.2;与三项目 LLMProvider 协议兼容)。
|
||||
|
||||
`response_mode="openai"` 返回 `completion` 完整单 choice 消息快照；
|
||||
不支持与 structured 并用，不执行工具。空正文但完整工具是合法结果；
|
||||
缺 [DONE] 不打捞。默认 text 保持既有文本投影。返回模式不绕过受管推理
|
||||
校验；raw-only 要求源级与请求级推理意图均为 None。
|
||||
|
||||
`overlay` 是采样参数覆盖层(`temperature`/`seed`/`max_tokens` 等),优先级
|
||||
高于源级 `extra_body`、低于结构化输出的注入。带默认值的 keyword-only
|
||||
参数不影响既有调用点(issue #4)。
|
||||
@@ -393,6 +401,23 @@ class GatewayClient:
|
||||
`CallDeadlineExceeded`,但到期**不等于未产出、未计费**——在途请求可能已发出、
|
||||
已被上游计费,且清理仍在 `finally` 里完成,故返回时刻 = 期限 + 清理耗时。
|
||||
"""
|
||||
if response_mode not in ("text", "openai"):
|
||||
raise ValueError("response_mode 必须为 text 或 openai")
|
||||
if response_mode == "openai":
|
||||
if structured is not None:
|
||||
raise ValueError("openai 模式不能与 structured 并用")
|
||||
if (
|
||||
overlay is not None
|
||||
and "n" in overlay
|
||||
and (type(overlay["n"]) is not int or overlay["n"] != 1)
|
||||
):
|
||||
raise ValueError("openai 模式仅支持 n=1")
|
||||
try:
|
||||
json.dumps(messages, allow_nan=False)
|
||||
except (ValueError, TypeError) as exc:
|
||||
raise ValueError("messages 必须可 JSON 序列化") from exc
|
||||
messages = copy.deepcopy(messages)
|
||||
overlay = copy.deepcopy(dict(overlay)) if overlay is not None else None
|
||||
if structured is not None and not self._structured_available:
|
||||
raise ImportError(
|
||||
"结构化输出未启用: 安装 pip install 'polygateway[structured]' 后重新装配"
|
||||
@@ -429,6 +454,7 @@ class GatewayClient:
|
||||
context = _CallContext(now=self._now)
|
||||
request = ChatRequest(
|
||||
messages=messages,
|
||||
response_mode=response_mode,
|
||||
session_id=session_id,
|
||||
parent_call_id=parent_call_id,
|
||||
cache_salt=cache_salt,
|
||||
|
||||
@@ -37,7 +37,7 @@ SOURCE_REASONS = frozenset(
|
||||
class PolyGatewayError(Exception):
|
||||
"""库内一切领域错误的基类,携带来源上下文便于遥测与日志定位。
|
||||
|
||||
`body_text` 是**非 2xx 响应体的摘要**——网关拒绝这次调用时说的话(issue #10)。
|
||||
`body_text` 是**HTTP 失败或 200 chat error 帧的摘要**——网关拒绝这次调用时说的话(issue #10)。
|
||||
它与 `ResultInvalidError.raw_text` 是两回事,严禁混用:
|
||||
|
||||
============== ==================================================
|
||||
|
||||
@@ -17,7 +17,15 @@ from typing import TYPE_CHECKING, Any
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from polygateway.types import ChatRequest, Effort, LLMResponse, ThinkingObservation
|
||||
from polygateway.types import (
|
||||
ChatRequest,
|
||||
Effort,
|
||||
LLMResponse,
|
||||
OpenAIChatCompletion,
|
||||
ResponseMode,
|
||||
ThinkingObservation,
|
||||
_has_completion_output,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Mapping
|
||||
@@ -107,6 +115,7 @@ def build_cache_key(
|
||||
*,
|
||||
sampling: Mapping[str, Any] | None = None,
|
||||
reasoning_effort: Effort | None = None,
|
||||
response_mode: ResponseMode = "text",
|
||||
) -> str:
|
||||
"""缓存 key 公式;salt 仅非 None 时参与(VT 旧键语义: 不传 salt 键形不变)。
|
||||
|
||||
@@ -127,6 +136,8 @@ def build_cache_key(
|
||||
"messages": digest_messages(messages),
|
||||
"namespace": namespace,
|
||||
}
|
||||
if response_mode != "text":
|
||||
key_obj["response_mode"] = response_mode
|
||||
if salt is not None:
|
||||
key_obj["salt"] = salt
|
||||
if sampling:
|
||||
@@ -176,6 +187,7 @@ class CacheMW:
|
||||
# 请求级档位必须逐次进 key: `self._fingerprint` 是装配期的集合级指纹,
|
||||
# 同一个 client 上 low 与 max 两次调用在它眼里毫无分别(issue #20)
|
||||
reasoning_effort=request.reasoning_effort,
|
||||
response_mode=request.response_mode,
|
||||
)
|
||||
cached = await self._safe_get(key)
|
||||
if cached is not None:
|
||||
@@ -183,7 +195,12 @@ class CacheMW:
|
||||
if hit is not None:
|
||||
return hit
|
||||
response = await call_next(request)
|
||||
await self._safe_set(key, self._serialize(response))
|
||||
try:
|
||||
serialized = self._serialize(response)
|
||||
except Exception as exc:
|
||||
logger.warning("缓存序列化失败，跳过缓存: {}", type(exc).__name__)
|
||||
else:
|
||||
await self._safe_set(key, serialized)
|
||||
return response
|
||||
|
||||
# —— 命中路径 ——
|
||||
@@ -193,6 +210,16 @@ class CacheMW:
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
fields = {k: v for k, v in data.items() if k in _RESPONSE_FIELDS}
|
||||
completion = fields.get("completion")
|
||||
if completion is not None:
|
||||
if not isinstance(completion, dict):
|
||||
raise ValueError("缓存 completion 必须为对象")
|
||||
fields["completion"] = OpenAIChatCompletion(**completion)
|
||||
if request.response_mode == "openai":
|
||||
if fields.get("completion") is None:
|
||||
raise ValueError("保真缓存缺 completion")
|
||||
if not _has_completion_output(fields.get("content", ""), fields["completion"]):
|
||||
raise ValueError("保真缓存无有效正文或工具产物")
|
||||
structured_data = self._rebuild_structured(fields.get("content", ""), request)
|
||||
# JSON 里存的是 StrEnum 的字符串值,不转就复活成裸 str,与字段注解分叉
|
||||
# (下游 `is ThinkingObservation.OBSERVED` 会在命中路径上静默为 False);
|
||||
|
||||
@@ -357,6 +357,7 @@ class RetryMW:
|
||||
gen_started = self._now()
|
||||
result = await self._transport.complete(
|
||||
messages=request.messages,
|
||||
response_mode=request.response_mode,
|
||||
source=source,
|
||||
stream=request.stream,
|
||||
overlay=request.overlay,
|
||||
@@ -739,6 +740,7 @@ class RetryMW:
|
||||
) -> LLMResponse:
|
||||
return LLMResponse(
|
||||
content=result.content,
|
||||
completion=result.completion,
|
||||
thinking=result.thinking,
|
||||
model=source.model,
|
||||
provider=source.provider,
|
||||
|
||||
@@ -20,6 +20,7 @@ from .types import (
|
||||
OcrLayoutTransportResult,
|
||||
OcrTextResult,
|
||||
OcrTextTransportResult,
|
||||
ResponseMode,
|
||||
SourceConfig,
|
||||
SourceStats,
|
||||
TelemetryStatus,
|
||||
@@ -40,6 +41,10 @@ class Middleware(Protocol):
|
||||
class Transport(Protocol):
|
||||
"""一次原始调用的协议细节(请求组装/流式解析/错误翻译);不含任何治理。
|
||||
|
||||
`response_mode` 为必填返回契约：openai 成功必须提供 completion，
|
||||
不支持者应 RequestRejected，不能静默退回文本。工具的实际增量与首次身份
|
||||
同正文/推理共用活性事件；同帧只置位一次，重复身份不算进展。
|
||||
|
||||
`reasoning_effort` 是本次调用要求的推理档位(`None` = 不表态,随源级配置)。
|
||||
它必须走**协议参数**而不能让 transport 自己去读 `ChatRequest`: 端口只收拆开的
|
||||
请求要素,是为了让 transport 不依赖洋葱内部的请求类型(P7 端口最内层)。
|
||||
@@ -70,6 +75,7 @@ class Transport(Protocol):
|
||||
reasoning_effort: Effort | None,
|
||||
first_token_event: asyncio.Event | None,
|
||||
content_event: asyncio.Event | None,
|
||||
response_mode: ResponseMode,
|
||||
) -> TransportResult: ...
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
"""OpenAI 单 choice 消息聚合：仅协议值处理，不做 I/O 或治理。"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from polygateway.errors import TransientError
|
||||
from polygateway.types import OpenAIChatCompletion, ResponseMode
|
||||
|
||||
|
||||
def _invalid(reason: str) -> TransientError:
|
||||
"""协议形态失败，不回显可能敏感的工具参数。"""
|
||||
return TransientError(f"OpenAI 协议失败: {reason}", operation="chat")
|
||||
|
||||
|
||||
def _object(value: Any, name: str) -> dict[str, Any]:
|
||||
"""外部对象先校验再读取。"""
|
||||
if not isinstance(value, dict):
|
||||
raise _invalid(f"{name} 必须为对象")
|
||||
return value
|
||||
|
||||
|
||||
def _identity(target: dict[str, Any], key: str, value: Any) -> bool:
|
||||
"""身份字段只接受同值重复，不把重复完整 ID 当分片拼接。"""
|
||||
if value is None:
|
||||
return False
|
||||
if not isinstance(value, str):
|
||||
raise _invalid(f"{key} 必须为字符串")
|
||||
if key in target and target[key] != value:
|
||||
raise _invalid(f"{key} 冲突")
|
||||
changed = key not in target and bool(value)
|
||||
target[key] = value
|
||||
return changed
|
||||
|
||||
|
||||
def _append(target: dict[str, Any], key: str, value: Any) -> str:
|
||||
"""字符串片段逐字拼接，保留缺省/null/空串区别。"""
|
||||
if value is None:
|
||||
target.setdefault(key, None)
|
||||
return ""
|
||||
if not isinstance(value, str):
|
||||
raise _invalid(f"{key} 必须为字符串或 null")
|
||||
target[key] = (target.get(key) or "") + value
|
||||
return value
|
||||
|
||||
|
||||
def _completion(message: dict[str, Any], metadata: dict[str, Any]) -> OpenAIChatCompletion:
|
||||
"""复用内核最终快照校验并翻译领域错误。"""
|
||||
try:
|
||||
return OpenAIChatCompletion(
|
||||
message, metadata.get("finish_reason"), metadata.get("id"), metadata.get("usage")
|
||||
)
|
||||
except (ValueError, TypeError) as exc:
|
||||
raise _invalid(str(exc)) from exc
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ChatDelta:
|
||||
"""一帧提取结果；progress 表示真实生成进展。"""
|
||||
|
||||
content: str
|
||||
reasoning_content: str
|
||||
progress: bool
|
||||
|
||||
|
||||
class ChatCompletionAccumulator:
|
||||
"""每次 transport 尝试独享的聚合器，文本模式不严格校验工具产物。"""
|
||||
|
||||
def __init__(self, *, response_mode: ResponseMode) -> None:
|
||||
"""初始化尝试私有状态。"""
|
||||
self.mode = response_mode
|
||||
self.message: dict[str, Any] = {}
|
||||
self.metadata: dict[str, Any] = {}
|
||||
self.tools: dict[int, dict[str, Any]] = {}
|
||||
|
||||
def add_chunk(self, chunk: dict[str, Any]) -> ChatDelta:
|
||||
"""聚合一个已解码帧，一帧至多一个活性事件。"""
|
||||
chunk = _object(chunk, "chunk")
|
||||
strict = self.mode == "openai"
|
||||
if strict:
|
||||
_identity(self.metadata, "id", chunk.get("id"))
|
||||
if chunk.get("usage") is not None:
|
||||
_object(chunk["usage"], "usage")
|
||||
self.metadata["usage"] = chunk["usage"]
|
||||
choices = chunk.get("choices", [])
|
||||
if not isinstance(choices, list):
|
||||
raise _invalid("choices 必须为数组")
|
||||
if not choices:
|
||||
return ChatDelta("", "", False)
|
||||
if strict and len(choices) != 1:
|
||||
raise _invalid("仅支持单 choice")
|
||||
choice = _object(choices[0], "choice")
|
||||
if strict:
|
||||
if type(choice.get("index")) is not int or choice["index"] != 0:
|
||||
raise _invalid("choice index 必须为 0")
|
||||
_identity(self.metadata, "finish_reason", choice.get("finish_reason"))
|
||||
delta = _object(choice.get("delta", {}), "delta")
|
||||
if strict and delta.keys() - {"role", "content", "reasoning_content", "tool_calls"}:
|
||||
raise _invalid("未知 delta 字段")
|
||||
if "role" in delta and strict:
|
||||
if delta["role"] != "assistant":
|
||||
raise _invalid("role 必须为 assistant")
|
||||
self.message["role"] = delta["role"]
|
||||
content = _append(self.message, "content", delta["content"]) if "content" in delta else ""
|
||||
reasoning = (
|
||||
_append(self.message, "reasoning_content", delta["reasoning_content"])
|
||||
if "reasoning_content" in delta
|
||||
else ""
|
||||
)
|
||||
progress = bool(content or reasoning)
|
||||
if "tool_calls" in delta:
|
||||
tools = delta["tool_calls"]
|
||||
if not isinstance(tools, list):
|
||||
if strict:
|
||||
raise _invalid("tool_calls 必须为数组")
|
||||
return ChatDelta(content, reasoning, progress)
|
||||
if strict:
|
||||
self.message.setdefault("tool_calls", [])
|
||||
for raw in tools:
|
||||
if not strict:
|
||||
# 文本只消费可识别的工具进展，不承诺工具完整性。
|
||||
if isinstance(raw, dict) and type(raw.get("index")) is int:
|
||||
index = raw["index"]
|
||||
tool = self.tools.setdefault(index, {})
|
||||
for key in ("id", "type"):
|
||||
if isinstance(raw.get(key), str) and raw[key] and key not in tool:
|
||||
tool[key] = raw[key]
|
||||
progress = True
|
||||
fn = raw.get("function")
|
||||
if isinstance(fn, dict):
|
||||
progress |= any(
|
||||
isinstance(fn.get(k), str) and bool(fn[k])
|
||||
for k in ("name", "arguments")
|
||||
)
|
||||
continue
|
||||
raw = _object(raw, "tool")
|
||||
index = raw.get("index")
|
||||
if type(index) is not int or index < 0:
|
||||
raise _invalid("tool index 必须为非负整数")
|
||||
if raw.keys() - {"index", "id", "type", "function"}:
|
||||
raise _invalid("未知工具增量字段")
|
||||
tool = self.tools.setdefault(index, {})
|
||||
for key in ("id", "type"):
|
||||
progress |= _identity(tool, key, raw.get(key))
|
||||
if "function" in raw:
|
||||
fn = _object(raw["function"], "function")
|
||||
if fn.keys() - {"name", "arguments"}:
|
||||
raise _invalid("未知 function 增量字段")
|
||||
target = tool.setdefault("function", {})
|
||||
for key in ("name", "arguments"):
|
||||
if key in fn:
|
||||
if not isinstance(fn[key], str):
|
||||
raise _invalid(f"function.{key} 必须为字符串")
|
||||
progress |= bool(_append(target, key, fn[key]))
|
||||
return ChatDelta(content, reasoning, progress)
|
||||
|
||||
def finish(self) -> OpenAIChatCompletion:
|
||||
"""线路完整性由调用者保证；此处只校验最终消息。"""
|
||||
message = dict(self.message)
|
||||
if self.tools:
|
||||
message["tool_calls"] = [self.tools[index] for index in sorted(self.tools)]
|
||||
return _completion(message, self.metadata)
|
||||
|
||||
|
||||
def parse_chat_completion(body: dict[str, Any]) -> OpenAIChatCompletion:
|
||||
"""非流式完整消息保留扩展字段，拒绝静默截取多 choice。"""
|
||||
body = _object(body, "body")
|
||||
choices = body.get("choices")
|
||||
if not isinstance(choices, list) or len(choices) != 1:
|
||||
raise _invalid("仅支持单 choice")
|
||||
choice = _object(choices[0], "choice")
|
||||
if "index" in choice and (type(choice["index"]) is not int or choice["index"] != 0):
|
||||
raise _invalid("choice index 必须为 0")
|
||||
metadata = {**body, "finish_reason": choice.get("finish_reason")}
|
||||
return _completion(_object(choice.get("message"), "message"), metadata)
|
||||
@@ -38,11 +38,19 @@ from polygateway.thinking import (
|
||||
validate_thinking_raw,
|
||||
)
|
||||
from polygateway.transports._http_errors import compose_message, summarize_body
|
||||
from polygateway.transports._openai_completion import (
|
||||
ChatCompletionAccumulator,
|
||||
ChatDelta,
|
||||
parse_chat_completion,
|
||||
)
|
||||
from polygateway.types import (
|
||||
Effort,
|
||||
EmbeddingTransportResult,
|
||||
OpenAIChatCompletion,
|
||||
ResponseMode,
|
||||
SourceConfig,
|
||||
TransportResult,
|
||||
_has_completion_output,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -62,29 +70,6 @@ def _sse_data_payload(raw: str) -> str | None:
|
||||
return line[len("data:") :].strip()
|
||||
|
||||
|
||||
def _sse_delta(chunk: dict[str, Any], usage_sink: dict[str, Any]) -> tuple[bool, str] | None:
|
||||
"""从 chunk 提取增量: (True, content) 或 (False, reasoning);usage 帧旁路进 sink。"""
|
||||
if chunk.get("usage"):
|
||||
usage_sink["usage"] = chunk["usage"]
|
||||
if "model" not in usage_sink:
|
||||
# 首个**有效**值即固定: 末帧的异常值不得覆盖它;但首帧报空串也不能锁死
|
||||
# sink——否则后续真实版本会丢(issue #3)
|
||||
reported = _coerce_model_reported(chunk.get("model"))
|
||||
if reported is not None:
|
||||
usage_sink["model"] = reported
|
||||
choices = chunk.get("choices") or []
|
||||
if not choices:
|
||||
return None
|
||||
delta = choices[0].get("delta") or {}
|
||||
content = delta.get("content")
|
||||
if content:
|
||||
return (True, content)
|
||||
reasoning = delta.get("reasoning_content")
|
||||
if reasoning:
|
||||
return (False, reasoning)
|
||||
return None
|
||||
|
||||
|
||||
def _stall_from_chunk_times(
|
||||
times: list[float], threshold_ms: int
|
||||
) -> tuple[float | None, float | None]:
|
||||
@@ -108,10 +93,25 @@ def _stall_from_chunk_times(
|
||||
return (total, float(count)) if count else (0.0, 0.0)
|
||||
|
||||
|
||||
def _check_chat_error(chunk: Any) -> None:
|
||||
"""所有模式拒绝顶层 error；诊断复用有界摘要。"""
|
||||
if not isinstance(chunk, dict):
|
||||
raise TransientError("chat 响应必须为 JSON 对象", operation="chat")
|
||||
if "error" in chunk:
|
||||
summary = summarize_body(json.dumps({"error": chunk["error"]}, ensure_ascii=False))
|
||||
raise TransientError(
|
||||
compose_message("上游 chat error 帧", summary), operation="chat", body_text=summary
|
||||
)
|
||||
if chunk.get("usage") is not None and not isinstance(chunk["usage"], dict):
|
||||
raise TransientError("chat usage 必须为对象或 null", operation="chat")
|
||||
|
||||
|
||||
async def _iter_sse_deltas(
|
||||
lines: AsyncIterator[str], usage_sink: dict[str, Any]
|
||||
) -> AsyncIterator[tuple[bool, str]]:
|
||||
"""逐行解析 SSE 流;[DONE] 置 sink["done"];畸形 JSON 帧 → 瞬时错误(可重试)。"""
|
||||
lines: AsyncIterator[str],
|
||||
usage_sink: dict[str, Any],
|
||||
accumulator: ChatCompletionAccumulator,
|
||||
) -> AsyncIterator[ChatDelta]:
|
||||
"""解析 SSE，一帧只产生一次进展事件。"""
|
||||
async for raw in lines:
|
||||
data = _sse_data_payload(raw)
|
||||
if data is None:
|
||||
@@ -122,11 +122,16 @@ async def _iter_sse_deltas(
|
||||
try:
|
||||
chunk = json.loads(data)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise TransientError(
|
||||
f"SSE 帧畸形(malformed_json): {data[:80]!r}", operation="chat"
|
||||
) from exc
|
||||
delta = _sse_delta(chunk, usage_sink)
|
||||
if delta is not None:
|
||||
raise TransientError("SSE 帧畸形(malformed_json)", operation="chat") from exc
|
||||
_check_chat_error(chunk)
|
||||
if chunk.get("usage") is not None:
|
||||
usage_sink["usage"] = chunk["usage"]
|
||||
if "model" not in usage_sink:
|
||||
model = _coerce_model_reported(chunk.get("model"))
|
||||
if model is not None:
|
||||
usage_sink["model"] = model
|
||||
delta = accumulator.add_chunk(chunk)
|
||||
if delta.progress:
|
||||
yield delta
|
||||
|
||||
|
||||
@@ -458,6 +463,7 @@ class OpenAICompatTransport:
|
||||
reasoning_effort: Effort | None,
|
||||
first_token_event: asyncio.Event | None,
|
||||
content_event: asyncio.Event | None,
|
||||
response_mode: ResponseMode,
|
||||
) -> TransportResult:
|
||||
"""一次原始调用;HTTP/线路/流式异常按 ARCH §6.2 翻译为领域错误。
|
||||
|
||||
@@ -468,6 +474,16 @@ class OpenAICompatTransport:
|
||||
`content_event` 为中段沉默对冲处置位(1.3.8): 同款约定,流式路径**每个**
|
||||
增量 chunk 置位一次,非流式路径收它但永不置位。
|
||||
"""
|
||||
if response_mode not in ("text", "openai"):
|
||||
raise RequestRejectedError(
|
||||
"非法 response_mode", source_name=source.name, operation="chat"
|
||||
)
|
||||
if response_mode == "openai":
|
||||
for layer in (source.extra_body, overlay):
|
||||
if "n" in layer and (type(layer["n"]) is not int or layer["n"] != 1):
|
||||
raise RequestRejectedError(
|
||||
"openai 模式仅支持 n=1", source_name=source.name, operation="chat"
|
||||
)
|
||||
profile = get_provider(source.provider, registry=self._registry)
|
||||
try:
|
||||
payload, applied_effort = self._build_payload(
|
||||
@@ -492,12 +508,30 @@ class OpenAICompatTransport:
|
||||
try:
|
||||
if stream:
|
||||
result = await self._complete_stream(
|
||||
client, url, payload, source, profile, first_token_event, content_event
|
||||
client,
|
||||
url,
|
||||
payload,
|
||||
source,
|
||||
profile,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
)
|
||||
else:
|
||||
result = await self._complete_once(
|
||||
client, url, payload, source, profile, first_token_event, content_event
|
||||
client,
|
||||
url,
|
||||
payload,
|
||||
source,
|
||||
profile,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
)
|
||||
except TransientError as exc:
|
||||
if exc.source_name is None:
|
||||
exc.source_name = source.name
|
||||
raise
|
||||
except StreamLivenessTimeout as exc:
|
||||
raise TransientError(f"{source.name} 流活性超时({exc.kind})", **ctx) from exc
|
||||
except httpx.TimeoutException as exc:
|
||||
@@ -585,6 +619,7 @@ class OpenAICompatTransport:
|
||||
profile: ProviderProfile,
|
||||
first_token_event: asyncio.Event | None,
|
||||
content_event: asyncio.Event | None,
|
||||
response_mode: ResponseMode,
|
||||
) -> TransportResult:
|
||||
started = time.monotonic()
|
||||
async with client.stream("POST", url, json=payload) as resp:
|
||||
@@ -594,8 +629,9 @@ class OpenAICompatTransport:
|
||||
source, resp.status_code, body, resp.headers, operation="chat"
|
||||
)
|
||||
sink: dict[str, Any] = {}
|
||||
accumulator = ChatCompletionAccumulator(response_mode=response_mode)
|
||||
guarded = stream_with_liveness_timeouts(
|
||||
_iter_sse_deltas(resp.aiter_lines(), sink),
|
||||
_iter_sse_deltas(resp.aiter_lines(), sink, accumulator),
|
||||
ttft_s=source.ttft_timeout_s or source.timeout_s,
|
||||
inter_token_s=source.inter_token_timeout_s or source.timeout_s,
|
||||
total_s=source.timeout_s,
|
||||
@@ -606,7 +642,7 @@ class OpenAICompatTransport:
|
||||
last = started
|
||||
max_gap = 0.0
|
||||
chunk_times: list[float] = []
|
||||
async for is_content, text in guarded:
|
||||
async for delta in guarded:
|
||||
now = time.monotonic()
|
||||
# 增量 chunk 时间戳(1.3.8 沉默计量): 循环体只会被真实增量触发——
|
||||
# 空 chunk/usage 帧/SSE keepalive 已在 _sse_data_payload/_sse_delta
|
||||
@@ -625,10 +661,18 @@ class OpenAICompatTransport:
|
||||
else:
|
||||
max_gap = max(max_gap, (now - last) * 1000)
|
||||
last = now
|
||||
(content_parts if is_content else thinking_parts).append(text)
|
||||
if delta.content:
|
||||
content_parts.append(delta.content)
|
||||
if delta.reasoning_content:
|
||||
thinking_parts.append(delta.reasoning_content)
|
||||
if response_mode == "openai" and not sink.get("done"):
|
||||
raise TransientError(
|
||||
"保真响应缺 [DONE]，禁止打捞", source_name=source.name, operation="chat"
|
||||
)
|
||||
completion_result = accumulator.finish() if response_mode == "openai" else None
|
||||
salvaged = self._check_done(sink, content_parts, thinking_parts, source)
|
||||
content, thinking = self._finalize_text(content_parts, thinking_parts, profile)
|
||||
self._reject_empty_completion(content, source)
|
||||
self._reject_empty_completion(content, completion_result, source)
|
||||
prompt, completion, usage_source = _resolve_stream_usage(sink, salvaged)
|
||||
reasoning_tokens = _coerce_reasoning_tokens(sink.get("usage"))
|
||||
stall_ms, stall_count = _stall_from_chunk_times(chunk_times, source.stall_threshold_ms)
|
||||
@@ -643,6 +687,7 @@ class OpenAICompatTransport:
|
||||
stall_ms=stall_ms,
|
||||
stall_count=stall_count,
|
||||
raw={"usage": sink.get("usage")},
|
||||
completion=completion_result,
|
||||
cached_prompt_tokens=_coerce_cached_tokens(sink.get("usage")),
|
||||
model_reported=_coerce_model_reported(sink.get("model")),
|
||||
reasoning_tokens=reasoning_tokens,
|
||||
@@ -670,14 +715,16 @@ class OpenAICompatTransport:
|
||||
raise TransientError(f"{source.name} SSE missing_done: 截断且无 [DONE]", **ctx)
|
||||
return True
|
||||
|
||||
def _reject_empty_completion(self, content: str, source: SourceConfig) -> None:
|
||||
def _reject_empty_completion(
|
||||
self, content: str, completion: OpenAIChatCompletion | None, source: SourceConfig
|
||||
) -> None:
|
||||
"""空补全 → 瞬时错误(2026-07-20 人类裁决,M1 验证发现)。
|
||||
|
||||
服务 200 且流程完整([DONE]/usage 正常)但 content 为空——MiniMax 等
|
||||
网关的间歇异常形态。视为服务抖动: 退避重试/换源,**绝不缓存空响应**;
|
||||
承 CHS "VLM 零 content"归瞬时的先例(invokers.py:309)。
|
||||
"""
|
||||
if not content.strip():
|
||||
if not _has_completion_output(content, completion):
|
||||
raise TransientError(
|
||||
f"{source.name} 空补全(empty_completion): 流程完整但零内容",
|
||||
source_name=source.name,
|
||||
@@ -704,6 +751,7 @@ class OpenAICompatTransport:
|
||||
profile: ProviderProfile,
|
||||
first_token_event: asyncio.Event | None,
|
||||
content_event: asyncio.Event | None,
|
||||
response_mode: ResponseMode,
|
||||
) -> TransportResult:
|
||||
"""非流式快路径(三项目均无,库新增): 单 JSON 响应,仅 total 超时。
|
||||
|
||||
@@ -723,16 +771,25 @@ class OpenAICompatTransport:
|
||||
raise TransientError(
|
||||
f"{source.name} 非流式响应非法 JSON", source_name=source.name, operation="chat"
|
||||
) from exc
|
||||
_check_chat_error(body)
|
||||
completion_result = parse_chat_completion(body) if response_mode == "openai" else None
|
||||
choices = body.get("choices") or []
|
||||
if not choices:
|
||||
raise TransientError(
|
||||
f"{source.name} 响应缺 choices", source_name=source.name, operation="chat"
|
||||
)
|
||||
message = choices[0].get("message") or {}
|
||||
if not isinstance(choices, list) or not isinstance(choices[0], dict):
|
||||
raise TransientError("chat choices 形态非法", operation="chat")
|
||||
message = choices[0].get("message", {})
|
||||
if not isinstance(message, dict) or any(
|
||||
message.get(key) is not None and not isinstance(message[key], str)
|
||||
for key in ("content", "reasoning_content")
|
||||
):
|
||||
raise TransientError("chat message 形态非法", operation="chat")
|
||||
content, thinking = self._finalize_text(
|
||||
[message.get("content") or ""], [message.get("reasoning_content") or ""], profile
|
||||
)
|
||||
self._reject_empty_completion(content, source)
|
||||
self._reject_empty_completion(content, completion_result, source)
|
||||
prompt, completion, usage_source = _resolve_usage(body.get("usage") or {})
|
||||
reasoning_tokens = _coerce_reasoning_tokens(body.get("usage"))
|
||||
return TransportResult(
|
||||
@@ -744,6 +801,7 @@ class OpenAICompatTransport:
|
||||
ttft_ms=None,
|
||||
max_inter_token_ms=None,
|
||||
raw={"usage": body.get("usage")},
|
||||
completion=completion_result,
|
||||
cached_prompt_tokens=_coerce_cached_tokens(body.get("usage")),
|
||||
model_reported=_coerce_model_reported(body.get("model")),
|
||||
reasoning_tokens=reasoning_tokens,
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
fake,字段顺序即公共承诺;新增字段只增不删且必带默认值。
|
||||
"""
|
||||
|
||||
import copy
|
||||
import dataclasses
|
||||
import json
|
||||
import math
|
||||
@@ -422,6 +423,73 @@ class _CallContext:
|
||||
return True
|
||||
|
||||
|
||||
ResponseMode = Literal["text", "openai"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OpenAIChatCompletion:
|
||||
"""单 choice 协议结果，构造时快照隔离；嵌套 JSON 仍可变。"""
|
||||
|
||||
message: dict[str, Any]
|
||||
finish_reason: str | None
|
||||
response_id: str | None
|
||||
usage: dict[str, Any] | None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""校验协议快照；transport 翻译错误，缓存坏条目按 miss。"""
|
||||
message = self.message
|
||||
if not isinstance(message, dict) or message.get("role") != "assistant":
|
||||
raise ValueError("completion message 必须含 assistant role")
|
||||
for key in ("content", "reasoning_content"):
|
||||
if message.get(key) is not None and not isinstance(message[key], str):
|
||||
raise ValueError(f"completion {key} 必须为字符串或 null")
|
||||
for value in (self.finish_reason, self.response_id):
|
||||
if value is not None and not isinstance(value, str):
|
||||
raise ValueError("completion 身份/结束原因必须为字符串或 null")
|
||||
if self.usage is not None and not isinstance(self.usage, dict):
|
||||
raise ValueError("completion usage 必须为对象或 null")
|
||||
if "tool_calls" in message:
|
||||
tools = message["tool_calls"]
|
||||
if not isinstance(tools, list):
|
||||
raise ValueError("completion tool_calls 必须为数组")
|
||||
seen: set[str] = set()
|
||||
for tool in tools:
|
||||
if not isinstance(tool, dict):
|
||||
raise ValueError("completion tool 必须为对象")
|
||||
identity = tool.get("id")
|
||||
fn = tool.get("function")
|
||||
if (
|
||||
not isinstance(identity, str)
|
||||
or not identity
|
||||
or identity in seen
|
||||
or tool.get("type") != "function"
|
||||
or not isinstance(fn, dict)
|
||||
or not isinstance(fn.get("name"), str)
|
||||
or not fn["name"]
|
||||
or not isinstance(fn.get("arguments"), str)
|
||||
):
|
||||
raise ValueError("completion tool 身份/函数字段缺失、冲突或非法")
|
||||
seen.add(identity)
|
||||
json.dumps({"message": message, "usage": self.usage}, allow_nan=False)
|
||||
object.__setattr__(self, "message", copy.deepcopy(message))
|
||||
object.__setattr__(self, "usage", copy.deepcopy(self.usage))
|
||||
|
||||
|
||||
def _has_completion_output(content: str, completion: OpenAIChatCompletion | None) -> bool:
|
||||
"""成功须有投影正文或完整工具；协议原文为空时不信任缓存旧投影。
|
||||
|
||||
只判定产物存在性，不替代协议结构校验，也不收紧快照类型的构造用途。
|
||||
transport 与缓存共用此规则，避免空响应从缓存旁路成功。
|
||||
"""
|
||||
if completion is not None:
|
||||
if completion.message.get("tool_calls"):
|
||||
return True
|
||||
raw_content = completion.message.get("content")
|
||||
if not isinstance(raw_content, str) or not raw_content.strip():
|
||||
return False
|
||||
return isinstance(content, str) and bool(content.strip())
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LLMResponse:
|
||||
"""一次治理调用的统一响应(与三项目超集兼容,ARCH §5.1)。"""
|
||||
@@ -487,6 +555,8 @@ class LLMResponse:
|
||||
call_stats: CallStats | None = None
|
||||
"""本次**逻辑调用**的统计快照(1.3.5);`None` = 未知,不得读成 0。"""
|
||||
|
||||
completion: OpenAIChatCompletion | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ChatRequest:
|
||||
@@ -543,6 +613,8 @@ class ChatRequest:
|
||||
洋葱各层经 `dataclasses.replace` 派生请求时保留**同一引用**(不是拷贝),
|
||||
重试/重问/分批才能共享同一个逻辑 ID 与计数。"""
|
||||
|
||||
response_mode: ResponseMode = "text"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Usage:
|
||||
@@ -627,6 +699,8 @@ class TransportResult:
|
||||
"""超阈值相邻间隔个数(归因成对报告,Gregg 原则);值域与 `None` 语义同
|
||||
`stall_ms`。"""
|
||||
|
||||
completion: OpenAIChatCompletion | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceConfig:
|
||||
|
||||
@@ -22,7 +22,13 @@ from polygateway.client import (
|
||||
)
|
||||
from polygateway.providers import get_provider
|
||||
from polygateway.transports.openai_compat import OpenAICompatTransport
|
||||
from polygateway.types import Effort, EmbeddingTransportResult, SourceConfig, TransportResult
|
||||
from polygateway.types import (
|
||||
Effort,
|
||||
EmbeddingTransportResult,
|
||||
ResponseMode,
|
||||
SourceConfig,
|
||||
TransportResult,
|
||||
)
|
||||
from tests.live_evidence import (
|
||||
AttemptEvidence,
|
||||
HttpEvidence,
|
||||
@@ -293,6 +299,7 @@ class ObservedTransport:
|
||||
reasoning_effort: Effort | None,
|
||||
first_token_event: asyncio.Event | None,
|
||||
content_event: asyncio.Event | None,
|
||||
response_mode: ResponseMode,
|
||||
) -> TransportResult:
|
||||
"""与生产端口逐参数同签名。"""
|
||||
with self._capture.attempt_context(call_id):
|
||||
@@ -306,6 +313,7 @@ class ObservedTransport:
|
||||
reasoning_effort=reasoning_effort,
|
||||
first_token_event=first_token_event,
|
||||
content_event=content_event,
|
||||
response_mode=response_mode,
|
||||
)
|
||||
|
||||
async def embed(
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
"""真实网关工具双轮冒烟；复用安全 Markdown 报告，不运行工具副作用。"""
|
||||
|
||||
import dataclasses
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from dotenv import dotenv_values
|
||||
|
||||
from polygateway import GatewayClient, GatewaySettings
|
||||
from tests.live_evidence import write_live_round
|
||||
|
||||
_ENV = {k: v for k, v in {**dotenv_values(".env"), **os.environ}.items() if v is not None}
|
||||
pytestmark = [pytest.mark.slow, pytest.mark.requires_llm]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stream", [True, False])
|
||||
async def test_tools_two_rounds(stream):
|
||||
"""已配置源中选首源，只改变测试隔离与 raw-only 意图，不收紧生产超时。"""
|
||||
if not any(k.startswith("LLM__") and k.endswith("__API_KEY") for k in _ENV):
|
||||
pytest.skip("未配置真实源，未覆盖")
|
||||
settings = GatewaySettings.from_env("LLM", env=_ENV)
|
||||
source = dataclasses.replace(settings.sources[0], enable_thinking=None, reasoning_effort=None)
|
||||
settings = dataclasses.replace(
|
||||
settings,
|
||||
sources=(source,),
|
||||
cache_backend="none",
|
||||
cache_namespace=None,
|
||||
cache_ttl_s=None,
|
||||
telemetry_backend="none",
|
||||
telemetry_auto_migrate=False,
|
||||
limiter_backend="memory",
|
||||
breaker_backend="memory",
|
||||
hedge_after_s=None,
|
||||
hedge_stall_after_s=None,
|
||||
)
|
||||
tools = [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "echo_value",
|
||||
"description": "Return the supplied value",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {"value": {"type": "string"}},
|
||||
"required": ["value"],
|
||||
},
|
||||
},
|
||||
}
|
||||
]
|
||||
messages = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Call echo_value with value hello. After receiving the tool result, reply with hello.",
|
||||
}
|
||||
]
|
||||
status, error_type, counts = "FAIL", None, {}
|
||||
try:
|
||||
async with GatewayClient.from_settings(settings) as client:
|
||||
first = await client.chat(
|
||||
messages,
|
||||
stream=stream,
|
||||
response_mode="openai",
|
||||
overlay={
|
||||
"tools": tools,
|
||||
"tool_choice": {"type": "function", "function": {"name": "echo_value"}},
|
||||
},
|
||||
)
|
||||
completion = first.completion
|
||||
assert completion is not None and completion.message["role"] == "assistant"
|
||||
assert completion.response_id and completion.finish_reason == "tool_calls"
|
||||
calls = completion.message["tool_calls"]
|
||||
counts["tools"] = len(calls)
|
||||
assert calls
|
||||
messages.append(completion.message)
|
||||
for call in calls:
|
||||
assert call["function"]["name"] == "echo_value"
|
||||
arguments = json.loads(call["function"]["arguments"])
|
||||
assert arguments["value"] == "hello"
|
||||
messages.append(
|
||||
{"role": "tool", "tool_call_id": call["id"], "content": arguments["value"]}
|
||||
)
|
||||
second = await client.chat(
|
||||
messages,
|
||||
stream=stream,
|
||||
response_mode="openai",
|
||||
overlay={"tools": tools, "tool_choice": "none"},
|
||||
)
|
||||
assert (
|
||||
second.completion is not None and second.completion.message["role"] == "assistant"
|
||||
)
|
||||
assert "hello" in (second.completion.message.get("content") or "").lower()
|
||||
counts["attempts"] = first.call_stats.attempts + second.call_stats.attempts
|
||||
status = "PASS"
|
||||
except Exception as exc:
|
||||
error_type = type(exc).__name__
|
||||
raise
|
||||
finally:
|
||||
write_live_round(
|
||||
Path("tests/outputs/issue29/live"),
|
||||
run_id=uuid4().hex,
|
||||
matrix_id="tools",
|
||||
round_index=1,
|
||||
safe_fields={
|
||||
"status": status,
|
||||
"stream": stream,
|
||||
"error_type": error_type,
|
||||
"counts": counts,
|
||||
"planned_rounds": 2,
|
||||
"completed_rounds": 2 if status == "PASS" else 0,
|
||||
},
|
||||
)
|
||||
@@ -172,3 +172,22 @@ class TestClientWithRealRedis:
|
||||
second = await client.chat(_MSGS)
|
||||
assert first.cache_hit is False and second.cache_hit is True
|
||||
assert second.content == "hello"
|
||||
|
||||
|
||||
async def test_real_completion_roundtrip(redis_cache, namespace):
|
||||
"""真实 Redis 保存完整嵌套协议对象，命中仍为值类型。"""
|
||||
from dataclasses import replace
|
||||
|
||||
from tests.unit.test_openai_completion import accumulate
|
||||
|
||||
completion = accumulate().finish()
|
||||
mw = CacheMW(backend=redis_cache, model_fingerprint="m", default_namespace=namespace, ttl_s=60)
|
||||
|
||||
async def terminal(_):
|
||||
return replace(_resp(), completion=completion)
|
||||
|
||||
request = ChatRequest(messages=_MSGS, response_mode="openai")
|
||||
first = await mw(request, terminal)
|
||||
second = await mw(request, terminal)
|
||||
assert second.cache_hit and second.completion == first.completion
|
||||
assert second.completion is not first.completion
|
||||
|
||||
@@ -86,6 +86,7 @@ class ScriptedTransport:
|
||||
reasoning_effort,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
):
|
||||
self.calls.append(source.name)
|
||||
if self.hang:
|
||||
|
||||
@@ -221,6 +221,7 @@ class ClockAdvancingTransport:
|
||||
reasoning_effort,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
):
|
||||
self.calls.append((source.name, call_id))
|
||||
advance, action = self.script.pop(0)
|
||||
|
||||
@@ -15,6 +15,7 @@ from polygateway.types import (
|
||||
ChatRequest,
|
||||
Effort,
|
||||
LLMResponse,
|
||||
OpenAIChatCompletion,
|
||||
SourceConfig,
|
||||
ThinkingObservation,
|
||||
)
|
||||
@@ -891,3 +892,113 @@ class TestCallStatsNotPoisoned:
|
||||
without = ChatRequest(messages=_MSGS)
|
||||
assert with_ctx.cache_namespace == without.cache_namespace
|
||||
assert digest_messages(with_ctx.messages) == digest_messages(without.messages)
|
||||
|
||||
|
||||
class TestOpenAICompletionCache:
|
||||
"""协议缓存不得混模式或把字典冒充公共值对象。"""
|
||||
|
||||
def test_mode_isolation_preserves_text_key(self):
|
||||
assert build_cache_key("m", _MSGS, "proj", None) == build_cache_key(
|
||||
"m", _MSGS, "proj", None, response_mode="text"
|
||||
)
|
||||
assert build_cache_key("m", _MSGS, "proj", None) != build_cache_key(
|
||||
"m", _MSGS, "proj", None, response_mode="openai"
|
||||
)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"completion",
|
||||
[
|
||||
None,
|
||||
{},
|
||||
{
|
||||
"message": {"content": "ok"},
|
||||
"finish_reason": "stop",
|
||||
"response_id": None,
|
||||
"usage": None,
|
||||
},
|
||||
],
|
||||
)
|
||||
async def test_missing_or_bad_completion_misses(self, completion):
|
||||
backend = InMemoryCache()
|
||||
mw = CacheMW(backend=backend, model_fingerprint="m", default_namespace="proj", ttl_s=60)
|
||||
raw = json.loads(mw._serialize(_resp()))
|
||||
raw["completion"] = completion
|
||||
assert (
|
||||
mw._rehydrate(json.dumps(raw), ChatRequest(messages=_MSGS, response_mode="openai"))
|
||||
is None
|
||||
)
|
||||
|
||||
async def test_serialization_failure_does_not_fail_call(self):
|
||||
mw = CacheMW(
|
||||
backend=InMemoryCache(), model_fingerprint="m", default_namespace="proj", ttl_s=60
|
||||
)
|
||||
response = _resp(cost=object())
|
||||
|
||||
async def terminal(_):
|
||||
return response
|
||||
|
||||
assert await mw(ChatRequest(messages=_MSGS), terminal) is response
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"message",
|
||||
[
|
||||
{"role": "assistant"},
|
||||
{"role": "assistant", "content": None, "tool_calls": []},
|
||||
{"role": "assistant", "content": "", "tool_calls": []},
|
||||
{"role": "assistant", "content": " \n", "tool_calls": []},
|
||||
{"role": "assistant", "reasoning_content": "reason only"},
|
||||
],
|
||||
)
|
||||
@pytest.mark.parametrize("cached_content", ["", "stale projection"])
|
||||
async def test_openai_cache_without_output_misses_and_refills(message, cached_content):
|
||||
"""空协议产物不能靠旧投影正文伪装成功；miss 后回源并覆写。"""
|
||||
backend = InMemoryCache()
|
||||
mw = _mw(backend)
|
||||
request = ChatRequest(messages=_MSGS, response_mode="openai")
|
||||
key = build_cache_key("m", _MSGS, "proj", None, response_mode="openai")
|
||||
bad = _resp(
|
||||
content=cached_content,
|
||||
completion=OpenAIChatCompletion(message, "stop", "bad", None),
|
||||
)
|
||||
await backend.set(key, mw._serialize(bad), ttl_s=60)
|
||||
fresh = _resp(
|
||||
completion=OpenAIChatCompletion(
|
||||
{"role": "assistant", "content": "cached"}, "stop", "fresh", None
|
||||
)
|
||||
)
|
||||
terminal = _Terminal(fresh)
|
||||
result = await mw(request, terminal)
|
||||
assert terminal.calls == 1 and result is fresh and not result.cache_hit
|
||||
hit = await mw(request, terminal)
|
||||
assert terminal.calls == 1 and hit.cache_hit
|
||||
assert hit.completion.response_id == "fresh"
|
||||
|
||||
|
||||
async def test_openai_tool_only_cache_hits_without_origin():
|
||||
"""有效工具本身是产物，空文本投影不妨碍命中。"""
|
||||
backend = InMemoryCache()
|
||||
mw = _mw(backend)
|
||||
completion = OpenAIChatCompletion(
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call-1",
|
||||
"type": "function",
|
||||
"function": {"name": "echo", "arguments": "{}"},
|
||||
}
|
||||
],
|
||||
},
|
||||
"tool_calls",
|
||||
"tool-response",
|
||||
None,
|
||||
)
|
||||
request = ChatRequest(messages=_MSGS, response_mode="openai")
|
||||
key = build_cache_key("m", _MSGS, "proj", None, response_mode="openai")
|
||||
await backend.set(key, mw._serialize(_resp(content="", completion=completion)), ttl_s=60)
|
||||
terminal = _Terminal(_resp())
|
||||
hit = await mw(request, terminal)
|
||||
assert hit.cache_hit and terminal.calls == 0
|
||||
assert hit.completion == completion
|
||||
|
||||
@@ -232,6 +232,7 @@ class _ScriptedGenClockTransport:
|
||||
reasoning_effort,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
):
|
||||
self.calls.append(call_id)
|
||||
result = self._results.pop(0)
|
||||
@@ -1844,6 +1845,7 @@ class _ClockJumpTransport:
|
||||
reasoning_effort,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
):
|
||||
self.calls.append(call_id)
|
||||
self._clock.advance(self._jump)
|
||||
|
||||
@@ -1397,9 +1397,7 @@ class TestStallConfig:
|
||||
def test_stall_after_s_single_source_warns(self):
|
||||
"""单源 scope 设 stall 阈值: 装配期 warning 放行(1.3.7 §5 同款)。"""
|
||||
with _captured_warnings() as warnings:
|
||||
s = GatewaySettings.from_env(
|
||||
"LLM", env=_env(**{"LLM__HEDGE__STALL_AFTER_S": "8"})
|
||||
)
|
||||
s = GatewaySettings.from_env("LLM", env=_env(**{"LLM__HEDGE__STALL_AFTER_S": "8"}))
|
||||
assert s.hedge_stall_after_s == 8.0
|
||||
assert any("单源" in m for m in warnings)
|
||||
|
||||
@@ -1422,9 +1420,7 @@ class TestStallConfig:
|
||||
"EMBED__BATCH_SIZE": "64",
|
||||
}
|
||||
with _captured_warnings() as warnings:
|
||||
EmbeddingSettings.from_env(
|
||||
"EMBED", env={**embed_env, "EMBED__HEDGE__AFTER_S": "8"}
|
||||
)
|
||||
EmbeddingSettings.from_env("EMBED", env={**embed_env, "EMBED__HEDGE__AFTER_S": "8"})
|
||||
assert any("仅对 chat scope 生效" in m for m in warnings)
|
||||
with _captured_warnings() as warnings:
|
||||
EmbeddingSettings.from_env(
|
||||
|
||||
@@ -65,6 +65,7 @@ class HedgeTransport:
|
||||
reasoning_effort,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
):
|
||||
self.calls.append(source.name)
|
||||
self.ft_events.append(first_token_event)
|
||||
|
||||
@@ -279,6 +279,7 @@ async def _complete(observed, *, call_id="a", stream=False):
|
||||
reasoning_effort=None,
|
||||
first_token_event=None,
|
||||
content_event=None,
|
||||
response_mode="text",
|
||||
)
|
||||
|
||||
|
||||
@@ -1270,6 +1271,7 @@ async def test_structured_first_attempt_requires_exact_initial_messages():
|
||||
reasoning_effort=None,
|
||||
first_token_event=None,
|
||||
content_event=None,
|
||||
response_mode="text",
|
||||
)
|
||||
event = capture.attempts(session_id="first", parent_call_id="parent")[0].http[0]
|
||||
assert not request_is_valid(event)
|
||||
|
||||
@@ -90,6 +90,7 @@ async def _complete(
|
||||
reasoning_effort=None,
|
||||
first_token_event=None,
|
||||
content_event=None,
|
||||
response_mode="text",
|
||||
):
|
||||
return await transport.complete(
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
@@ -100,9 +101,19 @@ async def _complete(
|
||||
reasoning_effort=reasoning_effort,
|
||||
first_token_event=first_token_event,
|
||||
content_event=content_event,
|
||||
response_mode=response_mode,
|
||||
)
|
||||
|
||||
|
||||
async def test_complete_helper_forwards_response_mode():
|
||||
"""测试封装不得将显式保真模式静默退化成文本模式。"""
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
transport = AsyncMock()
|
||||
await _complete(transport, _source(), response_mode="openai")
|
||||
assert transport.complete.call_args.kwargs["response_mode"] == "openai"
|
||||
|
||||
|
||||
# 单价刻意取"输出贵于输入"的真实形态: est_tokens 兜底把整估值塞进 completion
|
||||
# 时,虚高才显形(设计 §1 的 26 倍算例即此单价)。
|
||||
_PRICING = PricingTable({"qwen-max": ModelPrice(input_per_1m=1.0, output_per_1m=8.0)})
|
||||
@@ -161,6 +172,8 @@ class TestSsePureFunctions:
|
||||
assert _sse_data_payload("data: [DONE]") == "[DONE]"
|
||||
|
||||
async def test_iter_deltas_yields_and_flags_done(self):
|
||||
from polygateway.transports._openai_completion import ChatCompletionAccumulator, ChatDelta
|
||||
|
||||
async def lines():
|
||||
for raw in [
|
||||
_chunk(content="he"),
|
||||
@@ -173,16 +186,28 @@ class TestSsePureFunctions:
|
||||
yield line
|
||||
|
||||
sink = {}
|
||||
out = [d async for d in _iter_sse_deltas(lines(), sink)]
|
||||
assert out == [(True, "he"), (False, "think")]
|
||||
out = [
|
||||
d
|
||||
async for d in _iter_sse_deltas(
|
||||
lines(), sink, ChatCompletionAccumulator(response_mode="text")
|
||||
)
|
||||
]
|
||||
assert out == [ChatDelta("he", "", True), ChatDelta("", "think", True)]
|
||||
assert sink["done"] is True and sink["usage"] == _USAGE
|
||||
|
||||
async def test_malformed_frame_raises_transient(self):
|
||||
from polygateway.transports._openai_completion import ChatCompletionAccumulator
|
||||
|
||||
async def lines():
|
||||
yield "data: {not-json"
|
||||
|
||||
with pytest.raises(TransientError, match="malformed"):
|
||||
_ = [d async for d in _iter_sse_deltas(lines(), {})]
|
||||
_ = [
|
||||
d
|
||||
async for d in _iter_sse_deltas(
|
||||
lines(), {}, ChatCompletionAccumulator(response_mode="text")
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
class TestStreamHappyPath:
|
||||
|
||||
@@ -0,0 +1,227 @@
|
||||
"""协议聚合器的分片、防御与快照契约。"""
|
||||
|
||||
import copy
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from polygateway.errors import TransientError
|
||||
from polygateway.transports._openai_completion import (
|
||||
ChatCompletionAccumulator,
|
||||
parse_chat_completion,
|
||||
)
|
||||
|
||||
|
||||
def chunk(delta, **extra):
|
||||
"""构造单 choice 协议帧。"""
|
||||
return (
|
||||
"data: "
|
||||
+ json.dumps({"id": "upstream-1", "choices": [{"index": 0, "delta": delta, **extra}]})
|
||||
+ "\n\n"
|
||||
)
|
||||
|
||||
|
||||
def tool_frames():
|
||||
"""两个交错分片工具，保留空白及 Unicode。"""
|
||||
return [
|
||||
chunk({"role": "assistant", "content": None}),
|
||||
chunk(
|
||||
{
|
||||
"tool_calls": [
|
||||
{
|
||||
"index": 1,
|
||||
"id": "b",
|
||||
"type": "function",
|
||||
"function": {"name": "echo", "arguments": '{"v":'},
|
||||
},
|
||||
{
|
||||
"index": 0,
|
||||
"id": "a",
|
||||
"type": "function",
|
||||
"function": {"name": "echo", "arguments": "{"},
|
||||
},
|
||||
]
|
||||
}
|
||||
),
|
||||
chunk(
|
||||
{
|
||||
"tool_calls": [
|
||||
{"index": 0, "function": {"arguments": '"v": "中" }'}},
|
||||
{"index": 1, "function": {"arguments": "2}"}},
|
||||
]
|
||||
},
|
||||
finish_reason="tool_calls",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def accumulate():
|
||||
"""复用双工具网关帧样本。"""
|
||||
state = ChatCompletionAccumulator(response_mode="openai")
|
||||
for frame in tool_frames():
|
||||
state.add_chunk(json.loads(frame.removeprefix("data: ")))
|
||||
return state
|
||||
|
||||
|
||||
@pytest.mark.parametrize("role", [None, "user", "tool"])
|
||||
def test_role_is_required_everywhere(role):
|
||||
"""缺角色不能被协议成功掩盖。"""
|
||||
state = accumulate()
|
||||
state.message.pop("role")
|
||||
if role:
|
||||
state.message["role"] = role
|
||||
with pytest.raises(TransientError, match="role"):
|
||||
state.finish()
|
||||
with pytest.raises(TransientError, match="role"):
|
||||
parse_chat_completion({"choices": [{"message": state.message}]})
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"change",
|
||||
[
|
||||
lambda t: t.pop("id"),
|
||||
lambda t: t.update(type="custom"),
|
||||
lambda t: t["function"].pop("name"),
|
||||
lambda t: t["function"].pop("arguments"),
|
||||
lambda t: t["function"].update(arguments=12),
|
||||
],
|
||||
)
|
||||
def test_incomplete_tool_is_not_success(change):
|
||||
"""工具必需字段不能靠正文兜底。"""
|
||||
state = accumulate()
|
||||
state.message["content"] = "text"
|
||||
change(state.tools[0])
|
||||
with pytest.raises(TransientError):
|
||||
state.finish()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"delta",
|
||||
[
|
||||
[],
|
||||
{"audio": {}},
|
||||
{"tool_calls": {}},
|
||||
{"content": []},
|
||||
{"tool_calls": [{"index": True}]},
|
||||
{"tool_calls": [{"index": -1}]},
|
||||
{"tool_calls": [{"index": 0, "extra": 1}]},
|
||||
{"tool_calls": [{"index": 0, "function": {"arguments": None}}]},
|
||||
],
|
||||
)
|
||||
def test_invalid_delta_is_domain_error(delta):
|
||||
"""畸形输入不泄漏裸类型错误。"""
|
||||
state = ChatCompletionAccumulator(response_mode="openai")
|
||||
with pytest.raises(TransientError):
|
||||
state.add_chunk({"choices": [{"index": 0, "delta": delta}]})
|
||||
|
||||
|
||||
def test_identity_repeat_does_not_refresh_progress():
|
||||
"""身份只初次生效算进展；同值重复不能续命。"""
|
||||
state = accumulate()
|
||||
repeated = {
|
||||
"choices": [
|
||||
{"index": 0, "delta": {"tool_calls": [{"index": 0, "id": "a", "type": "function"}]}}
|
||||
]
|
||||
}
|
||||
assert not state.add_chunk(repeated).progress
|
||||
repeated["choices"][0]["delta"]["tool_calls"][0]["id"] = "other"
|
||||
with pytest.raises(TransientError, match="冲突"):
|
||||
state.add_chunk(repeated)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("delta", [{"role": "assistant"}, {}, {"content": ""}])
|
||||
def test_metadata_not_progress(delta):
|
||||
"""空帧和角色不算 token。"""
|
||||
state = ChatCompletionAccumulator(response_mode="openai")
|
||||
assert not state.add_chunk({"choices": [{"index": 0, "delta": delta}]}).progress
|
||||
|
||||
|
||||
def test_nonstream_snapshot_and_length_preserved():
|
||||
"""非流式扩展字段与参数原文独立保存，不修复 length 产物。"""
|
||||
message = accumulate().finish().message
|
||||
message["extension"] = {"nested": [1]}
|
||||
message["tool_calls"][0]["function"]["arguments"] = '{"unfinished":'
|
||||
original = copy.deepcopy(message)
|
||||
result = parse_chat_completion(
|
||||
{
|
||||
"id": "id",
|
||||
"choices": [{"message": message, "finish_reason": "length"}],
|
||||
"usage": {"extra": 3},
|
||||
}
|
||||
)
|
||||
message["extension"]["nested"].append(2)
|
||||
assert result.message == original
|
||||
assert result.finish_reason == "length" and result.usage == {"extra": 3}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("choices", [[], [{"index": 1, "message": {}}], [{}, {}]])
|
||||
def test_nonstream_single_choice(choices):
|
||||
"""不会静默取第一个 choice。"""
|
||||
with pytest.raises(TransientError):
|
||||
parse_chat_completion({"choices": choices})
|
||||
|
||||
|
||||
def test_explicit_empty_tools_remains_present():
|
||||
"""显式空工具列表与缺省不同，不删除已接收字段。"""
|
||||
state = ChatCompletionAccumulator(response_mode="openai")
|
||||
state.add_chunk(
|
||||
{
|
||||
"choices": [
|
||||
{"index": 0, "delta": {"role": "assistant", "content": "ok", "tool_calls": []}}
|
||||
]
|
||||
}
|
||||
)
|
||||
assert state.finish().message["tool_calls"] == []
|
||||
|
||||
|
||||
def test_fragmented_name_and_arguments_with_usage():
|
||||
"""名称/参数分片逐字拼接，usage 扩展原样快照。"""
|
||||
state = ChatCompletionAccumulator(response_mode="openai")
|
||||
state.add_chunk(
|
||||
{
|
||||
"id": "response",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"delta": {
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{
|
||||
"index": 3,
|
||||
"id": "call",
|
||||
"type": "function",
|
||||
"function": {"name": "ec", "arguments": " "},
|
||||
}
|
||||
],
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
)
|
||||
progress = state.add_chunk(
|
||||
{
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"delta": {
|
||||
"content": "text",
|
||||
"reasoning_content": "reason",
|
||||
"tool_calls": [
|
||||
{"index": 3, "function": {"name": "ho", "arguments": "{} "}}
|
||||
],
|
||||
},
|
||||
"finish_reason": "length",
|
||||
}
|
||||
]
|
||||
}
|
||||
)
|
||||
assert (
|
||||
progress.content == "text" and progress.reasoning_content == "reason" and progress.progress
|
||||
)
|
||||
usage = {"prompt_tokens": 7, "completion_tokens": 2, "extra": {"nested": 9}}
|
||||
assert not state.add_chunk({"choices": [], "usage": usage}).progress
|
||||
result = state.finish()
|
||||
usage["extra"]["nested"] = 10
|
||||
assert result.message["tool_calls"][0]["function"] == {"name": "echo", "arguments": " {} "}
|
||||
assert result.usage["extra"]["nested"] == 9
|
||||
assert result.finish_reason == "length"
|
||||
@@ -0,0 +1,354 @@
|
||||
"""OpenAI 消息保真回归：按网关 Chat Completions 形态构造。"""
|
||||
|
||||
import json
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from polygateway.errors import AllSourcesExhausted, TransientError
|
||||
from tests.unit.test_client import _client
|
||||
from tests.unit.test_openai_compat import _complete, _source, _sse_stream, _transport_for
|
||||
from tests.unit.test_openai_completion import accumulate, chunk, tool_frames
|
||||
|
||||
|
||||
async def test_error_before_done_is_failure():
|
||||
"""文本模式也不能接受 error 后的 DONE。"""
|
||||
transport = _transport_for(
|
||||
lambda _: _sse_stream(chunk({"content": "partial"}), 'data: {"error": null}\n\n')
|
||||
)
|
||||
try:
|
||||
with pytest.raises(TransientError):
|
||||
await _complete(transport, _source())
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
async def test_mixed_content_reasoning_survive():
|
||||
"""同一帧双文本不互斥。"""
|
||||
transport = _transport_for(
|
||||
lambda _: _sse_stream(chunk({"content": "answer", "reasoning_content": "reason"}))
|
||||
)
|
||||
try:
|
||||
result = await _complete(transport, _source())
|
||||
assert (result.content, result.thinking) == ("answer", "reason")
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
async def test_tools_roundtrip_and_retry():
|
||||
"""真实 transport 经过治理，失败重试不污染工具，消息可直接往返。"""
|
||||
requests = []
|
||||
|
||||
def handler(request):
|
||||
requests.append(json.loads(request.content))
|
||||
if len(requests) == 1:
|
||||
return _sse_stream(
|
||||
chunk({"content": "partial"}),
|
||||
'data: {"error": {"code": "upstream_stream_read_error"}}\n\n',
|
||||
)
|
||||
return _sse_stream(*tool_frames())
|
||||
|
||||
client = _client(handler=handler)
|
||||
try:
|
||||
response = await client.chat(
|
||||
[{"role": "user", "content": "hi"}],
|
||||
response_mode="openai",
|
||||
overlay={"reasoning_effort": "max"},
|
||||
)
|
||||
assert response.call_stats.attempts == 2
|
||||
message = response.completion.message
|
||||
assert message["role"] == "assistant" and message["content"] is None
|
||||
assert [t["id"] for t in message["tool_calls"]] == ["a", "b"]
|
||||
assert message["tool_calls"][0]["function"]["arguments"] == '{"v": "中" }'
|
||||
assert response.completion.response_id == "upstream-1"
|
||||
assert response.completion.finish_reason == "tool_calls"
|
||||
messages = [message, {"role": "tool", "tool_call_id": "a", "content": "ok"}]
|
||||
await client.chat(messages, response_mode="openai")
|
||||
assert requests[-1]["messages"] == messages
|
||||
assert requests[0]["reasoning_effort"] == "max"
|
||||
finally:
|
||||
await client._transport.aclose()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("role", [None, "user"])
|
||||
async def test_final_role_required(role):
|
||||
"""非流式缺失或错误 role 必须协议失败。"""
|
||||
message = {"content": "ok"}
|
||||
if role:
|
||||
message["role"] = role
|
||||
client = _client(
|
||||
handler=lambda _: httpx.Response(200, json={"choices": [{"message": message}]})
|
||||
)
|
||||
try:
|
||||
with pytest.raises(AllSourcesExhausted):
|
||||
await client.chat(
|
||||
[{"role": "user", "content": "hi"}], stream=False, response_mode="openai"
|
||||
)
|
||||
finally:
|
||||
await client._transport.aclose()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stream", [True, False])
|
||||
async def test_completion_cache_roundtrip(stream):
|
||||
"""完整结果回放，修改返回对象不毒化缓存；模式隔离。"""
|
||||
from polygateway.backends.memory.cache import InMemoryCache
|
||||
|
||||
completion = accumulate().finish()
|
||||
|
||||
def handler(_):
|
||||
if stream:
|
||||
return _sse_stream(*tool_frames())
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "upstream-1",
|
||||
"choices": [{"message": completion.message, "finish_reason": "tool_calls"}],
|
||||
},
|
||||
)
|
||||
|
||||
client = _client(
|
||||
handler=handler, cache=InMemoryCache(), cache_namespace="tools", cache_ttl_s=60
|
||||
)
|
||||
try:
|
||||
first = await client.chat([], stream=stream, response_mode="openai")
|
||||
second = await client.chat([], stream=stream, response_mode="openai")
|
||||
assert not first.cache_hit and second.cache_hit
|
||||
assert second.completion == first.completion
|
||||
assert second.call_id != first.call_id and second.call_stats.attempts == 0
|
||||
second.completion.message["tool_calls"].clear()
|
||||
third = await client.chat([], stream=stream, response_mode="openai")
|
||||
assert len(third.completion.message["tool_calls"]) == 2
|
||||
finally:
|
||||
await client._transport.aclose()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"kwargs",
|
||||
[
|
||||
{"response_mode": "bad"},
|
||||
{"response_mode": "openai", "structured": "json"},
|
||||
{"response_mode": "openai", "overlay": {"n": 2}},
|
||||
{"response_mode": "openai", "overlay": {"n": True}},
|
||||
],
|
||||
)
|
||||
async def test_invalid_mode_combinations_fail_before_http(kwargs):
|
||||
"""组合错误入口拒绝。"""
|
||||
|
||||
def unexpected(_):
|
||||
pytest.fail("不应发送 HTTP")
|
||||
|
||||
client = _client(handler=unexpected)
|
||||
try:
|
||||
with pytest.raises(ValueError):
|
||||
await client.chat([], **kwargs)
|
||||
finally:
|
||||
await client._transport.aclose()
|
||||
|
||||
|
||||
async def test_openai_never_salvages():
|
||||
"""已收完整工具也必须有 DONE。"""
|
||||
transport = _transport_for(lambda _: _sse_stream(*tool_frames(), done=False))
|
||||
try:
|
||||
with pytest.raises(TransientError, match="DONE"):
|
||||
await transport.complete(
|
||||
messages=[],
|
||||
source=_source(missing_done="salvage"),
|
||||
stream=True,
|
||||
overlay={},
|
||||
call_id="c",
|
||||
reasoning_effort=None,
|
||||
first_token_event=None,
|
||||
content_event=None,
|
||||
response_mode="openai",
|
||||
)
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
async def test_tool_progress_sets_events():
|
||||
"""工具首进展驱动两个事件与时间观测。"""
|
||||
import asyncio
|
||||
|
||||
first, progress = asyncio.Event(), asyncio.Event()
|
||||
transport = _transport_for(lambda _: _sse_stream(*tool_frames()))
|
||||
try:
|
||||
result = await transport.complete(
|
||||
messages=[],
|
||||
source=_source(),
|
||||
stream=True,
|
||||
overlay={},
|
||||
call_id="c",
|
||||
reasoning_effort=None,
|
||||
first_token_event=first,
|
||||
content_event=progress,
|
||||
response_mode="openai",
|
||||
)
|
||||
assert first.is_set() and progress.is_set()
|
||||
assert result.ttft_ms is not None and result.stall_count == 0
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
async def test_real_transport_stall_hedge_cancels_loser():
|
||||
"""真实工具流沉默触发异源对冲，连接关闭且赢家不混入半成品。"""
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
from polygateway.sources import RoundRobinSelector
|
||||
from polygateway.types import ChatRequest, _CallContext
|
||||
from tests.unit.test_retry import _src
|
||||
from tests.unit.test_stall_hedge import _harness
|
||||
|
||||
entered, closed = asyncio.Event(), asyncio.Event()
|
||||
|
||||
class Hanging(httpx.AsyncByteStream):
|
||||
"""工具首帧后阻塞，取消由真实 httpx 流处理。"""
|
||||
|
||||
async def __aiter__(self):
|
||||
yield tool_frames()[0].encode()
|
||||
yield tool_frames()[1].encode()
|
||||
entered.set()
|
||||
await asyncio.Event().wait()
|
||||
|
||||
async def aclose(self):
|
||||
closed.set()
|
||||
|
||||
count = 0
|
||||
|
||||
def handler(_):
|
||||
nonlocal count
|
||||
count += 1
|
||||
if count == 1:
|
||||
return httpx.Response(200, stream=Hanging())
|
||||
return _sse_stream(*tool_frames())
|
||||
|
||||
transport = _transport_for(handler)
|
||||
sources = [_src("s1"), _src("s2")]
|
||||
mw, limiter, *_ = _harness(
|
||||
sources, transport, stall_after_s=0.05, selector=RoundRobinSelector()
|
||||
)
|
||||
context = _CallContext(now=time.monotonic)
|
||||
try:
|
||||
response = await asyncio.wait_for(
|
||||
mw(ChatRequest(messages=[], response_mode="openai", call_context=context)), 3
|
||||
)
|
||||
assert entered.is_set() and closed.is_set()
|
||||
assert response.source_name == "s2" and len(response.completion.message["tool_calls"]) == 2
|
||||
assert context.snapshot().hedges == 1 and context.snapshot().attempts == 2
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stream", [True, False])
|
||||
async def test_bad_usage_is_domain_failure(stream):
|
||||
"""坏 usage 不能穿透成裸 AttributeError，包括默认文本模式。"""
|
||||
transport = _transport_for(
|
||||
lambda _: (
|
||||
_sse_stream(chunk({"content": "ok"}), 'data: {"choices": [], "usage": [1]}\n\n')
|
||||
if stream
|
||||
else httpx.Response(
|
||||
200, json={"choices": [{"message": {"content": "ok"}}], "usage": [1]}
|
||||
)
|
||||
)
|
||||
)
|
||||
try:
|
||||
with pytest.raises(TransientError):
|
||||
await _complete(transport, _source(), stream=stream)
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
async def test_nonstream_tool_result_has_no_liveness_events():
|
||||
"""非流式工具不制造流式观测。"""
|
||||
import asyncio
|
||||
|
||||
completion = accumulate().finish()
|
||||
first, progress = asyncio.Event(), asyncio.Event()
|
||||
transport = _transport_for(
|
||||
lambda _: httpx.Response(
|
||||
200, json={"choices": [{"message": completion.message, "finish_reason": "tool_calls"}]}
|
||||
)
|
||||
)
|
||||
try:
|
||||
result = await transport.complete(
|
||||
messages=[],
|
||||
source=_source(),
|
||||
stream=False,
|
||||
overlay={},
|
||||
call_id="c",
|
||||
reasoning_effort=None,
|
||||
first_token_event=first,
|
||||
content_event=progress,
|
||||
response_mode="openai",
|
||||
)
|
||||
assert result.completion.message == completion.message
|
||||
assert not first.is_set() and not progress.is_set()
|
||||
assert result.ttft_ms is None and result.stall_ms is None
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
async def test_openai_does_not_bypass_managed_conflict():
|
||||
"""保真模式不静默撤销源级受管意图。"""
|
||||
from polygateway.errors import RequestRejectedError
|
||||
|
||||
def unexpected(_):
|
||||
pytest.fail("冲突请求不应发送 HTTP")
|
||||
|
||||
transport = _transport_for(unexpected)
|
||||
try:
|
||||
with pytest.raises(RequestRejectedError):
|
||||
await transport.complete(
|
||||
messages=[],
|
||||
source=_source(enable_thinking=True),
|
||||
stream=True,
|
||||
overlay={"reasoning_effort": "max"},
|
||||
call_id="c",
|
||||
reasoning_effort=None,
|
||||
first_token_event=None,
|
||||
content_event=None,
|
||||
response_mode="openai",
|
||||
)
|
||||
finally:
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
async def test_cancel_tool_stream_releases_resources():
|
||||
"""显式取消真实工具流，连接关闭且配额归还。"""
|
||||
import asyncio
|
||||
|
||||
from polygateway.backends.memory.limiter import InMemoryLimiter
|
||||
from polygateway.types import GlobalLimits
|
||||
|
||||
entered, closed = asyncio.Event(), asyncio.Event()
|
||||
|
||||
class Hanging(httpx.AsyncByteStream):
|
||||
"""可确定取消窗口的 HTTP 流。"""
|
||||
|
||||
async def __aiter__(self):
|
||||
for frame in tool_frames()[:2]:
|
||||
yield frame.encode()
|
||||
entered.set()
|
||||
await asyncio.Event().wait()
|
||||
|
||||
async def aclose(self):
|
||||
closed.set()
|
||||
|
||||
source = _source(max_concurrency=1)
|
||||
limiter = InMemoryLimiter(
|
||||
scope="llm", sources={source.name: source}, global_limits=GlobalLimits(1, 0, 0)
|
||||
)
|
||||
client = _client(
|
||||
sources=[source], limiter=limiter, handler=lambda _: httpx.Response(200, stream=Hanging())
|
||||
)
|
||||
task = asyncio.create_task(client.chat([], response_mode="openai"))
|
||||
try:
|
||||
await asyncio.wait_for(entered.wait(), 2)
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
assert closed.is_set()
|
||||
assert (await limiter.source_stats(source.name)).inflight == 0
|
||||
finally:
|
||||
task.cancel()
|
||||
await client._transport.aclose()
|
||||
@@ -88,3 +88,15 @@ def test_every_promised_export_is_actually_importable():
|
||||
"""
|
||||
missing = [name for name in polygateway.__all__ if not hasattr(polygateway, name)]
|
||||
assert not missing, f"__all__ 承诺了但没绑上的符号: {missing}"
|
||||
|
||||
|
||||
def test_apache_license_metadata():
|
||||
"""许可正文和包元数据同时存在，不以版本说明替代许可。"""
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
assert "Apache License" in (root / "LICENSE").read_text()
|
||||
assert "Version 2.0" in (root / "LICENSE").read_text()
|
||||
project = tomllib.loads((root / "pyproject.toml").read_text())["project"]
|
||||
assert project["license"] == {"file": "LICENSE"}
|
||||
|
||||
@@ -83,6 +83,7 @@ class _DummyTransport:
|
||||
reasoning_effort,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
):
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
@@ -90,6 +90,7 @@ class FakeTransport:
|
||||
reasoning_effort,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
):
|
||||
self.calls.append((source.name, call_id))
|
||||
self.efforts.append(reasoning_effort)
|
||||
@@ -132,6 +133,7 @@ class _GenClockTransport:
|
||||
reasoning_effort,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
):
|
||||
advance = self._advances.pop(0)
|
||||
stall = self._stalls.pop(0) if self._stalls is not None else None
|
||||
@@ -144,6 +146,7 @@ class _GenClockTransport:
|
||||
reasoning_effort=reasoning_effort,
|
||||
first_token_event=first_token_event,
|
||||
content_event=content_event,
|
||||
response_mode=response_mode,
|
||||
)
|
||||
if stall is not None:
|
||||
result = dataclasses.replace(result, stall_ms=stall[0], stall_count=stall[1])
|
||||
|
||||
@@ -78,6 +78,7 @@ class StallHedgeTransport:
|
||||
reasoning_effort,
|
||||
first_token_event,
|
||||
content_event,
|
||||
response_mode,
|
||||
):
|
||||
self.calls.append(source.name)
|
||||
self.ft_events.append(first_token_event)
|
||||
@@ -271,9 +272,7 @@ class TestStallHedgeTrigger:
|
||||
"""参数化四格(设计 §5.2 逐字): 两阈值独立生效,各格触发语义不串。"""
|
||||
# 格 1: 仅 AFTER_S + 流式 pre-TTFT 挂起 → AFTER_S 触发(1.3.7 语义不变)
|
||||
s1, s2 = _src("s1"), _src("s2")
|
||||
transport = StallHedgeTransport(
|
||||
{"s1": [("hang",), ("ok", "late")], "s2": [("ok", "fast")]}
|
||||
)
|
||||
transport = StallHedgeTransport({"s1": [("hang",), ("ok", "late")], "s2": [("ok", "fast")]})
|
||||
mw, _, _ = _harness([s1, s2], transport, hedge_after_s=_STALL_AFTER_S)
|
||||
ctx = _ctx()
|
||||
resp = await asyncio.wait_for(mw(_req(ctx=ctx)), timeout=5)
|
||||
@@ -312,9 +311,7 @@ class TestStallHedgeTrigger:
|
||||
|
||||
# 格 3b: 两者皆设 + pre-TTFT 挂起 → 阶段 1 AFTER_S 触发(两阶段各管一段)
|
||||
s1, s2 = _src("s1"), _src("s2")
|
||||
transport = StallHedgeTransport(
|
||||
{"s1": [("hang",), ("ok", "late")], "s2": [("ok", "fast")]}
|
||||
)
|
||||
transport = StallHedgeTransport({"s1": [("hang",), ("ok", "late")], "s2": [("ok", "fast")]})
|
||||
mw, _, _ = _harness(
|
||||
[s1, s2], transport, hedge_after_s=_STALL_AFTER_S, stall_after_s=_STALL_AFTER_S
|
||||
)
|
||||
@@ -340,9 +337,7 @@ class TestStallHedgeTrigger:
|
||||
async def test_stall_only_non_stream_degrades_to_single_leg(self):
|
||||
"""仅 stall 已设 + `stream=False` → 逐字单路(中段沉默无非流式概念,入口门)。"""
|
||||
s1, s2 = _src("s1"), _src("s2")
|
||||
transport = StallHedgeTransport(
|
||||
{"s1": [("hang",), ("ok", "x")], "s2": [("ok", "other")]}
|
||||
)
|
||||
transport = StallHedgeTransport({"s1": [("hang",), ("ok", "x")], "s2": [("ok", "other")]})
|
||||
mw, _, _ = _harness([s1, s2], transport, stall_after_s=_STALL_AFTER_S)
|
||||
ctx = _ctx()
|
||||
task = asyncio.ensure_future(mw(_req(stream=False, ctx=ctx)))
|
||||
|
||||
@@ -141,6 +141,7 @@ async def test_salvage_override_stays_in_domain(usage):
|
||||
reasoning_effort=None,
|
||||
first_token_event=None,
|
||||
content_event=None,
|
||||
response_mode="text",
|
||||
)
|
||||
assert result.usage_source in USAGE_SOURCES
|
||||
|
||||
|
||||
Reference in New Issue
Block a user