docs: document the optional call deadline and cancellation settlement
The deadline governs waiting, not the moment a call returns: cleanup still runs in finally, so the return time is the deadline plus the cleanup cost (5-7x the deadline in the measured fixture). An expiry therefore does not mean nothing was produced or nothing was billed. Leaving the key unset keeps 1.3.5 semantics verbatim, which also keeps its two long waits: a pure 429 sequence can still wait for a long time, and a large finite Retry-After is still slept in full because the library deliberately does not clamp the hint with backoff_max_s. - CHANGELOG unreleased section states those three sentences, records the cancellation settlement change (an attempt cancelled after the port started but before the settlement is known keeps its reservation at the source estimate: over-charge rather than refund something the upstream may already have billed; known settlements and unclassified escapes are untouched) and warns that except GatewayUnavailableError does not catch CallDeadlineExceeded - README in four places: the capability table, the exception handling example, the "which exceptions reach the caller" table and the error model section, which now spells out the remaining large-but-finite Retry-After wait - .env.example documents LLM__CALL_DEADLINE_S as commented out - new findings file indexes the red/green evidence, the commands and their exit codes, what was not run and who covers it, and repeats the three residual risks No version bump, no tag, no release: those belong to the release checklist.
This commit is contained in:
@@ -16,6 +16,7 @@
|
||||
| 熔断 | 双通道(连续失败 + 失败率窗口,健康证据抑制误熔);半开单探针带租约(持有者死亡自动回收);epoch fencing 拒绝迟到写回;开路时长指数递增;**开路时当场失败还是等冷却可配**(`CIRCUIT_OPEN`,单源 scope 应配 `wait`) |
|
||||
| 自适应并发 | AIMD:429 削减、成功缓升,防止打爆上游 |
|
||||
| 背压与判死 | 配额满与熔断开路**各自**可选等待或快速失败(`QUOTA_FULL` / `CIRCUIT_OPEN`,两键不可互相替代);等待期按双条件判死(本地非生产性等待与全局无进展**同时**超窗)。stall 窗口只计**非生产性**等待(429 退避/配额轮询/熔断冷却),与 `TIMEOUT_S` 无耦合 |
|
||||
| 调用期限 | 一次逻辑调用可选一条**墙钟硬边界**(`{SCOPE}__CALL_DEADLINE_S` 或 `chat(call_deadline_s=...)`,缺省不启用):治理的是**等待**——重试退避、配额轮询、熔断冷却、结构化重问与 embedding 分批共享同一份期限。三条须知:①**返回时刻 = 期限 + 清理耗时**(遥测/结算/缓存收尾在 `finally` 里跑完,允许超期;实测构造达期限的 5–7 倍),库只承诺切断等待、不给返回上界;②**到期 ≠ 未产出、≠ 未计费**,上游可能已算完并计费;③**不配置即逐字保持 1.3.5 语义**(纯 429 序列仍可能长等、有限大 `Retry-After` 仍照睡)。到期抛 `CallDeadlineExceeded`,**不属四分类、不属 `GatewayUnavailableError` 族** |
|
||||
| 响应缓存 | Redis/内存;key 含 model + messages 摘要 + namespace(缓存隔离单位)+ salt + 采样参数 + 请求级推理档位(同 messages 跑 low 与 max 不互相命中),多模态 content 先摘要再 hash(防毒化);可 per-call 绕过(科研重采样) |
|
||||
| 流式看门狗 | TTFT / inter-token / 总超时三层活性;thinking token 刷活性不计结果;截断流(缺 `[DONE]`)判瞬时不入缓存 |
|
||||
| 推理可观测性 | "这次到底推理没推理"由多信号裁定(推理正文压倒 usage 明细),三态落在 `LLMResponse.thinking_observation`:`observed` / `absent` / `unknown`——**`unknown` 是"本次判不出",不是"没推理"**;本次实发档位与实测观测矛盾时按 `(源, 模型, 生效档位)` 各告警一次(能力表过期、开启未生效、注入了却观测不到;同一模型的 low 与 max 是两个独立的矛盾,不共用节流键);裁定结果随遥测落库 |
|
||||
@@ -185,13 +186,17 @@ vectors = (await embed.embed(["文本 a", "文本 b"])).vectors
|
||||
### 4. 业务侧异常处理
|
||||
|
||||
```python
|
||||
from polygateway import GatewayUnavailableError, RequestRejectedError
|
||||
from polygateway import CallDeadlineExceeded, GatewayUnavailableError, RequestRejectedError
|
||||
|
||||
try:
|
||||
resp = await client.chat(messages)
|
||||
except GatewayUnavailableError as exc:
|
||||
# 整个 scope 暂时无源可用: 延期重投,不消耗业务失败预算
|
||||
schedule_retry(after_s=exc.retry_after_s) # exc.reason / exc.per_source_reasons 供诊断
|
||||
except CallDeadlineExceeded as exc:
|
||||
# 只在自己配了调用期限时出现: 不在 GatewayUnavailableError 族内,上一条接不住;
|
||||
# 且无 retry_after_s(到期不含"何时可再试"),重投时机由业务侧定
|
||||
schedule_retry(after_s=None) # exc.scope / exc.deadline_s 供诊断
|
||||
except RequestRejectedError:
|
||||
... # 请求本身有问题(400/格式拒绝): 不重试,直接失败
|
||||
```
|
||||
@@ -463,6 +468,8 @@ SQLite 侧**不建议**对着一个大库文件跑 `DELETE` + `VACUUM`,而应**
|
||||
|
||||
**网关拒绝的理由不会丢失**(1.2.0 起):非 2xx 的响应体经折叠与截断后同时进入异常 message 与 `exc.body_text`,故遥测表的 `error` 列里就能看到网关的原话——不必再为查一次 400 单独埋点。截断保头保尾(总长 2048 字符),JSON 错误体尾部的 `code` / `request_id` 不会被切掉。**经中转部署时请注意**:第三方中转服务自身抖动也会回 400,从状态码上与"你的输入有问题"无法区分;库仍按确定性失败处理(直连供应商时重试只会白烧配额),批处理下游宜据 `body_text` 自备兜底分类。
|
||||
|
||||
**有限大的 `Retry-After` 仍照睡**: 能力表那条「尊重 `Retry-After`」是字面意思——库**有意不用 `backoff_max_s` 去夹服务端给的提示**(夹住就是提前重打已明确说「还没好」的网关),服务端给 3600s 就真睡 3600s;**唯一的制约手段是 1.3.6 的调用期限**(上表「调用期限」行)。自 1.3.6 起,`inf` / `-inf` / `1e999` 这类**非有限**取值按「无提示」处理(退回纯指数退避 + 一条带源名的 warning),不再造成无限等待;`nan`、空串、负数、HTTP-date 的既有语义不变。
|
||||
|
||||
### 哪些异常会到达调用方
|
||||
|
||||
上表的"库内行为"一列描述的是**治理动作**,不是调用方要处理的东西。四类里有两类**根本到不了调用方**——它们被重试循环接住,预算耗尽时统一包成 `AllSourcesExhausted`。这个区分只看类型树和 docstring 是读不出来的,曾让下游据此写错整段设计文档,故在此列明:
|
||||
@@ -473,9 +480,12 @@ SQLite 侧**不建议**对着一个大库文件跑 `DELETE` + `VACUUM`,而应**
|
||||
| `RequestRejectedError` | `SourceDeadError`(立即熔断该源并换源,同上) |
|
||||
| `ResultInvalidError` | |
|
||||
| `SourceNotConfiguredError` | |
|
||||
| `CallDeadlineExceeded`(1.3.6 起) | |
|
||||
|
||||
**`GovernanceBackendError` 属于第一列**: 限流/熔断的状态后端(如 Redis)自身故障时库 fail-closed——一个请求都发不出去,这就是"整个 scope 暂时不可用"。它继承 `GatewayUnavailableError`,所以 §4 那段 `except GatewayUnavailableError` 一条即覆盖完整,无需为它单列分支。`retry_after_s` 默认 5 秒(后端恢复时间不可知,取 0 会让积压任务零延迟冲击已挂掉的后端)。
|
||||
|
||||
**`CallDeadlineExceeded` 既不属四分类、也不属 `GatewayUnavailableError` 族**: 它直接继承 `PolyGatewayError`,表达的是「调用方自己设的墙钟边界到了」而非网关不可用,故**只有显式配了 `call_deadline_s` / `{SCOPE}__CALL_DEADLINE_S` 的调用方才可能遇到它**,不配就永远不会出现。它**无 `retry_after_s`**,且 §4 那段 `except GatewayUnavailableError` **接不住**它——要处理就得单列一条分支。
|
||||
|
||||
**`SourceNotConfiguredError` 有意不在第一列的族内**: 源名不在限流后端的配置字典中是**装配缺陷**而非暂时故障,它应当消耗失败预算、进死信、让人看见——归入可重投家族只会让配置写错的任务永远重投且无人告警。
|
||||
|
||||
## 配置参考
|
||||
|
||||
Reference in New Issue
Block a user