iomgaa/PolyGateway
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases 17 Wiki Activity

feat: add a content event and stall accounting to the transport port

Browse Source 
T1(计划 §4):观测原语与端口契约(issue #27/#28 底层)。

- Transport.complete 加必填 kw content_event(顺序在 first_token_event
  后,无默认值,同款约定):流式每增量 chunk 置位一次,非流式/对冲路
  恒不置位,None = 不观测
- TransportResult 增 stall_ms/stall_count(带默认 None):超阈值相邻
  增量 chunk 沉默总额与段数,chunk<2/非流式 → None,尾部不计
- SourceConfig 增 stall_threshold_ms(默认 1000,正整数拒 bool,
  _validate_watchdog 同款落点)
- _complete_stream 记增量 chunk 时间戳(空 chunk/usage 帧/keepalive
  不产生记录点)+ 模块级纯函数 _stall_from_chunk_times(严格大于
  阈值才减);max_inter_token_ms 原语义一字不动
- §2.2 同步清单全量:11 实现 + 11 调用点(重跑 grep 复核零遗漏)
- 批次 A 12 用例先红(TypeError/ImportError/AttributeError)后绿,
  证据 tests/outputs/138/t1/
This commit is contained in:
iomgaa
2026-09-14 00:35:35 -04:00
parent 6f3a620c73
commit 901a193514
15 changed files with 398 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/polygateway/middleware/retry.py
+8 -1
View File
@@ -273,6 +273,7 @@ class RetryMW:
reasons,
attempt_fails,
first_token_event=None,
content_event=None,
generation_sink=generation_sink,
)
else:
@@ -312,6 +313,7 @@ class RetryMW:
attempt_fails: dict[str, int],
*,
first_token_event: asyncio.Event | None,
content_event: asyncio.Event | None,
generation_sink: list[int],
) -> LLMResponse | _Failed:
call_id = str(uuid.uuid4())
@@ -342,6 +344,9 @@ class RetryMW:
# 对冲编排(计划 §3.4): 原路携带首 token 事件,对冲路恒 None(v1 单路,
# 不再梯次);未启用对冲时 __call__ 传 None = 调用方不观测首 token
first_token_event=first_token_event,
# 中段沉默处置位(1.3.8): 本阶段(T1)所有调用点恒传 None = 不观测;
# T4 才在启用 stall 对冲的流式原路上接真事件(设计 §5.2 N3③)
content_event=content_event,
)
generation_sink.append(int((self._now() - gen_started) * 1000))
if result.usage_source == "unavailable":
@@ -434,6 +439,7 @@ class RetryMW:
reasons,
attempt_fails,
first_token_event=first_token,
content_event=None,
generation_sink=sink_p,
)
)
@@ -473,7 +479,7 @@ class RetryMW:
self._record_generation(request, sink_p)
return outcome
# Phase 4 启动对冲路(v1 单路,H5): 对冲路恒传 first_token_event=None,
# 不再触发梯次对冲
# 不再触发梯次对冲;content_event 同恒 None(N3③,v1 不梯次)
sink_h: list[int] = []
hedge = asyncio.create_task(
self._attempt(
@@ -482,6 +488,7 @@ class RetryMW:
reasons,
attempt_fails,
first_token_event=None,
content_event=None,
generation_sink=sink_h,
)
)
src/polygateway/ports.py
+7
View File
@@ -51,6 +51,12 @@ class Transport(Protocol):
`first_token_event` 同一约定(1.3.7 对冲 H2): `None` = 调用方不观测首 token
(未启用对冲);非流式实现**永不置位**(物理上无中途信号,事件自然退化为纯时间
阈值),流式实现在首个增量(内容或思考)到达时置位。
`content_event` 同一约定(1.3.8 中段沉默对冲,issue #27/#28): `None` = 调用方
不观测增量 chunk(未启用中段对冲);**流式**实现每到达一个增量 chunk(`content`
或 `reasoning_content`,与看门狗/首 token 活性口径一致)置位一次;空 chunk、
usage 帧、SSE keepalive 不算;非流式实现**永不置位**;对冲路恒收 `None`
(v1 不梯次,设计 §5.2 N3③)。
"""
async def complete(
@@ -63,6 +69,7 @@ class Transport(Protocol):
call_id: str,
reasoning_effort: Effort | None,
first_token_event: asyncio.Event | None,
content_event: asyncio.Event | None,
) -> TransportResult: ...
src/polygateway/transports/openai_compat.py
+45 -3
View File
@@ -85,6 +85,29 @@ def _sse_delta(chunk: dict[str, Any], usage_sink: dict[str, Any]) -> tuple[bool,
return None
def _stall_from_chunk_times(
times: list[float], threshold_ms: int
) -> tuple[float | None, float | None]:
"""增量 chunk 时间戳序列 → (stall_ms, stall_count);chunk <2 → (None, None)。
沉默 = 相邻间隔**严格大于**阈值(恰好 = 阈值不减,pi-tps 口径);
尾部(末 chunk → 流结束)不计——墙钟基底口径,尾部归活跃侧(设计 §4.2 N1)。
无超阈段但 chunk ≥2 时返回 (0.0, 0.0) 而非 None——"观测过且零沉默"与
"无从观测"是两回事,下游预算公式以 `stall_ms is None` 为分支条件(设计 §4.2 N2)。
"""
if len(times) < 2:
return None, None
total = 0.0
count = 0
# strict=False: 两序列恒差 1(times[1:] 少末尾),B905 要求显式表态——不等长是本设计的本意
for prev, cur in zip(times, times[1:], strict=False):
gap_ms = (cur - prev) * 1000
if gap_ms > threshold_ms:
total += gap_ms
count += 1
return (total, float(count)) if count else (0.0, 0.0)
async def _iter_sse_deltas(
lines: AsyncIterator[str], usage_sink: dict[str, Any]
) -> AsyncIterator[tuple[bool, str]]:
@@ -434,6 +457,7 @@ class OpenAICompatTransport:
call_id: str,
reasoning_effort: Effort | None,
first_token_event: asyncio.Event | None,
content_event: asyncio.Event | None,
) -> TransportResult:
"""一次原始调用;HTTP/线路/流式异常按 ARCH §6.2 翻译为领域错误。
@@ -441,6 +465,8 @@ class OpenAICompatTransport:
在 `_build_payload` 里由 `effective_effort` 裁定,本层只负责把它送到。
`first_token_event` 为对冲处置位: `None` = 不观测首 token;仅流式路径
(`_complete_stream`)在首个增量到达时置位,非流式路径收它但永不置位。
`content_event` 为中段沉默对冲处置位(1.3.8): 同款约定,流式路径**每个**
增量 chunk 置位一次,非流式路径收它但永不置位。
"""
profile = get_provider(source.provider, registry=self._registry)
try:
@@ -466,11 +492,11 @@ class OpenAICompatTransport:
try:
if stream:
result = await self._complete_stream(
client, url, payload, source, profile, first_token_event
client, url, payload, source, profile, first_token_event, content_event
)
else:
result = await self._complete_once(
client, url, payload, source, profile, first_token_event
client, url, payload, source, profile, first_token_event, content_event
)
except StreamLivenessTimeout as exc:
raise TransientError(f"{source.name} 流活性超时({exc.kind})", **ctx) from exc
@@ -558,6 +584,7 @@ class OpenAICompatTransport:
source: SourceConfig,
profile: ProviderProfile,
first_token_event: asyncio.Event | None,
content_event: asyncio.Event | None,
) -> TransportResult:
started = time.monotonic()
async with client.stream("POST", url, json=payload) as resp:
@@ -578,8 +605,17 @@ class OpenAICompatTransport:
ttft_ms: float | None = None
last = started
max_gap = 0.0
chunk_times: list[float] = []
async for is_content, text in guarded:
now = time.monotonic()
# 增量 chunk 时间戳(1.3.8 沉默计量): 循环体只会被真实增量触发——
# 空 chunk/usage 帧/SSE keepalive 已在 _sse_data_payload/_sse_delta
# 被过滤,这里逐字追加即可;阈值判定不在热路径,派生收敛在
# _stall_from_chunk_times(设计 §3 派生原则)
chunk_times.append(now)
if content_event is not None:
# 每个增量 chunk 置位一次: 中段沉默对冲的活性复位信号
content_event.set()
if ttft_ms is None:
ttft_ms = (now - started) * 1000
# 首个增量即对冲语义上的"首 token"(思考增量同样是存活证据,
@@ -595,6 +631,7 @@ class OpenAICompatTransport:
self._reject_empty_completion(content, source)
prompt, completion, usage_source = _resolve_stream_usage(sink, salvaged)
reasoning_tokens = _coerce_reasoning_tokens(sink.get("usage"))
stall_ms, stall_count = _stall_from_chunk_times(chunk_times, source.stall_threshold_ms)
return TransportResult(
content=content,
thinking=thinking,
@@ -603,6 +640,8 @@ class OpenAICompatTransport:
usage_source=usage_source,
ttft_ms=ttft_ms,
max_inter_token_ms=(max_gap if ttft_ms is not None else None),
stall_ms=stall_ms,
stall_count=stall_count,
raw={"usage": sink.get("usage")},
cached_prompt_tokens=_coerce_cached_tokens(sink.get("usage")),
model_reported=_coerce_model_reported(sink.get("model")),
@@ -664,11 +703,14 @@ class OpenAICompatTransport:
source: SourceConfig,
profile: ProviderProfile,
first_token_event: asyncio.Event | None,
content_event: asyncio.Event | None,
) -> TransportResult:
"""非流式快路径(三项目均无,库新增): 单 JSON 响应,仅 total 超时。
接收 `first_token_event` 但**永不置位**: 非流式无中途信号,事件自然退化
为纯时间阈值(对冲只能靠 `hedge_after_s` 触发)。
为纯时间阈值(对冲只能靠 `hedge_after_s` 触发)。`content_event`(1.3.8)
同款: 接收但**永不置位**——非流式无"中段"概念,stall 两值走
TransportResult 默认 None(排除条款)。
"""
resp = await client.post(url, json=payload)
if resp.status_code != 200:
src/polygateway/types.py
+31
View File
@@ -585,6 +585,17 @@ class TransportResult:
`medium` → 实发 `low`),中间件事后再算一遍必然算成请求档。默认 `None` 是
"未表态/不注入推理参数"(OCR、embedding 等 transport 沉默即此)。"""
stall_ms: float | None = None
"""超阈值相邻增量 chunk 沉默总额(1.3.8, issue #27;毫秒,未取整——换算唯一处
在 retry 层,设计 §4.2 N8)。
非流式/增量 chunk <2 → `None`("无从观测");观测过但零超阈沉默是 `0.0`,
两者不得混——下游预算公式以 `stall_ms is None` 为分支条件(设计 §4.2 N2)。
尾部(末 chunk → 流结束)不计,归活跃侧(设计 §4.2 N1)。"""
stall_count: float | None = None
"""超阈值相邻间隔个数(归因成对报告,Gregg 原则);值域与 `None` 语义同
`stall_ms`。"""
@dataclass(frozen=True)
class SourceConfig:
@@ -638,10 +649,19 @@ class SourceConfig:
后者对未知值是 fail-closed(按 `error` 处理),不会替配置兜错,漏判的结果
就是 `EFFORT_FALLBAK` 这种拼写错误静默失效。"""
stall_threshold_ms: int = 1000
"""流式中段沉默的计量阈值(1.3.8, issue #27): 相邻增量 chunk 间隔**严格大于**
本值才计入 `TransportResult.stall_ms`/`stall_count`。
纯计量口径,不触发任何动作;触发中段对冲的动作阈值是 scope 级
`HEDGE__STALL_AFTER_S`(设计 §6 两参数分工)。scope 级键
`STREAM__STALL_THRESHOLD_MS` 经装配层 `dataclasses.replace` 分发进本字段。"""
def __post_init__(self) -> None:
self._validate_identity()
self._validate_gates()
self._validate_watchdog()
self._validate_stall_threshold()
self._validate_thinking()
self._freeze_extra_body()
@@ -680,6 +700,17 @@ class SourceConfig:
):
raise ValueError("看门狗不变式要求 0 < inter_token < ttft < timeout_s")
def _validate_stall_threshold(self) -> None:
# bool 是 int 子类: True/False 会被当成 1/0 静默放行,必须显式拒(P5)
if (
isinstance(self.stall_threshold_ms, bool)
or not isinstance(self.stall_threshold_ms, int)
or self.stall_threshold_ms <= 0
):
raise ValueError(
f"SourceConfig.stall_threshold_ms 必须是正整数: {self.stall_threshold_ms!r}"
)
def _validate_thinking(self) -> None:
"""推理两键的**归一化**、值域与互不矛盾(issue #20 设计 §4.2)。