feat: record which tier a call actually ran at
Twenty-five columns and not one of them answered "which tier was this?", so the question the whole issue exists to settle - does a higher tier buy anything - had no way to group its data. The three emit entry points deliberately disagree, the way sampling already does. A successful attempt records what the transport actually sent: with EFFORT_FALLBACK=nearest a request for medium goes out as low, and recomputing here would file the row under a tier that never left the process. A failed attempt has no response to read, so it falls back to the requested tier - which is exactly right for the tier errors that are rejected before any HTTP happens, because the rejected tier is the signal. Cache hits and terminal failures have no chosen source at all, so a source-level tier is not a thing they could report. emit_attempt now demands to be told whether the path reasons at all. Embedding and OCR share the emitter but never send reasoning parameters; without the flag a source that mistakenly carries ENABLE_THINKING would hang a tier on a call that could not possibly have run at one. The value lands as a plain str. StrEnum is a str subclass and asyncpg promises nothing about encoding subclasses, and a telemetry write that fails is only a warning - Postgres would just quietly lose the column. NULL means nobody declared a tier, which is not the same statement as 'none', and the two must never be folded together.
This commit is contained in:
@@ -23,7 +23,8 @@ from polygateway.errors import (
|
||||
SourceNotConfiguredError,
|
||||
)
|
||||
from polygateway.middleware.cache import digest_messages
|
||||
from polygateway.types import ThinkingObservation, canonical_sampling_json, merge_sampling
|
||||
from polygateway.thinking import effective_effort
|
||||
from polygateway.types import Effort, ThinkingObservation, canonical_sampling_json, merge_sampling
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable, Mapping
|
||||
@@ -80,6 +81,70 @@ def _normalize_observation(raw: object) -> str:
|
||||
return ThinkingObservation.UNKNOWN.value
|
||||
|
||||
|
||||
def _normalize_effort(raw: object) -> str | None:
|
||||
"""实际档位 → 落库用的裸 str;不表态与域外取值都落 `NULL`。
|
||||
|
||||
**不写 `raw.value`**,理由与 `_normalize_observation` 逐字相同: `LLMResponse`
|
||||
是无运行时校验的 frozen dataclass,测试替身写 `applied_effort="low"` 完全自然,
|
||||
而 `.value` 会当场抛 `AttributeError`,被 `_record` 的 `except Exception` 吞成
|
||||
一条泛化 warning —— 丢的不是这一列,是**整行**。
|
||||
|
||||
域外取值降级为 `None` 而不抛,方向与 `CacheMW._coerce_applied_effort` 一致
|
||||
(设计 §4.4): 多项目共用一套后端时,更新版本的进程可能带来本版没有的档位名,
|
||||
归因字段不该有能力废掉一整行遥测。降级到 `None` 也是唯一诚实的说法——库确实
|
||||
不知道这次跑在哪档,随便挑一档等于替上游声称了一件它没说过的事。
|
||||
|
||||
注意 `None` 在本列有**两个**来源(不表态 / 读不懂),二者都不可折叠进 `'none'`:
|
||||
`'none'` 是"明确要求不推理",是一次表态。
|
||||
"""
|
||||
if raw is None:
|
||||
return None
|
||||
try:
|
||||
return Effort(raw).value
|
||||
except ValueError:
|
||||
logger.warning(
|
||||
"推理档位取值 {!r} 不在本版档位词汇内,本行 reasoning_effort 降级记为 NULL"
|
||||
"(其余列照常落库)",
|
||||
raw,
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def _attempt_effort(
|
||||
*,
|
||||
request: ChatRequest,
|
||||
source: SourceConfig,
|
||||
response: LLMResponse | None,
|
||||
applies: bool,
|
||||
) -> str | None:
|
||||
"""一次尝试该记哪一档: 成功读**实发档**,失败退回**请求档**(设计 §6)。
|
||||
|
||||
成功行一律读 `response.applied_effort` 而**绝不重算**: 源上开了
|
||||
`EFFORT_FALLBACK=nearest` 时,请求 `medium` 而模型只有 low/high/max,实发的是
|
||||
`low`;此处重算 `effective_effort` 必然算成请求档,于是整行被挂在一个从未发出
|
||||
过的分组下——而两个值在没开映射的源上恒等,这个错在本地跑不出来。
|
||||
|
||||
失败尝试没有响应,实发档无从得知,故退回请求档并**接受这层含义差别**: 开了映射
|
||||
的源上,成功行是映射后的档、失败行是请求档,两种行不是同一把尺子。仍然记而不是
|
||||
留空,是因为档位错误(`resolve_thinking` 的 Phase 2/4/5)根本没发 HTTP 就被拒,
|
||||
这类行记的正是**被拒绝的那一档**——"哪一档配错了"是压测与排障要的信号。
|
||||
|
||||
回落走 `effective_effort` 而非裸读两个字段: `enable_thinking` 也是一次表态
|
||||
(语法糖),漏掉它就会把一次明确要求推理的调用记成"没表态"。
|
||||
"""
|
||||
if not applies:
|
||||
return None
|
||||
if response is not None:
|
||||
return _normalize_effort(response.applied_effort)
|
||||
return _normalize_effort(
|
||||
effective_effort(
|
||||
request_effort=request.reasoning_effort,
|
||||
source_effort=source.reasoning_effort,
|
||||
enable_thinking=source.enable_thinking,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _cap_text(text: str, cap: int | None) -> str:
|
||||
"""超出 cap 时头部硬切并附省略标记 `…(略 N 字)`;cap 为 None 原样返回。"""
|
||||
if cap is None or len(text) <= cap:
|
||||
@@ -161,7 +226,7 @@ class _AttemptUsage:
|
||||
|
||||
|
||||
class TelemetryEmitter:
|
||||
"""从请求与结果组装 25 字段并写入 recorder;一切写失败降级 warning。"""
|
||||
"""从请求与结果组装 26 字段并写入 recorder;一切写失败降级 warning。"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
@@ -192,8 +257,17 @@ class TelemetryEmitter:
|
||||
latency_ms: int,
|
||||
response: LLMResponse | None,
|
||||
error: str | None,
|
||||
reasoning_applies: bool,
|
||||
) -> None:
|
||||
"""逐次尝试记录(RetryMW 调用);失败尝试无用量可言,记 0 并标 unavailable。"""
|
||||
"""逐次尝试记录(三个 Client 的重试层调用);失败尝试无用量可言,记 0 并标 unavailable。
|
||||
|
||||
`reasoning_applies` 声明**这条调用路径有没有推理语义**: chat 路径为
|
||||
`True`,embedding / OCR 路径为 `False`。它不能由 emitter 自己推断——三条路径
|
||||
共用同一个 `SourceConfig` 类型,一个误配了 `ENABLE_THINKING` 的 embedding 源
|
||||
会让下面的回落算出 `auto`,给一次从来不带推理参数的调用挂上一个从未发出过的
|
||||
档。**不设默认值**: 与 `TelemetryRecorder` 同一约定,库外无第三方调用者,漏传
|
||||
当场 TypeError,好过被静默当成"没表态"。
|
||||
"""
|
||||
usage = _AttemptUsage.of(response)
|
||||
await self._record(
|
||||
request=request,
|
||||
@@ -219,6 +293,9 @@ class TelemetryEmitter:
|
||||
sampling=canonical_sampling_json(merge_sampling(source.extra_body, request.sampling)),
|
||||
tenant_id=request.tenant_id,
|
||||
meta=request.meta,
|
||||
reasoning_effort=_attempt_effort(
|
||||
request=request, source=source, response=response, applies=reasoning_applies
|
||||
),
|
||||
)
|
||||
|
||||
async def emit_cache_hit(self, *, request: ChatRequest, response: LLMResponse) -> None:
|
||||
@@ -254,6 +331,9 @@ class TelemetryEmitter:
|
||||
# 记到上一个租户头上,两边的账同时错且无任何报错(issue #11 设计 §4.3)
|
||||
tenant_id=request.tenant_id,
|
||||
meta=request.meta,
|
||||
# 与 sampling 同一口径: 命中行没有选中源,源级档位与 `nearest` 映射
|
||||
# 都无从谈起,只记调用方这次要的档(response 里那个是历史那次实发的)
|
||||
reasoning_effort=_normalize_effort(request.reasoning_effort),
|
||||
)
|
||||
|
||||
async def emit_terminal_failure(
|
||||
@@ -286,6 +366,8 @@ class TelemetryEmitter:
|
||||
# 源不可知,但租户归属是已知的——终态失败行恰是审计最需要的
|
||||
tenant_id=request.tenant_id,
|
||||
meta=request.meta,
|
||||
# 可能根本没选出源,故与 sampling 同样只取请求档
|
||||
reasoning_effort=_normalize_effort(request.reasoning_effort),
|
||||
)
|
||||
|
||||
async def _record(
|
||||
@@ -317,6 +399,10 @@ class TelemetryEmitter:
|
||||
# issue #11: 未归一化的调用方维度,归一化在本方法内收口(recorder 只落库)
|
||||
tenant_id: str | None,
|
||||
meta: Mapping[str, Any],
|
||||
# issue #20: 已由各入口按自己的口径定型成裸 str/None(口径差别见三个入口的
|
||||
# 注释),本方法只搬运——把定型放这里就得再传一遍 response/source,等于把
|
||||
# "唯一 record_llm_call 调用点"换成"两处口径判断",那正是要避免的复制
|
||||
reasoning_effort: str | None,
|
||||
) -> None:
|
||||
try:
|
||||
# 成本换算(M2 §6): 成功行按单价换算;缓存命中 0.0(未产生新调用);
|
||||
@@ -370,6 +456,7 @@ class TelemetryEmitter:
|
||||
# 保证接受,而遥测写失败只降级成一条 warning——不会当场炸,只会让
|
||||
# Postgres 那一路悄悄少一列数据
|
||||
thinking_observation=_normalize_observation(thinking_observation),
|
||||
reasoning_effort=reasoning_effort,
|
||||
)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
|
||||
Reference in New Issue
Block a user