feat: make telemetry degradation a first-class state
Telemetry degradation used to be a single warning and a private boolean. In a long-running process that is indistinguishable from telemetry working: issue #15 was only found by hand-reconciling milestone log lines against llm_calls rows, after 19 calls had silently gone unrecorded. The SQLite side was worse — once init failed, every write returned without even a log line. Degradation now has one shared owner. TelemetryStatusTracker holds the state machine (enter/recover/drop/should-retry), announces entry and recovery once each, and repeats the drop count under a row-and-time double threshold so a degraded backend neither floods the log nor goes quiet. Both recorders hold one; both count the rows they drop. For programmatic consumers, TelemetryStatus is a frozen snapshot exposed as telemetry_status on all three clients, resolved through a single isinstance check. It is a separate optional port rather than a member of TelemetryRecorder: that protocol is @runtime_checkable, so adding an attribute would make every implementation that only defines record_llm_call stop satisfying it — downstream isinstance assertions would break on upgrade. The existing assertion in test_ports.py is what keeps that decision honest. Failure criteria are deliberately untouched here: Postgres still treats a pool failure as permanent, only now visibly. `_failed` and the tracker therefore both carry the verdict for the span of this one change; the cooldown rework collapses them into the tracker alone.
This commit is contained in:
@@ -25,7 +25,7 @@ from typing import TYPE_CHECKING, Any
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from polygateway.client import _aclose_component
|
||||
from polygateway.client import _aclose_component, _telemetry_status_of
|
||||
from polygateway.config import EmbeddingSettings
|
||||
from polygateway.errors import (
|
||||
AllSourcesExhausted,
|
||||
@@ -46,6 +46,7 @@ from polygateway.types import (
|
||||
ChatRequest,
|
||||
EmbeddingResponse,
|
||||
LLMResponse,
|
||||
TelemetryStatus,
|
||||
strip_unsupported_extra_body,
|
||||
validate_caller_dimensions,
|
||||
)
|
||||
@@ -454,6 +455,15 @@ class EmbeddingClient:
|
||||
known = [c for c in costs if c is not None]
|
||||
return sum(known) if known else None
|
||||
|
||||
@property
|
||||
def telemetry_status(self) -> TelemetryStatus | None:
|
||||
"""遥测后端的可写状态;无遥测或注入的 recorder 不提供状态时为 None。
|
||||
|
||||
判定收敛在 `_telemetry_status_of` 一处(不是三处各自探测): 三个 client
|
||||
的 `aclose` 曾各持一份逐字复制,漂移的结果就是越权关闭(设计 §3.3/§3.4)。
|
||||
"""
|
||||
return _telemetry_status_of(self._telemetry)
|
||||
|
||||
async def aclose(self) -> None:
|
||||
"""幂等释放**自建**资源(与 GatewayClient 对称);注入的组件一律不碰。"""
|
||||
if self._closed:
|
||||
|
||||
Reference in New Issue
Block a user