feat: make telemetry degradation a first-class state
Telemetry degradation used to be a single warning and a private boolean. In a long-running process that is indistinguishable from telemetry working: issue #15 was only found by hand-reconciling milestone log lines against llm_calls rows, after 19 calls had silently gone unrecorded. The SQLite side was worse — once init failed, every write returned without even a log line. Degradation now has one shared owner. TelemetryStatusTracker holds the state machine (enter/recover/drop/should-retry), announces entry and recovery once each, and repeats the drop count under a row-and-time double threshold so a degraded backend neither floods the log nor goes quiet. Both recorders hold one; both count the rows they drop. For programmatic consumers, TelemetryStatus is a frozen snapshot exposed as telemetry_status on all three clients, resolved through a single isinstance check. It is a separate optional port rather than a member of TelemetryRecorder: that protocol is @runtime_checkable, so adding an attribute would make every implementation that only defines record_llm_call stop satisfying it — downstream isinstance assertions would break on upgrade. The existing assertion in test_ports.py is what keeps that decision honest. Failure criteria are deliberately untouched here: Postgres still treats a pool failure as permanent, only now visibly. `_failed` and the tracker therefore both carry the verdict for the span of this one change; the cooldown rework collapses them into the tracker alone.
This commit is contained in:
@@ -16,9 +16,10 @@ from polygateway.ports import (
|
||||
SourceSelector,
|
||||
StructuredOutputStrategy,
|
||||
TelemetryRecorder,
|
||||
TelemetryStatusProvider,
|
||||
Transport,
|
||||
)
|
||||
from polygateway.types import LLMResponse, SourceStats
|
||||
from polygateway.types import LLMResponse, SourceStats, TelemetryStatus
|
||||
|
||||
|
||||
def _resp() -> LLMResponse:
|
||||
@@ -141,6 +142,28 @@ def test_protocols_are_runtime_checkable(impl, protocol):
|
||||
assert isinstance(impl, protocol)
|
||||
|
||||
|
||||
class _DummyStatusProvider(_DummyRecorder):
|
||||
@property
|
||||
def telemetry_status(self) -> TelemetryStatus:
|
||||
return TelemetryStatus(
|
||||
degraded=False,
|
||||
fatal=False,
|
||||
reason=None,
|
||||
degraded_for_s=None,
|
||||
dropped_rows=0,
|
||||
retry_after_s=None,
|
||||
)
|
||||
|
||||
|
||||
def test_status_provider_is_a_separate_optional_port():
|
||||
"""状态**不得**并进 TelemetryRecorder: 那会让只实现 record_llm_call 的对象
|
||||
当场不再满足 @runtime_checkable 的结构检查(设计 §3.3,Codex 审查)。"""
|
||||
assert isinstance(_DummyStatusProvider(), TelemetryStatusProvider)
|
||||
assert isinstance(_DummyStatusProvider(), TelemetryRecorder)
|
||||
assert not isinstance(_DummyRecorder(), TelemetryStatusProvider)
|
||||
assert isinstance(_DummyRecorder(), TelemetryRecorder) # 这条断言是那条决策的执法点
|
||||
|
||||
|
||||
def _decision(**overrides) -> GateDecision:
|
||||
base = {
|
||||
"source_name": "qwen_1",
|
||||
|
||||
Reference in New Issue
Block a user