fix: judge telemetry failures by nature, not by step
The pool exhaustion in issue #15 was fatal only because min_size=10 forced a transient error to surface at pool creation, and that step was hardcoded to permanent death. Step is the wrong axis: it conflates "the DSN cannot be parsed" with "someone else holds all the connections right now". Failures are now classified by two rules. Fatal means the cause lies entirely inside this process and cannot change, which only the construction-time DSN satisfies. Everything else splits on whether the failure has anything to do with this row's data: row-level failures drop one row and keep trying, environment-level failures cool down for 60s and then get exactly one retry, so a restarted database or a DBA creating the table heals on its own. 42703 (missing column) is the single named exception and stays row-level even though every row fails alike: issue #13 promised that the manual mode trims the INSERT and exposes drift per row, and that promise outranks the rule. Any future exception owes the same argument. The _failed boolean is gone; the tracker is the only degradation state, because two copies of the same fact drift apart. Closing stays outside that state: it is the caller's own decision, not an anomaly to recover from, so the snapshot reports it through dropped_rows and the drop reason instead of raising the degraded flag on every clean shutdown.
This commit is contained in:
@@ -413,13 +413,13 @@ class TestLeastPrivilegeDeployment:
|
||||
await conn.close()
|
||||
|
||||
async def test_records_land_without_schema_create_privilege(self, least_privilege_dsn):
|
||||
"""修复前: 建表被拒 → _failed → 整个进程一条不落(下游 150 次调用全丢)。"""
|
||||
"""修复前: 建表被拒 → 整体判死 → 整个进程一条不落(下游 150 次调用全丢)。"""
|
||||
low_dsn, schema = least_privilege_dsn
|
||||
recorder = _recorder(low_dsn, auto_migrate=True)
|
||||
try:
|
||||
await _record_minimal(recorder, call_id=_cid("lp1"))
|
||||
await _record_minimal(recorder, call_id=_cid("lp2"), cost=1.5)
|
||||
assert recorder._failed is False # 判死开关不得被建表权限触发
|
||||
assert recorder.telemetry_status.degraded is False # 建表权限不得触发降级
|
||||
rows = await _fetch(
|
||||
low_dsn,
|
||||
"SELECT call_id, cost FROM llm_calls WHERE call_id LIKE $1 ORDER BY call_id",
|
||||
@@ -665,15 +665,15 @@ class TestCallerDimensionsAcceptance:
|
||||
async def test_backfill_failure_degrades_per_row_not_wholesale(
|
||||
self, least_privilege_pre_tenant_dsn, captured_warnings
|
||||
):
|
||||
"""补列失败的降级方向: 记 warning、不置 `_failed`、后续 INSERT 仍照发。
|
||||
"""补列失败的降级方向: 记 warning、不整体降级、后续 INSERT 仍照发。
|
||||
|
||||
置 `_failed` 会让整个进程从此一条遥测都不写(比逐行丢弃严重得多),
|
||||
且一旦 DBA 补上列也不会自愈——必须等重启。
|
||||
整体降级会让整个进程停写(比逐行丢弃严重得多),而缺列(SQLSTATE 42703)
|
||||
是判据的唯一具名例外: 必须逐行暴露,好让下游看见 schema 漂移(issue #13)。
|
||||
"""
|
||||
recorder = _recorder(least_privilege_pre_tenant_dsn, auto_migrate=True)
|
||||
try:
|
||||
await _record_minimal(recorder, call_id=_cid("lpp1")) # 不得抛
|
||||
assert recorder._failed is False
|
||||
assert recorder.telemetry_status.degraded is False
|
||||
assert any("补列失败" in m for m in captured_warnings)
|
||||
# 缺列的表上 INSERT 必然失败;逐行 warning 正是"INSERT 照发了"的证据
|
||||
assert any("写入失败" in m for m in captured_warnings)
|
||||
@@ -867,7 +867,7 @@ class TestManualSchemaModeAcceptance:
|
||||
|
||||
assert [m for m in captured_warnings if "补列失败" in m] == []
|
||||
assert [m for m in captured_warnings if "写入失败" in m] == []
|
||||
assert recorder._failed is False
|
||||
assert recorder.telemetry_status.degraded is False
|
||||
notices = [m for m in captured_warnings if "auto_migrate=False" in m]
|
||||
assert len(notices) == 1 # 准备期一次,第二行不再重复
|
||||
assert "以下维度不会被记录: tenant_id, meta" in notices[0]
|
||||
|
||||
Reference in New Issue
Block a user