feat: cap telemetry bodies at a configurable length

Chat rows stored full message and response text with no upper bound, so
downstream contracts and tenders lived in llm_calls indefinitely. Add
_cap_text/_cap_messages in the single telemetry exit (_record), applied
after digest_messages and before json.dumps, plus to response/thinking.

Capping is per text, not over the serialized JSON: cutting the whole
string would emit invalid JSON into an unvalidated TEXT column. The cap
builds new dicts and never mutates in place — digest_messages passes
non-list content straight through as the same object, so an in-place cut
would silently poison the caller's messages and the cache key.

text_cap is required on TelemetryEmitter (internal class, three known
construction sites) and defaults to None on the three public clients, so
the default behaviour stays byte-for-byte identical. Settings wiring
lands separately.
This commit is contained in:
2026-08-19 13:39:17 -04:00
parent e0a33ecf93
commit 33ed7ecdfc
9 changed files with 352 additions and 51 deletions
+50 -1
View File
@@ -9,7 +9,8 @@ import pytest
from polygateway.backends.memory.cache import InMemoryCache
from polygateway.errors import ResultInvalidError, TransientError
from polygateway.middleware.cache import CacheMW, build_cache_key, digest_messages
from polygateway.types import ChatRequest, LLMResponse
from polygateway.middleware.telemetry import TelemetryEmitter
from polygateway.types import ChatRequest, LLMResponse, SourceConfig
_MSGS = [{"role": "user", "content": "hi"}]
@@ -327,3 +328,51 @@ class TestStructuredRehydration:
key = build_cache_key("m", _MSGS, "proj", None)
raw = await backend.get(key)
assert raw is not None and "structured_data" not in json.loads(raw)
class TestTelemetryCapDoesNotPoisonTheCacheKey:
"""红线之一(issue #12): 遥测截断绝不能改到缓存 key。
`digest_messages` 对 content 非 list 的消息**原样透传同一个 dict 对象**
(本文件上方公式测试依赖的也是这份对象),遥测拿到的与算 key 用的是同一份。
就地截断会让同一组 messages 在遥测前后算出两个不同的 key——全量 miss、
且没有任何报错。故这里测的是"截断没有就地改掉调用方的对象",不只是
"截断函数是纯的"
"""
class _Rows:
def __init__(self):
self.rows = []
async def record_llm_call(self, **fields):
self.rows.append(fields)
async def test_key_is_byte_identical_across_a_capped_emit(self):
messages = [
{"role": "user", "content": "合同正文" * 31},
{"role": "user", "content": [{"type": "text", "text": "标书正文" * 30}]},
]
before = build_cache_key("m", messages, "proj", None)
rec = self._Rows()
await TelemetryEmitter(rec, text_cap=8).emit_attempt(
request=ChatRequest(messages=messages),
source=SourceConfig(
name="s1",
provider="p",
base_url="https://gw.example/v1",
api_key="sk",
model="m",
timeout_s=10.0,
),
call_id="c",
latency_ms=1,
response=_resp(),
error=None,
)
# 截断确实发生了(否则本用例恒真)
logged = json.loads(rec.rows[0]["messages"])
assert "(略 116 字)" in logged[0]["content"]
assert "(略 112 字)" in logged[1]["content"][0]["text"]
assert build_cache_key("m", messages, "proj", None) == before